Nutrition Monitoring: Establishing a Model Program (Chapter Report,
07/19/95, GAO/PEMD-95-19).

GAO was asked to identify features of a model nutrition monitoring
system and examine approaches to incorporating those features in the
National Nutrition Monitoring and Related Research Program, which
consists of a combination of national surveys, federally-supported
surveillance systems operated by the states, and other research and data
collection activities. GAO identified four features of a model program.
It would have a coordinated set of activities, provide data
continuously, generate reliable inferences about important
subpopulations and small geographic areas, and support state and local
monitoring activities.

--------------------------- Indexing Terms -----------------------------

 REPORTNUM:  PEMD-95-19
     TITLE:  Nutrition Monitoring: Establishing a Model Program
      DATE:  07/19/95
   SUBJECT:  Nutrition surveys
             Nutrition research
             Monitoring
             Food supply
             Health research programs
             Interagency relations
             Data integrity
             Demographic data
             Data collection operations
IDENTIFIER:  National Nutrition Monitoring and Related Research Program
             HNIS Nationwide Food Consumption Survey
             USDA Continuing Survey of Food Intakes by Individuals
             USDA Thrifty Food Plan
             PHS Pediatric Nutrition Surveillance System
             PHS Pregnancy Nutrition Surveillance System
             Special Supplemental Food Program for Women, Infants, and 
             Children
             FDA Food Label and Package Survey
             HHS Healthy People 2000 Program
             High Performance Computing and Communications Program
             Census Bureau Current Population Survey
             WIC
             
******************************************************************
** This file contains an ASCII representation of the text of a  **
** GAO report.  Delineations within the text indicating chapter **
** titles, headings, and bullets are preserved.  Major          **
** divisions and subdivisions of the text, such as Chapters,    **
** Sections, and Appendixes, are identified by double and       **
** single lines.  The numbers on the right end of these lines   **
** indicate the position of each of the subsections in the      **
** document outline.  These numbers do NOT correspond with the  **
** page numbers of the printed product.                         **
**                                                              **
** No attempt has been made to display graphic images, although **
** figure captions are reproduced.  Tables are included, but    **
** may not resemble those in the printed version.               **
**                                                              **
** Please see the PDF (Portable Document Format) file, when     **
** available, for a complete electronic file of the printed     **
** document's contents.                                         **
**                                                              **
** A printed copy of this report may be obtained from the GAO   **
** Document Distribution Center.  For further details, please   **
** send an e-mail message to:                                   **
**                                                              **
**                                            **
**                                                              **
** with the message 'info' in the body.                         **
******************************************************************


Cover
================================================================ COVER


Report to the Ranking Minority Member, Committee on Science, House of
Representatives

July 1995

NUTRITION MONITORING -
ESTABLISHING A MODEL PROGRAM

GAO/PEMD-95-19

Features of a Model Nutrition Monitoring Program

(973397)


Abbreviations
=============================================================== ABBREV

  ARS - Agricultural Research Service
  BRFSS - Behavioral Risk Factor Surveillance Survey
  CPS - Current Population Survey
  CSFII - Continuing Survey of Food Intake by Individuals
  DHKS - Diet and Health Knowledge Survey
  HHANES - Hispanic Health and Nutrition Examination Survey
  HHS - Department of Health and Human Services
  HPCCP - High Performance Computing and Communications Program
  NCST - National Council on Science and Technology
  NFCS - Nationwide Food Consumption Survey
  NHANES - National Health and Nutrition Examination Survey
  NNMRRP - National Nutrition Monitoring and Related Research Program
  OMB - Office of Management and Budget
  ONDCP - Office of National Drug Control Policy
  OSTP - Office of Science and Technology Policy
  PATCH - Planned Approach to Community Health
  PedNSS - Pediatric Nutrition Surveillance System
  PHS - Public Health Service
  PNSS - Pregnancy Nutrition Surveillance System
  USDA - U.S.  Department of Agriculture
  WIC - Special Supplemental Food Program for Women, Infants, and
     Children

Letter
=============================================================== LETTER


B-261325

July 19, 1995

The Honorable George E.  Brown, Jr.
Ranking Minority Member
Committee on Science
House of Representatives

Dear Mr.  Brown: 

This report responds to your request that we identify features of a
model nutrition monitoring system and examine approaches to
incorporating those features in the National Nutrition Monitoring and
Related Research Program (NNMRRP).  It builds on two earlier reports. 
The first, Nutrition Monitoring:  Progress in Developing a
Coordinated Program (GAO/PEMD-94-23), assessed the NNMRRP's planning
activities.  The second, Nutrition Monitoring:  Data Serve Many
Purposes; Users Recommend Improvements (GAO/PEMD-95-15), surveyed
users of nutrition monitoring data about how the data are used and
what changes are needed to the data collection activities.  This
report describes features of a model nutrition monitoring system and
presents the strengths and limitations of various strategies for
achieving four of the model features. 

As arranged with your office, we will be sending copies of this
report to the Director of the Office of Management and Budget, the
Interagency Board for Nutrition Monitoring and Related Research, the
agencies responsible for data collection, and interested
congressional committees.  We will also make copies available to
others upon request. 

The major contributors to this report are listed in appendix V.  If
you have any questions or would like additional information, please
call me at (202) 512-3092. 

Sincerely yours,

Kwai-Cheung Chan
Director of Program Evaluation
 in Physical Systems Areas


EXECUTIVE SUMMARY
============================================================ Chapter 0


   PURPOSE
---------------------------------------------------------- Chapter 0:1

Nutrition-related data are used to monitor food safety, study the
relationship between diet and health, inform agricultural policies,
evaluate food assistance programs, and help food industries develop
new products.  However, past evaluations have criticized federal
nutrition monitoring activities as poorly coordinated and
insufficiently responsive to the needs of data users.  At the request
of the former House Committee on Science, Space, and Technology, GAO
defined the features of a model monitoring program and compared
current and potential approaches to responding to the needs of data
users. 


   BACKGROUND
---------------------------------------------------------- Chapter 0:2

The current National Nutrition Monitoring and Related Research
Program (NNMRRP) consists of a combination of national surveys,
federally- supported surveillance systems operated by the states, and
other research and data collection activities.  The 1990 National
Nutrition Monitoring and Related Research Act (P.L.  101-445)
established an Interagency Board that gave the Departments of
Agriculture (USDA) and Health and Human Services (HHS) primary
responsibility for developing and implementing a 10-year
comprehensive plan for federal nutrition monitoring.\1

In reviewing the Interagency Board's progress in coordinating the
program, an earlier GAO report found that coordination between USDA
and HHS had improved, but the 10-year plan was still inadequate.  It
neither established priorities nor provided a framework for
evaluating existing or potential monitoring activities.\2 Drawing on
a survey of users of nutrition monitoring data, a second GAO report
found that the data collected by the NNMRRP support an extensive
range of purposes, but changes could increase the utility and
credibility of the data.\3 For this report, GAO first defined the
features of a model program by reviewing past evaluations of federal
nutrition monitoring and the 10-year plan, consulting experts, and
surveying users of nutrition data.  Then, with the assistance of
expert advisers and studies of illustrative programs, GAO identified
and compared current and potential approaches to incorporating model
features in the NNMRRP.  (See chapter 1.)


--------------------
\1 Ten-Year Comprehensive Plan for the National Nutrition Monitoring
and Related Research Program, 58 Fed.  Reg.  111 (June 11, 1993), pp. 
32752-806. 

\2 See Nutrition Monitoring:  Progress in Developing a Coordinated
Program (GAO/PEMD-94-23; May 27, 1994). 

\3 See Nutrition Monitoring:  Data Serve Many Purposes; Users
Recommend Improvements (GAO/PEMD-95-15; June 20, 1995). 


   RESULTS IN BRIEF
---------------------------------------------------------- Chapter 0:3

GAO identified four features of a model program:  It would have a
coordinated set of activities, provide data continuously, generate
reliable inferences about important subpopulations and small
geographic areas, and support state and local monitoring activities. 
While the NNMRRP already has some elements of a model program, other
strategies may lead to improved nutrition monitoring capabilities. 
Alternate approaches to coordination, such as an independent central
office with authority over the NNMRRP agencies or a single lead
agency, may not provide any clear advantages to the current
structure, which relies on the Interagency Board.  However, such
approaches do offer useful tools that could be adopted by the Board,
such as the use of formal budget reviews and the establishment of a
central contact for data users.  The current source of continuous
data--the state-based surveillance systems--cannot meet the needs for
national-level information or for in-depth dietary intake
information.  Alternatives are to attach modules of nutrition-related
questions to other ongoing surveys or to field a core set of
questions continuously, supplemented periodically by questions of
intermittent or emerging interest.  Current approaches to providing
more information on subpopulation groups and small geographic areas
are to oversample selected groups as part of the national surveys and
to collect data on specific high-risk groups through the surveillance
systems.  These approaches could be complemented by special studies
and indirect estimation.  Finally, to support state and local
monitoring activities, HHS provides a combination of technical and
financial assistance for the state-based surveillance systems.  One
option to provide more relevant data to localities is community-based
data collection. 


   GAO'S ANALYSIS
---------------------------------------------------------- Chapter 0:4


      FEATURES OF A MODEL
      NUTRITION MONITORING PROGRAM
-------------------------------------------------------- Chapter 0:4.1

Coordination of the data collection activities is needed to ensure
that the diverse needs for the data are met efficiently.  For
example, NNMRRP's major national surveys collect complementary data,
with one focusing on health and nutritional status and the others
emphasizing food consumption.  Combining these data would increase
their utility.  (See chapter 2.) Continuous data collection,
"repeated regularly and frequently" (as defined in the 10-year plan),
is needed to track trends in diet-related health risks, such as those
targeted in HHS' year 2000 health objectives.  In addition,
continuous data are important in evaluating the effects of policy
changes, such as replacing current food assistance programs with
block grants. 

The dietary habits and nutritional problems of a subpopulation or
small geographic area may deviate from the national profile.  Data on
these groups are needed to appropriately target programs that address
their nutrition-related problems, such as diabetes in some minority
groups.  Finally, states and localities need relevant nutrition
monitoring data to plan programs that respond to the specific health
and nutrition concerns of their populations.  For example, currently
available data have been used to initiate "5-a-Day" programs that
encourage consumption of fruits and vegetables and to establish
anemia as a priority issue for nutrition education. 


      CURRENT AND POTENTIAL
      APPROACHES TO ACHIEVING A
      MODEL PROGRAM
-------------------------------------------------------- Chapter 0:4.2

While the Interagency Board appears to have improved communication
and cooperation among the agencies, its 10-year plan did not
establish clear priorities for nutrition monitoring across agencies. 
The Board has conducted a priority-setting exercise, but it has made
little progress in developing a framework to guide NNMRRP
decision-making.  The two alternate approaches to coordinating
nutrition monitoring that GAO examined suggest mechanisms for
strengthening the work of the Interagency Board.  Appointing an
NNMRRP administrator, for example, could provide a central contact
point for users of nutrition monitoring data.  In addition, regular
budget reviews could be conducted to ensure that priority objectives
of the programs are funded.  (See chapter 3.)

None of the three major national surveys are implemented
continuously.  Moreover, the surveillance systems, which are the
current source of continuous data, cannot meet the needs of data
users, who depend on national-level data because of limitations in
the amount and kind of data they gather and in the populations they
target.  A continuous national survey could be more efficient in
terms of costs per respondent than the current, intermittent surveys,
especially if a smaller, core set of data elements are collected
continuously and supplemented periodically by topical modules. 
However, activities that are now implemented sequentially would have
to be funded and staffed to operate concurrently.  Another approach
to gathering information more frequently is the inclusion of
nutrition-related questions on other ongoing surveys.  (See chapter
4.)

At present, the state-based surveillance systems and oversampling in
the national surveys provide information on subpopulations and small
geographic areas.  Two surveillance systems focus on low-income
pregnant women and children, but not all states participate and data
are collected only from participants in public health and nutrition
programs.  Oversampling of specific subpopulations as part of a
national survey is most appropriate for groups that are
geographically clustered and that do not require different data
collection procedures than those used for the general population. 
Its advantage is that the data on the group of interest can be
compared to national estimates. 

Two alternate ways of gathering information on subpopulations and
small geographic areas are special studies and indirect estimation
programs.  Special studies are appropriate for populations that are
geographically dispersed or require different data collection
procedures, but they can be costly relative to other approaches. 
Indirect estimates, which use data from other sources to predict the
value for the group or area of interest, are likely to be less costly
than collecting new data for direct estimation.  However, gauging the
bias introduced by the models that produce the indirect estimates is
difficult; thus, they may not be as useful as direct estimates.  (See
chapter 5.)

Through the state surveillance systems, HHS provides assistance in
the collection, analysis, and interpretation of state and local data. 
Although the data collection instruments can be modified from state
to state, the systems are organized to collect standardized
information across the participating states.  As a result, they may
have limited sensitivity to state or local differences.  An alternate
strategy is to design monitoring programs specifically for local
needs.  However, unlike the data collected by the surveillance
systems, community-based data could probably not be aggregated to
describe statewide needs.  (See chapter 6.)


   RECOMMENDATIONS
---------------------------------------------------------- Chapter 0:5

GAO is not making recommendations in this report. 


   AGENCY COMMENTS
---------------------------------------------------------- Chapter 0:6

In their written comments on a draft of this report, officials from
USDA and HHS (1) found the report to be factual, (2) agreed with the
model program features identified by GAO, (3) agreed with the
importance of the model features discussed in the report, and (4)
provided additional detail on the progress that has been made by the
Interagency Board and its working groups toward improving NNMRRP. 
Both USDA's and HHS' comments as well as comments received from three
members of the National Nutrition Monitoring Advisory Council noted
that limited resources affect the NNMRRP's ability to incorporate the
model features.  (See appendixes III and IV for comments from USDA
and HHS and our response.)


THE CURRENT NUTRITION MONITORING
PROGRAM
============================================================ Chapter 1

This chapter first provides background information on the National
Nutrition Monitoring and Related Research Program.  Then, the
objectives, scope, and methodology of our review of current and
potential approaches to achieving a model program are described.  The
chapter concludes with an overview of the organization of the rest of
the report. 


   CORE DATA COLLECTION ACTIVITIES
   OF THE NNMRRP
---------------------------------------------------------- Chapter 1:1

The NNMRRP is a complex system of data collection and research
activities, including national surveys, state surveillance
activities, and a variety of research programs.  Over time, the
NNMRRP has developed activities focused on five content areas:  (1)
food and nutrient consumption; (2) nutritional and health status; (3)
dietary knowledge, attitudes, and behavior; (4) food composition; and
(5) food supply.  As shown in table 1.1, the information produced by
these activities is used for a variety of purposes, from supporting
basic research on human nutritional needs to informing policy
decisions about health, agriculture, and food programs. 



                                    Table 1.1
                     
                     Selected Uses of NNMRRP Data by Content
                                       Area

Content area               Selected uses\a
-------------------------  -----------------------------------------------------
Food and nutrient          Assess potential exposure of children to pesticide
consumption                residues

                           Develop standards for food stamp eligibility

                           Evaluate impact of farm commodity programs on demand
                           and pricing

                           Monitor trends in diet-related risks of chronic
                           diseases

Nutritional and health     Evaluate food assistance programs
status

                           Research the relationship between fat consumption and
                           cancer

                           Target food fortification to foods eaten by people
                           with nutrient deficiencies

                           Update infant growth charts

                           Inform policies on cholesterol screening and
                           treatment

Dietary knowledge,         Develop food labeling policies
attitudes, and behavior

                           Inform nutrition education programs

Food composition           Support estimation of the consumption of nutrients
                           and food components related to chronic disease

                           Plan meals for military personnel and other groups

                           Monitor trends in nutrient availability and food
                           safety

Food supply                Manage federal marketing and agricultural policies
--------------------------------------------------------------------------------
\a These uses were selected to illustrate the variety of purposes
supported by NNMRRP data.  They are not an exhaustive list. 

Table 1.2 lists specific activities managed by the Departments of
Agriculture and Health and Human Services, which have major
responsibilities for the five areas.  Other agencies, including
Commerce, Defense, and the Environmental Protection Agency, also
participate in the NNMRRP. 



                         Table 1.2
          
          Major NNMRRP Activities by USDA and HHS


Content area        USDA                HHS
------------------  ------------------  ------------------
Food and nutrient   Nationwide Food     National Health
consumption         Consumption         and Nutrition
                    Survey              Examination
                                        Survey
                    Continuing Survey
                    of Food Intake by   Total Diet Study
                    Individuals

Nutritional and     No major role       National Health
health status                           and Nutrition
                                        Examination
                                        Survey

                                        Pregnancy
                                        Nutrition
                                        Surveillance
                                        System

                                        Pediatric
                                        Nutrition
                                        Surveillance
                                        System

Dietary knowledge,  Diet and Health     Health and Diet
attitudes, and      Knowledge Survey    Survey
behavior
                                        Behavioral Risk
                                        Factor
                                        Surveillance
                                        System

Food composition    National Nutrient   Food Label and
                    Data Bank           Package Survey

Food supply         Food and Nutrition  No major role
                    Supply Series
----------------------------------------------------------

      USDA ACTIVITIES
-------------------------------------------------------- Chapter 1:1.1

USDA has major responsibilities for collecting information in all of
the content areas except nutritional and health status.  USDA gathers
data on food and nutrient consumption through two national
surveys--the Nationwide Food Consumption Survey (NFCS) and the
Continuing Survey of Food Intake by Individuals (CSFII).  In the
past, NFCS has gathered nationally-representative information on the
food consumption behavior of households and individuals.  It provides
detailed data on household costs for food.  One of the major uses for
these data is the development of the Thrifty Food Plan, which is the
basis for calculating food stamp benefits.  Implemented decennially,
NFCS suffered from severe response rate problems (less than 40
percent) in 1987-88.  As a result, the individual food consumption
portion of the NFCS is expected to be dropped in the future. 

Since the mid-1980s, CSFII has supplemented NFCS by providing regular
information on individual dietary intake.  The data collected in its
most recent implementation (1994-96) will be used to describe both
general and low-income populations.  The Diet and Health Knowledge
Survey (DHKS), which collects data on dietary knowledge, attitudes,
and behavior, is a follow-up to CSFII.  Together, CSFII and DHKS are
intended to inform policies relating to food production and
marketing, food safety, food assistance, and nutrition education. 

The USDA activities focused on food composition and food supply are
not surveys.  For food composition, USDA gathers data from food
industries and other sources on the nutrient content of foods.  These
data support the dietary surveys by translating the foods consumed
into their nutrient components.  Food supply is estimated by
deducting data on exports, year-end inventories, and nonfood use from
data on production, imports, and beginning inventories. 


      HHS ACTIVITIES
-------------------------------------------------------- Chapter 1:1.2

HHS is responsible for the National Health and Nutrition Examination
Survey (NHANES) and the state-based surveillance systems.  These data
collection activities provide information on the content areas (1)
food and nutrient consumption, (2) nutritional and health status, and
(3) dietary knowledge, attitudes, and behavior.  Like NFCS and CSFII,
NHANES collects data from a nationally-representative sample. 
However, NHANES' unique contribution is its use of physical
examinations and clinical and laboratory tests as well as traditional
survey methods to gather information.  NHANES' data support research
on the relationship between diet and health and inform health policy
decisions, such as the promotion of cholesterol screening.  After two
earlier implementations in the 1970s, NHANES has just completed its
third administration (1988-94).  NHANES has been supplemented by the
Hispanic Health and Nutrition Examination Survey (HHANES) in the
early 1980s, follow-up surveys of respondents to NHANES I, and
follow-up matches of the records from NHANES II to the National Death
Index and other vital statistics records. 

The state-based surveillance systems were set up to provide quick
information to states to use in planning and managing nutrition and
health programs.  They include the Pediatric Nutrition Surveillance
System (PedNSS), Pregnancy Nutrition Surveillance System (PNSS), and
Behavioral Risk Factor Surveillance System (BRFSS).  Participating
states collect the data for these systems with technical and other
kinds of assistance from HHS.  Both PedNSS and PNSS rely on data from
clinic records from publicly funded health, nutrition, and food
assistance programs, primarily the Special Supplemental Food Program
for Women, Infants, and Children (WIC).  PedNSS monitors nutritional
status among low-income, high-risk children, while PNSS focuses on
low-income, high-risk pregnant women, measuring nutrition-related
problems and behavioral risk factors associated with low birthweight. 

In contrast to PedNSS and PNSS, data for BRFSS are gathered through
telephone interviews, with respondents (adults 18 years and over)
sampled through random digit dialing.  In addition to a core set of
questions on various health risk factors, BRFSS includes optional
modules for the assessment of dietary fat and fruit and vegetable
consumption. 

Other major HHS monitoring activities include the Total Diet Study,
which analyzes nutrient and contaminant levels in the food supply;
the Food Label and Package Survey, which monitors nutrition labeling
practices; and the Health and Diet Survey, which assesses dietary
knowledge and practices as they relate to health problems. 


   STRUCTURE OF THE NNMRRP
---------------------------------------------------------- Chapter 1:2

Although the United States has one of the most comprehensive
monitoring programs in the world, several problems with nutrition
monitoring activities have been identified over the past two decades. 
Of key concern has been the lack of coordination and compatibility of
different data collection activities.  This encompasses differences
across surveys in methods for assessing dietary intake and
nutritional status, sampling designs, population descriptors and
other measures, and the timing and reporting of results. 

To improve the coordination of federal nutrition monitoring
activities and the quality of the data collected, the Congress passed
the National Nutrition Monitoring and Related Research Act of 1990
(P.L.  101-445).  The act established an Interagency Board, jointly
chaired by USDA and HHS, to coordinate activities across the various
agencies involved in nutrition monitoring.  The Interagency Board was
charged with developing a strategic plan that would establish a
comprehensive nutrition monitoring and related research program. 
This plan--known as the 10-year comprehensive plan--was published in
the Federal Register on June 11, 1993.  It outlines a set of planning
activities, including a general time frame and lead agencies for each
activity.  The activities are organized around six objectives, which
are to

  -- provide for a comprehensive NNMRRP through continuous and
     coordinated data collection;

  -- improve the comparability and quality of data across the NNMRRP;

  -- improve the research base for nutrition monitoring;

  -- develop and strengthen state and local capacity for continuous
     and coordinated nutrition monitoring data collection that
     complements national nutrition surveys;

  -- improve methodologies to enhance comparability of NNMRRP data
     across federal, state, and local levels; and

  -- improve the quality of state and local nutrition monitoring
     data. 

In addition, an Advisory Council of experts from outside the federal
government was created to guide the Interagency Board on scientific
and technical matters.  (Chapter 3 contains more information on the
Interagency Board and its activities.)


   OBJECTIVES, SCOPE, AND
   METHODOLOGY
---------------------------------------------------------- Chapter 1:3


      OBJECTIVES
-------------------------------------------------------- Chapter 1:3.1

This is the third and final report in a series responding to a
request from the former House Committee on Science, Space, and
Technology.  The first report reviewed past evaluations of federal
nutrition monitoring and examined the progress of the NNMRRP since
the passage of the 1990 act.\1 It concluded that (1) a coherent
program for nutrition monitoring was not yet in place and (2)
although there has been progress in coordinating the program, the
10-year plan is incomplete because it does not include a framework
for evaluating current and potential activities or detailed plans for
achieving the objectives. 

Based on a survey of users of nutrition monitoring data, the second
report described the purposes for which nutrition data are used and
summarized respondents' suggestions for improving NNMRRP
activities.\2 These suggestions, which addressed such issues as the
timing of the surveys, their coverage of subpopulations, and the ease
with which the data could be used, were consistent with the concerns
raised by past evaluations of federal nutrition monitoring and
indicated a continued need to address the long-standing problems of
federal nutrition monitoring. 

Completing our response to the Committee's request, this report
builds on our earlier work to meet two objectives:  (1) define a
model nutrition monitoring program and (2) compare the current system
with potential options for implementing the components of a model
program in the NNMRRP. 


--------------------
\1 Nutrition Monitoring:  Progress in Developing a Coordinated
Program (GAO/PEMD-94-23). 

\2 Nutrition Monitoring:  Data Serve Many Purposes; Users Recommend
Improvements (GAO/PEMD-95-15). 


      SCOPE AND METHODOLOGY
-------------------------------------------------------- Chapter 1:3.2


         DEFINING THE FEATURES OF
         A MODEL PROGRAM
------------------------------------------------------ Chapter 1:3.2.1

Before defining the features of a model program, we first limited our
review to three of the five content areas covered by the NNMRRP:  (1)
food consumption and dietary intake; (2) health and nutritional
status; and (3) knowledge, attitudes, and behavior.  These three
variables were selected for both substantive and methodological
reasons.  Substantively, the selected elements provide the data that
support the planning and evaluation of interventions that directly
affect health, such as nutrition education and food assistance
programs.  A substantive argument could also be made for the
inclusion of food composition data, which are used to translate the
information on what foods are eaten into estimates of nutrient
intake.  However, because an earlier GAO project focused on the
NNMRRP's food composition activities, we did not include this in our
review.\3

Methodologically, the three selected variables are linked because
they rely on data obtained from individuals through surveys and
physical examinations.  In contrast, food composition information is
based on chemical analyses of foods, and food supply is determined
from macroeconomic data.  Our focus on the components of the NNMRRP
that rely on surveys facilitates the comparison of current and
potential approaches to achieving a model program. 

To identify features of a model nutrition monitoring program, we used
four sources:  reviews of previous evaluations of federal nutrition
monitoring activities, review of the objectives and related
activities outlined in the 10-year comprehensive plan developed by
the Interagency Board, consultation with expert advisers, and our
survey of data users.  These sources are detailed below, and
supporting material is provided in appendix II.  (See chapter 2 for a
description of the features.)

Past Evaluations of Federal Nutrition Monitoring.  Past evaluations
by such groups as the National Academy of Public Administrators, the
Joint Nutrition Monitoring Evaluation Committee, and the National
Research Council identified several concerns about the federal
nutrition monitoring program.  Because these evaluations also
informed the Interagency Board's development of its 10-year plan, we
used them as the starting point for our identification of features of
a model nutrition monitoring program.  Our report, Nutrition
Monitoring (GAO/PEMD-94-23), discusses these past evaluations and
NNMRRP progress in addressing their recommendations.  (Table II.1 in
appendix II lists the criticisms identified in these evaluations and
categorizes them by the features of a model program that they
suggest.)

The 10-Year Comprehensive Plan.  As described above, the NNMRR Act
required the Interagency Board to develop a plan for the program. 
The plan outlined six objectives, three with a federal focus and
three emphasizing state and local monitoring, and listed 68 planned
activities.  These were reviewed for responsiveness to the model
features suggested by the past evaluations.  (See table II.2 in
appendix II for examples of the 68 activities listed in the plan.)

Consultation With Expert Advisers.  To assist us at critical decision
points in this project, we organized three panels.  (The members of
each panel are listed in appendix I.) The Core Policy Panel consisted
of nationally-known experts in nutrition and nutrition monitoring
policy.  These panelists were consulted throughout the project.  In
addition, they helped us develop a framework of purposes for
nutrition monitoring data that guided our survey of data users. 

The Methodology Panel included renowned experts in such fields as
sampling, survey design, dietary assessment, and nutritional
epidemiology.  This panel met to help us identify promising
approaches to critical elements of a nutrition monitoring system.  In
addition, the panelists assisted us on issues related to their areas
of specialization. 

The Data Users Panel consisted of users of the nutrition monitoring
data, chosen to reflect the broad range of purposes that the data
must serve, including the support of state and local nutrition
programs, academic research, food industry research, and the
development and evaluation of federal food assistance programs.  As
with the Methodology Panel, this panel was convened once to help us
identify promising approaches to nutrition monitoring.  Individual
panelists were consulted later about specific issues related to their
expertise. 

Through the process of reviewing materials and participating in panel
meetings, the expert advisers generated several suggestions for
possible changes to the NNMRRP, examples of which are given in table
II.3 in appendix II.  In addition, they reviewed a draft of this
report. 

Suggestions From the Survey of Data Users.  Our survey of users of
nutrition monitoring data focused on primary users of 14 of the
NNMRRP surveys and surveillance systems.  Primary users were defined
as those who directly access the data rather than use information
that has already been processed and interpreted by others in reports
and other documents.  Since there is no single list of primary users
of NNMRRP data, we obtained lists of known and potential data users
such as people who had requested the data from NNMRRP agencies,
attendees at nutrition-related workshops, state and local government
officials working in nutrition, and members of associations for
nutrition professionals.\4

A major portion of the survey was dedicated to determining how the
respondents used the data.  We also asked whether changes are needed
to better meet the respondent's information and data quality needs. 
If the respondent indicated a need for change, we asked for
suggestions in the following categories:  (1) data elements
collected, (2) data collection methods, (3) units of analysis, (4)
time of data collection, (5) population group coverage, (6)
geographic area coverage, and (7) ease of use.  These comments were
analyzed to identify major themes for each of three groups of data
collection activities--USDA surveys, HHS surveys, and HHS state-based
surveillance systems.  In appendix II, table II.4 identifies the
themes associated with the features of a model program. 


--------------------
\3 See Food Nutrition:  Better Guidance Needed to Improve Reliability
of USDA's Food Composition Data (GAO/RCED-94-30; Oct.  25, 1993). 

\4 See Nutrition Monitoring (GAO/PEMD-95-15) for details of the
survey. 


         COMPARING APPROACHES TO
         ATTAINING THE MODEL
         FEATURES
------------------------------------------------------ Chapter 1:3.2.2

From the features that were identified, we selected four as the focus
of the second objective--the comparison of current and potential
approaches to achieving the model program.  We focused on features
that reflect long-standing concerns about federal nutrition
monitoring, that encompass other desired characteristics, and that
generate debate about how they should be addressed. 

For each of the selected features, we identified current activities
of the NNMRRP through interviews with staff in NNMRRP agencies,
attendance at meetings of the Interagency Board and the Advisory
Council, and reviews of program documents.  Potential approaches were
identified through literature review, analysis of our survey results,
and consultation with the expert advisers.  We did not identify the
universe of potential approaches.  Instead, our search was focused on
those approaches deemed promising and feasible by our expert
advisers. 

To assess each of the potential approaches, we first identified
programs from the same set of sources that helped us define the
features of a model program--that is, the literature, our survey of
data users, and expert advisers.  Where possible, we limited our
consideration to programs that had some linkage to nutrition
monitoring.  For example, for separate studies of subpopulation
groups, we looked at the experience with the Hispanic Health and
Nutrition Examination Survey in the early 1980s.  To describe the
strengths and weaknesses of the programs illustrating the potential
options, we reviewed program documents and related literature and
interviewed managers and staff. 

This review was conducted between October 1993 and December 1994 in
accordance with generally accepted government auditing standards. 


      STRENGTHS AND LIMITATIONS OF
      OUR METHODOLOGY
-------------------------------------------------------- Chapter 1:3.3

This report describes the results of a systematic examination of
current and potential approaches to selected features of a model
nutrition monitoring program.  The strength of the review is its
reliance on multiple sources of information.  In addition to
surveying users of nutrition monitoring data, consulting with
experts, and reviewing both technical literature and program
documents, we also interviewed officials and program staff in
nutrition monitoring programs and in programs illustrating the
alternatives.  The major limitation of our work is its prospective
nature.  Because we were examining potential changes to the NNMRRP,
hard evidence of the costs or effectiveness of the options was not
available.  Instead, the strengths and weaknesses of the options
relative to the current nutrition monitoring system are supported
primarily by logic and stated in tentative terms.  Given this
limitation, the report makes no recommendations for specific changes
to the NNMRRP. 


   ORGANIZATION OF THE REPORT
---------------------------------------------------------- Chapter 1:4

In response to the first objective of the project, chapter 2
describes the model program and provides information on the selection
of four model features as the focus of the report.  (The sources used
to develop the model are described above.) The second
objective--comparing current and potential approaches to each model
feature--is addressed in chapters 3-6, organized by the four model
features.  Specifically, chapter 3 examines coordination options;
chapter 4 compares alternate approaches to providing continuous data;
chapter 5 discusses different methods of supporting inferences about
subpopulation groups and small geographic areas; and chapter 6
reviews approaches to assisting state and local monitoring
activities.  Appendix I provides additional detail on the expert
advisers to the project, and appendix II describes the sources for
the model features.  Agency comments on a draft of the report are in
appendixes III and IV. 


A MODEL NUTRITION MONITORING
PROGRAM
============================================================ Chapter 2

This chapter responds to the first objective of our review--the
definition of a model nutrition monitoring program.  First, the major
features identified from the sources described in chapter 1 are
outlined.  From this set of features, we selected four as the focus
for our response to the second objective of the review--the
comparison of current and other approaches to achieving the model
characteristics.  The chapter describes our selection process and
details the importance of the four features that are the subject of
the rest of the report. 


   FEATURES OF A MODEL PROGRAM
---------------------------------------------------------- Chapter 2:1

Depending on the purposes that the data serve, the specific elements
of a model nutrition monitoring program change.  For example,
researchers and program managers interested in food safety need
detailed information on dietary intake, including specific brand
names of the foods consumed.  In contrast, a nutrition educator may
place a higher priority on information about dietary knowledge,
attitudes, and behaviors.  However, at a more general level, some
common ideal characteristics can be identified.  Focusing on this
general level, we used the sources described in chapter 1 to identify
a number of features of a model program.  Table 2.1 lists these
features and the sources that support them. 



                               Table 2.1
                
                Features of a Model Nutrition Monitoring
                       Program and Their Sources

Feature                                             Source\a
--------------------------------------------------  ------------------
Coordinated system that responds efficiently        Past evaluations
to the diverse needs of the data users
                                                    10-year plan

                                                    Expert advisers

Continuous or more frequent collection of data      Past evaluations

                                                    10-year plan

                                                    Expert advisers

                                                    Survey of data
                                                    users

Support for reliable inferences about               Past evaluations
subpopulation
groups and small geographic areas                   10-year plan

                                                    Expert advisers

                                                    Survey of data
                                                    users

Assistance to states and localities                 Past evaluations

                                                    10-year plan

                                                    Expert advisers

                                                    Survey of data
                                                    users

Strong research base, including improved methods    Past evaluations
for assessing dietary intake and nutritional
status                                              10-year plan

                                                    Expert advisers

                                                    Survey of data
                                                    users

High response rates and low respondent burden       Past evaluations

                                                    Survey of data
                                                    users

Timely processing and dissemination of survey       Past evaluations
information
                                                    10-year plan

                                                    Survey of data
                                                    users

On-going evaluation of the system's content and     Expert advisers
methods, including a review of the information
needs

Collection of both household and individual data    Expert advisers

                                                    Survey of data
                                                    users

Maintenance of data comparability over time         Expert advisers

Longitudinal data (collection of information on     Expert advisers
the
same individuals or households over time)           Survey of data
                                                    users
----------------------------------------------------------------------
\a For more information on these sources, see the discussion in
chapter 1 and the tables in appendix II. 

All of the features identified are clearly important elements of a
comprehensive nutrition monitoring program.  However, we selected
four characteristics as the focus of our review:  (1) a coordinated
set of activities that (2) provides data on a continuous basis, (3)
supports inferences about important population groups, and (4)
assists state and local monitoring activities.  These features
encompass other desired characteristics, respond to long-standing
concerns about federal nutrition monitoring, and generate debate
about how they can be achieved. 

The first criterion for selection was to focus on the most general
concerns.  With these features, other desirable characteristics can
be considered even though they are not emphasized.  Specifically, a
mechanism for evaluating the NNMRRP's content and methods, including
a review of the information needs of the data users, is described in
chapter 3 as an element of a coordinated program.  The theme of
evaluating options in relation to the needs for data also underlies
the discussion of the alternatives for the other features.  Three
other features--the comparability of data over time, the collection
of longitudinal data, and the timeliness of data release--are related
to the continuous collection of data and, as such, are considered
briefly in chapter 4. 

The four features were also selected because they respond to
long-standing criticisms of federal nutrition monitoring activities. 
Concerns about coordination, the continuity and timeliness of the
data, the availability of information on subpopulations, and the role
of states and localities were raised as early as 1977 by witnesses
before a House subcommittee.\1 In contrast, the concerns with
response rates and the level of the data (individual or household)
can be traced to the problems with the last NFCS (described in
chapter 1), which USDA has taken steps to avoid in the future. 

The selected features are also not easily addressed; that is, there
is debate about the best approach to achieving each feature.  For
example, some expert advisers stated that assistance to states and
localities should focus on data analysis and interpretation, while
others argued for a larger state and local role in data collection. 
Similarly, to provide information on subpopulation groups, national
surveys could be supplemented by such means as the surveillance
systems and oversampling or the national surveys could be abandoned
and their resources dedicated to special studies of specific groups. 
In contrast to the debates about how best to achieve these and the
other selected features, there has been consensus about the kind of
dietary intake methodology that will be used by the NNMRRP surveys
and ongoing research to improve these methods through automation and
other means. 


--------------------
\1 Hearings before the House Subcommittee on Domestic and
International Scientific Planning, Analysis and Cooperation,
Committee on Science and Technology, July 26, 27, and 28, 1977. 


   THE SELECTED FEATURES
---------------------------------------------------------- Chapter 2:2

An ideal federal nutrition monitoring program would have a
coordinated set of activities that provides data on a continuous
basis, covers important population groups, and supports state and
local monitoring activities.  Coordination is the key both to the
efficiency of the system and its responsiveness to the needs of data
users.  A continuous flow of data would ensure that the information
on the nation's nutritional status was up-to-date and would also
enable the tracking of dietary behavior and nutritional status over
time.  Information on population groups that are vulnerable or
growing rapidly is needed to plan, manage, and evaluate programs
intended to prevent or ameliorate nutritional problems.  Because
state and local governments are often the location of such programs,
they need assistance in either interpreting available data or
collecting their own information.  The importance of these features
is further explained in the following sections. 


      A COORDINATED NUTRITION
      MONITORING PROGRAM
-------------------------------------------------------- Chapter 2:2.1

The coordination of nutrition monitoring activities has implications
for both the utility of the information produced and the costs of the
program as a whole.  The utility of the information is constrained
when data from different data collection activities cannot be easily
combined.  For example, research on the relationship between diet and
health could be strengthened if CSFII's data on dietary intake could
be combined with NHANES' data on health and nutritional status. 
However, because of differences in the sampling designs and nutrition
measures, combining the data from the two surveys is difficult and
controversial.  Similarly, poor integration of the data collected by
the state-based surveillance systems with the national surveys
presents a barrier to meaningful comparisons of state and national
populations. 

To the extent that the lack of coordination results in unnecessary
duplication, a fragmented system can also increase the costs of
nutrition monitoring.  For example, the current NNMRRP includes two
surveys focused on dietary knowledge, attitudes, and behaviors--one
operated by USDA and the other by HHS.  The Interagency Board plans
to review these surveys for duplication.  While some overlap can be
useful as a quality check on data from different sources, unnecessary
redundancy in the system uses resources that could be used for
currently unmet data needs--such as the need for information on
specific subpopulations at risk for nutrition-related problems. 

The need for a coordinated system was supported by three of the four
sources used to identify features of a model program.  In addition to
a general concern about the lack of coordination, past evaluations of
federal nutrition monitoring have criticized the incompatibility of
the data gathered by different enterprises.  Specifically, these
evaluations have called for compatible methods of assessing dietary
intake, a core set of standardized measures for the major surveys,
compatible sampling techniques in the national surveys, and
integrated reporting.  Another criticism related to coordination
focused on the absence of a systematic process for determining the
needs for nutrition monitoring data across the different data
collection activities.  Although needs are assessed for individual
activities, no comprehensive assessment of needs in relation to the
total system of activities has taken place. 

Coordination is also a major theme of the 10-year plan.  Four of the
six objectives discussed in the plan focus on coordinating data
collection and improving the comparability of the data.  Of the
specific activities listed for each objective, some of those focused
on coordination are:  coordinating the planning for coverage,
tracking, and reporting of findings from surveys and surveillance
systems; identifying ways to increase comparability within a dietary
method to improve the quality and usefulness of data; and
establishing a mechanism for improved coordination among federal
agencies that collect and use survey information about knowledge,
attitudes, and behavior to assess gaps and duplications in existing
surveys. 

The third source of support for the need for coordination comes from
meetings of the advisory panels.  The panelists noted that the
different agencies involved in nutrition monitoring have different
missions and priorities and, hence, coordination is difficult.  Their
major suggestions for improving coordination were to give
coordination responsibility to a single lead agency, to coordinate
from an interagency body with permanent staff and enforcement
authority, to locate nutrition monitoring in statistical agencies
within the user departments or within a central statistical agency,
to centralize the congressional appropriations process for nutrition
monitoring activities, and to ensure informed review of data
collection plans by qualified staff at the Office of Management and
Budget. 

Our survey queried respondents about changes to specific data
collection activities, rather than the NNMRRP as a whole, so
coordination was not a major theme of the comments provided by the
data users. 


      THE CONTINUOUS COLLECTION OF
      DATA
-------------------------------------------------------- Chapter 2:2.2

The Interagency Board defines continuous data as data collection that
is "repeated regularly and frequently."\2 Two consequences follow
when data are not regularly available.  First, because the kinds of
foods that are available and the eating patterns of the American
people change rapidly, the data become outdated quickly.  Compounded
by delays between data collection and data release, long intervals
between administrations of the surveys diminish the relevance of the
data to the current situation and, hence, their utility in program
planning and management.  For example, if any of the policy changes
currently being considered are implemented-- such as the
consolidation of food assistance programs--up-to-date information at
regular intervals before and after the change will be needed to
monitor any positive or negative effects of such policy changes on
the population. 

A second consequence of pauses in data collection is that potential
efficiencies of a continuous survey operation are lost.  Each
implementation of a national survey requires extensive planning,
including reviews of the needs of data users, development and testing
of data collection procedures, and all the steps involved in
approving a contract.  An ongoing data collection operation could
streamline some of these processes.  In addition, when surveys are
not in the field continuously or even at dependable intervals, they
may attempt to meet as many of the needs of data users as possible
when they are administered.  For example, the low response rates of
the 1987-88 NFCS have been partially attributed to the burden on
survey respondents resulting from its attempt to obtain both
household and individual data with one interview.  In contrast, an
ongoing survey could consist of a core set of questions and rotating
modules of questions that address the needs of specific users. 

The continuous collection of data was a theme in all four of our
sources for the features of a model program.  Past evaluations of
nutrition monitoring have called for the continuous collection and
timely release of nutrition-related data.  Although the 10-year plan
does not list specific activities focused on the continuous
collection of data, two of the plan's objectives indicate the
Interagency Board's concern with the timeliness of the data collected
at the federal and state and local levels.  The expert advisers also
emphasized the need for regularly available data and suggested the
following mechanisms for the ongoing collection of the information: 

  -- continuous national nutrition surveys,

  -- addition of nutrition-related modules to existing surveys,

  -- reliance on program data that are already collected, and

  -- collection of longitudinal data (that is, data collected from
     the same sample over time). 

Finally, responses to our survey of the users of nutrition data not
only stated a desire for continuous data, but also indicated that
data that are collected at regular and frequent intervals serve some
important purposes.  For example, data that are currently available
on a regular basis have been used to measure progress toward the
Healthy People 2000 objectives and to evaluate policies such as the
fortification of infant formula.\3


--------------------
\2 Ten-Year Comprehensive Plan, 58 Fed.  Reg.  32806 (1993). 

\3 The Healthy People 2000 objectives, published in 1990, are targets
for improving the health of the population by the year 2000.  The
targets are phrased in specific terms and with reference to baseline
information so that progress toward the objectives is measurable. 
Nutrition-related objectives address such issues as overweight,
breast-feeding, consumption of fruits and vegetables, iron
deficiency, and others. 


      SUPPORT FOR RELIABLE
      INFERENCES ABOUT
      SUBPOPULATION GROUPS AND
      SMALL GEOGRAPHIC AREAS
-------------------------------------------------------- Chapter 2:2.3

The need for continued and improved information on subpopulation
groups and small geographic areas is supported by several arguments. 
First, information is needed on specific populations known to be at
risk for nutrition-related problems, such as Native Americans or
homeless persons, in order to identify their needs and develop and
target assistance programs.  Second, some subpopulations, including
Hispanics and the elderly, are growing rapidly.  Their dietary
patterns or nutritional needs may be different from the population as
a whole; thus, information about these groups is needed to monitor
their needs and to understand their effect on estimates of the
prevalence of various nutritional problems in the overall population. 
Finally, the samples for the three major NNMRRP surveys are designed
to yield national estimates.  However, much of the planning for
health and nutrition programs is conducted at the state and local
levels.  Hence, states and localities also need information on
nutrition-related indicators for their populations. 

For the reasons given above, past evaluations of federal nutrition
monitoring have criticized the program for not covering specific
population groups and geographic areas.  Although none of the overall
objectives of the 10-year comprehensive plan focus on the need for
information on subpopulation groups, several of the activities do. 
For example, one planned action is to develop and implement a plan
for improved coverage of groups at nutritional risk.  Our advisory
panels also discussed the importance of information on
subpopulations, noting differences in the kinds of foods consumed in
different regions of the country and by different ethnic and racial
groups as well as differences in nutritional needs at different ages. 
Their suggestions for improving the availability of data on
subpopulation groups and small geographic areas included different
sampling strategies for different populations, contracts with states
and localities to gather information on geographically-based
populations, and indirect estimation to support inferences about
subpopulation groups and small geographic areas. 

The availability of data on important subpopulations and states and
localities was one of the changes to the current data collection
activities requested by respondents to our survey of data users.  The
users also emphasized the importance of information on subpopulations
by describing the uses supported by currently available data on
subpopulation groups such as determining dietary needs of the
elderly, assessing differences between blacks and whites in the
effect of obesity on diabetes, informing policies on the
fortification of infant and toddler foods, and targeting a blood
pressure screening program to the Mexican-American population. 


      ASSISTANCE TO STATES AND
      LOCALITIES FOR NUTRITION
      MONITORING
-------------------------------------------------------- Chapter 2:2.4

State and local governments are interested not only in the
applicability of federal nutrition monitoring data to their
jurisdictions, but also in having federal assistance in collecting
and interpreting their own data.  The major justification for the
emphasis on state and local monitoring is the wide range of uses that
states and localities have for nutrition data.  The examples in table
2.2, drawn from responses to our survey of data users, illustrate the
utility of existing NNMRRP data for state and local governments. 
While NNMRRP data collection systems meet some of the state and local
needs for nutrition monitoring data, state and local officials have
called for additional technical assistance in analyzing and
interpreting existing sources of data and for federal support in
collecting their own data. 



                               Table 2.2
                
                  Examples of State and Local Uses of
                            Nutrition Data\a

Purpose                 Specific use
----------------------  ----------------------------------------------
Problem identification  Determine nutritional habits of school-age
                        children

                        Identify which nutrition problems are most
                        prevalent among WIC participants

                        Assess fruit and vegetable consumption and
                        need for intervention

Program planning or     Plan cardiovascular interventions for state
policy-making           residents with diabetes

                        Establish anemia as a priority issue for
                        nutrition education

                        Develop a program targeted toward the
                        prevention of obesity in children

                        Seek iron fortification of flour used in
                        ethnic and large county bakeries

                        Launch a "5-a-Day" program to encourage
                        consumption of fruits and vegetables

                        Select ethnic target groups for Preventive
                        Health Block Grants

                        Map anemia rates to identify highest risk
                        areas for targeting resources

Program evaluation or   Monitor progress toward Healthy People 2000
program management      objectives, such as improvement in rates of
                        breast-feeding

                        Assess access to food assistance

                        Justify procurement of hematological equipment

                        Expand nutrition technical assistance to
                        county health departments
----------------------------------------------------------------------
\a Provided by state and local officials who responded to our survey. 

In addition to the support provided by the survey of data users, our
other sources indicated the importance of building state and local
capacity for nutrition monitoring.  For example, past evaluations
recommended assisting state and local nutrition monitoring
activities.  These recommendations are mirrored in the Interagency
Board's 10-year comprehensive plan, which clearly signals the
importance of states and localities in the NNMRRP by devoting three
of its six objectives to strengthening state and local monitoring
activities. 

The expert advisers also noted the role of states and localities in
nutrition monitoring, but disagreed about the responsibility states
should have for data collection.  Some argued for state-based data
collection that feeds into a federal system, and others argued for
less state responsibility for data collection, but for increased
consideration of state needs in federal data collection activities. 
Their suggestions for assisting states and localities include

  -- providing financial assistance to states to determine their own
     data needs,

  -- creating a federal-state partnership in which states can provide
     funds for some extra sampling or extra questions on federal
     surveys,

  -- developing standardized modules of interest for state data
     collection activities,

  -- assisting state collection of data on subpopulations, and

  -- providing technical assistance in data interpretation. 


   APPROACHES TO ACHIEVING THE
   SELECTED MODEL FEATURES
---------------------------------------------------------- Chapter 2:3

The rest of the report describes the strengths and limitations of
current and potential approaches to achieving the selected model
features.  These approaches are listed in table 2.3.  The alternate
strategies were selected from the suggestions generated by the expert
advisers using the criteria of responsiveness to criticisms of the
current approach and feasibility.  For example, the current approach
to coordination--the Interagency Board--is criticized for its lack of
authority over the member agencies.  In contrast, an independent
central authority could have influence over the NNMRRP agencies.  The
options were considered feasible if they were already used in other
programs with similar issues (such as the lead agency approach for
other cross-agency programs), past activities of the NNMRRP (such as
the special study approach for information on subpopulations), or
related current activities by NNMRRP agencies (such as indirect
estimation). 



                         Table 2.3
          
          Features of a Model Program and Current
                  and Potential Approaches

Model feature       Current approach    Potential option
------------------  ------------------  ------------------
Coordinated system  Interagency Board   Coordination
of activities       and Working Groups  through a central
                                        authority

                                        Coordination from
                                        a single lead
                                        agency

Continuous          State-based         Continuous
collection of data  surveillance        nationally-
                    systems             representative
                                        survey

                                        Addition of
                                        nutrition
                                        questions to
                                        existing ongoing
                                        surveys

Support for         Oversampling of     Regular, separate
reliable            selected groups in  studies of
inferences about    the national        subpopulations
subpopulation       surveys
groups and small
geographic areas

                    State-based         Development of
                    surveillance        indirect
                    systems             estimation
                                        programs

Assistance to       State-based         Community-based
states and          surveillance        nutrition
localities          systems             monitoring
----------------------------------------------------------

APPROACHES TO COORDINATING THE
NUTRITION MONITORING PROGRAM
============================================================ Chapter 3

As described in chapter 2, a model program would have a mechanism for
coordinating the various nutrition monitoring activities to maximize
the utility of the data and minimize the costs of its collection. 
This chapter first reviews the status of current NNMRRP activities to
improve coordination.  Then, two other possible coordination
mechanisms-- coordination by a central authority and coordination by
a single lead agency--are examined.  (Table 3.1 provides an overview
of the strengths and limitations of the various approaches to
coordination.)



                                    Table 3.1
                     
                       Approaches to Coordinating Nutrition
                              Monitoring Activities

Approach                  Strength                    Limitation
------------------------  --------------------------  --------------------------
Current: Interagency      Improved communication and  Difficulties setting
Board and Working Groups  cooperation among the       priorities across
                          agencies                    agencies

                          Compilation of a            Inadequate assessment of
                          coordinated budget report   the needs for data by
                                                      different kinds of users

                                                      Difficulties in
                                                      disaggregating the costs
                                                      of the nutrition component
                                                      of multipurpose programs

                                                      Limited authority, staff,
                                                      and financial resources
                                                      for coordination

Potential

Coordination through a    Increased ability to        Conflict between research
central authority         establish priorities        concerns and policy
                                                      concerns
                          Alignment of agency
                          priorities with             Difficulties in
                          overarching goals through   disaggregating the costs
                          centralized budget          of multipurpose programs
                          reviews
                                                      Limited staff and
                          A central contact point     financial resources for
                          for state and local         coordination
                          governments, interest
                          groups, Congress, and
                          other federal agencies

Coordination from a       Improved communication and  Difficulties setting
single lead agency        cooperation among the       priorities across
                          agencies                    agencies

                          Compilation of a            Difficulties in
                          coordinated budget report   disaggregating the costs
                                                      of multipurpose programs
                          A central contact point
                          for state and local         Limited authority, staff,
                          governments, interest       and financial resources
                          groups, Congress, and       for coordination
                          other federal agencies
                                                      Decreased attention to the
                                                      data needs of other
                                                      agencies
--------------------------------------------------------------------------------

   CURRENT APPROACH:  THE
   INTERAGENCY BOARD
---------------------------------------------------------- Chapter 3:1

As described in chapter 1, the NNMRRP meets multiple needs for
nutrition-related data.  Yet, historically, these needs have not been
met by an integrated program with the capacity for evaluating data
needs and making adjustments as those needs change.  Instead, a
fragmented system of activities developed over the decades as new
needs for nutrition data were identified.  For example, in the early
1930s, USDA developed its first national survey of household food
consumption because data on the food supply provided no information
about the distribution of food at the household and individual
levels.  Similarly, the nutrition component was added to the National
Health Examination Survey in the early 1970s in response to a need
for more information about hunger, and state-based surveillance
systems were established in recognition of the primary role of states
in providing services to populations at risk of nutritional problems. 

To address concerns about the lack of coordination across the
agencies involved in nutrition monitoring, the National Nutrition
Monitoring and Related Research Act of 1990 required the Secretaries
of Agriculture and Health and Human Services to implement a
coordinated program of nutrition monitoring.  The act specified
several tools:  an Interagency Board, the development of a
comprehensive plan for the program, a council of outside advisers,
and an integrated budget. 

The Interagency Board created by the act has the difficult task of
coordinating numerous data collection and analysis activities across
several agencies that have traditionally had separate and distinct
missions and operations.  The Board has two chairpersons, one
selected by the Secretary of Agriculture and one by the Secretary of
Health and Human Services.  For USDA, the chair is the Under
Secretary for Research, Education, and Economics.  The HHS chair is
the Assistant Secretary for Health.  Membership on the Board includes
representatives of various agencies in USDA and HHS, as well as the
Bureau of the Census, Agency for International Development, Bureau of
Labor Statistics, Department of Defense, Department of Veterans
Affairs, and Environmental Protection Agency, among others.  The
Executive Secretary for the Board rotates between USDA and HHS every
2 years.  To facilitate coordination, the Interagency Board
established working groups focused on survey comparability, food
composition, and federal-state linkages and information
dissemination. 

The Secretaries and the Interagency Board were charged with
developing a 10-year strategic plan, which was published in the
Federal Register on June 11, 1993.  The plan outlines a set of
planning activities, including a general time frame and the lead
agencies for each activity.  The activities are organized around six
objectives, listed in chapter 1 (see p.  14).  The Interagency Board
clearly recognizes the need for improved coordination since four of
these objectives focus on either the coordination of the data
collection activities or the comparability of the data. 

To advise the Board on the development and implementation of the
NNMRRP, the act established the National Nutrition Monitoring
Advisory Council.  The members of the Council represent academic
institutions and other interested parties drawn from outside the
federal government.  The act also required the Interagency Board to
submit annually a coordinated budget for nutrition monitoring. 


      STRENGTHS
-------------------------------------------------------- Chapter 3:1.1

Both the concern that preceded passage of the act and the structure
it created appear to have improved communication and cooperation
among the agencies.  The Board and its working groups provide
mechanisms for communication and joint decision-making.  Specific
actions that demonstrate the increased coordination include the
development of

  -- common population descriptors for use in conducting and
     reporting the 1994-96 CSFII and the next NHANES,

  -- a marketing and distribution plan for NNMRRP reports,

  -- an automated dietary intake interview that would facilitate
     timely data release and linkage across CSFII and NHANES, and

  -- a common set of questions on food security (a concept that
     addresses the certainty about having enough to eat) to be used
     in the Current Population Survey. 

In addition, a jointly funded research project explored the possible
linkage of CSFII and NHANES sampling plans.  Alternate sampling
designs were evaluated using the criteria of (1) ability to satisfy
the separate objectives of the two surveys, (2) benefits in overall
costs or analytic power, and (3) feasibility, especially in terms of
burden on the survey respondents.  The draft report from the
contractor on the project emphasized the compromises one or both
surveys would have to make to link their sample designs.\1 For
example, while the combination of the NHANES and CSFII into a single
survey could yield a rich database of information on diet and health,
the likely increase in respondent burden could reduce response rates
and response quality.  Another alternative--using linked samples for
NHANES and CSFII--could decrease CSFII's precision because NHANES'
sampling design is determined by the survey's reliance on mobile
examination centers to conduct physical exams of the respondents.\2
The Interagency Board concluded that the two surveys should remain
independent, although work on improving the comparability of the data
should continue. 

As required by the act, the Interagency Board submits an annual
budget for the NNMRRP to the Congress.  The budget report includes
costs allocated for data collection, related research, information
dissemination and exchange, and technical assistance; however, these
different types of costs are not distinguished.  Instead, the funds
dedicated to nutrition monitoring and related activities are reported
only by agency, not by type of activity.  The budget report for
fiscal years 1994-96 indicated that a total of $157.7 million was
dedicated to nutrition monitoring or related research in 1994.  Of
that, $30 million was reported by the Centers for Disease Control and
Prevention (which has responsibility for the HHS surveys and
state-based surveillance systems) and $9.3 million was reported by
the Human Nutrition Information Service (which had responsibility for
NFCS and CSFII).\3 The remainder was accounted for by agencies whose
primary involvement in nutrition monitoring is related research.  For
example, HHS' National Institutes of Health reported $25.9 million
dedicated to NNMRRP activities and USDA's Agricultural Research
Service accounted for $50.7 million. 

The budget report is useful in communicating to the Congress a
general sense of the cost of the NNMRRP across agencies.  However,
funds for nutrition monitoring cannot always be disaggregated from
other purposes of the data collection and research programs.  As a
result, the budget report contains only approximate amounts dedicated
to nutrition monitoring and related research.  Moreover, with the
recent incorporation of the office responsible for NFCS and CSFII
into ARS, determining which funds are dedicated to the monitoring
activities and which are used for related research will be more
difficult. 


--------------------
\1 An Evaluation of Linked Survey Designs for NHANES and CSFII,
prepared for NCHS by Westat, July 29, 1994. 

\2 HHS maintains only three of these expensive mobile units and, as a
result, has a sampling design that minimizes the geographic
dispersion of respondents.  However, HHS is reviewing alternatives to
the mobile examination centers. 

\3 The Human Nutrition Information Service and its functions have
been merged with the Agricultural Research Service (ARS). 


      LIMITATIONS
-------------------------------------------------------- Chapter 3:1.2

The literature on the development of objectives to increase
accountability for program results indicates that (1) objectives
should be written in terms that can be used to judge progress toward
achieving them and (2) implementation plans and specific measures of
progress should be developed for the goals and objectives.\4 In the
10-year plan, the objectives are stated in general, global terms so
that it is not clear when an objective can be considered achieved. 

For example, the first objective is to "provide for a comprehensive
NNMRRP through continuous and coordinated data collection." Neither
the objective itself, nor the text following it clearly defines what
the terms "comprehensive" or "coordinated" mean.  Similarly, the
activities are too vague to be considered implementation plans for
the objectives; for example, no activity directly relates to the
development of continuous data collection. 

As another illustration, the activity, "identify ways to increase
comparability within a dietary method to improve the quality and
usefulness of data," specifies neither the degree of quality required
nor the uses that will be facilitated.  Without more concrete,
measurable objectives, there is little accountability for the program
because progress toward the objectives cannot be assessed.  (Other
examples of activities listed in the 10-year plan can be found in
table II.2 in appendix II.)

In addition, the plan did not describe how activities would be ranked
by importance or addressed within current fiscal constraints. 
However, since the plan was published, the Interagency Board drafted
an approach for ranking the 68 activities listed in the plan into
three categories of priorities:  (1) essential (mandatory,
legislatively required), (2) necessary (critical but not mandatory),
and (3) beneficial.  An interagency implementation group, involving
around 60 agency representatives, applied the approach.  Twenty-five
of the activities were ranked in the high-priority category, 33 were
ranked as next most important, and only 10 were ranked as beneficial
but not critical.  While this is an important first step in setting
priorities, the Interagency Board has not yet linked the top-ranked
tasks to the costs and benefits of completing them.  Such a framework
could be used to understand the trade-offs in selecting one or
another approach to each task. 

Since an assessment of the effectiveness of the different approaches
to the objectives requires information on the uses of the data,
another obstacle to the Interagency Board's implementation of a
coordinated system is the absence of a comprehensive assessment of
how nutrition monitoring data are used across the federal government
and by data users in other settings.  For example, although USDA and
HHS held a joint workshop to assess the needs of users of dietary
intake data in August 1994, the workshop included only
representatives of federal agencies. 

Finally, the ability of the Interagency Board to coordinate the
NNMRRP is limited by the lack of resources dedicated to coordination. 
The Interagency Board has no staff, although two people--one from
USDA and one from HHS--have been given primary responsibility for
organizing NNMRRP activities.  The NNMRR Act gives the Secretaries
the option of appointing an administrator for the program, but so far
they have chosen not to exercise that option. 


--------------------
\4 Performance Measurement:  An Important Tool in Managing for
Results (GAO/T-GGD-92-35; May 5, 1992). 


   POTENTIAL OPTIONS
---------------------------------------------------------- Chapter 3:2

While recognizing the progress made by the Interagency Board, we also
considered other coordination mechanisms.  Two options for improving
coordination--through a central authority or by a lead agency--are
reviewed in detail in this section.  In addition, other options
discussed by our expert advisers are briefly presented. 


      COORDINATION THROUGH A
      CENTRAL AUTHORITY
-------------------------------------------------------- Chapter 3:2.1

The suggestion of coordination through a central authority came in
response to the lack of enforcement power held by the Interagency
Board over the member agencies.  The kind of central, coordinating
agency envisioned by the expert advisers is most clearly exemplified
by the executive offices in the White House.  Therefore, to examine
the advantages and disadvantages of having a central authority
provide coordination, we reviewed the literature (including prior GAO
reports and congressional hearings) and interviewed agency officials
in three White House offices that have coordination responsibilities: 
Office of Management and Budget (OMB), Office of Science and
Technology Policy (OSTP), and Office of National Drug Control Policy
(ONDCP).  As indicated by the brief descriptions of each office
provided in table 3.2, the coordination tools used by the White House
offices are similar to those used by NNMRRP's Interagency Board,
including interagency committees and working groups, development of
plans, and review of budgets. 



                               Table 3.2
                
                Responsibilities and Coordination Tools
                      of Three White House Offices

Office    Responsibility                 Tool
--------  -----------------------------  -----------------------------
OMB       Ensure coordination and        Review and approve agency
          cooperation where federal      budgets
          agency lines of authority
          overlap                        Review and approve data
                                         collection instruments
          Assist in the development of
          efficient coordinating
          mechanisms

          Expand interagency
          cooperation

OSTP      Coordinate research and        National Council on Science
          development activities in      and Technology (NCST), which
          other agencies                 is chaired by the President
                                         and includes Cabinet
          Review and participate in the  Secretaries and selected
          annual formulation of the      agency heads
          budgets related to science,
          research, and development      NCST subcommittees and
                                         working groups

                                         Review of agency budgets for
                                         interagency programs

                                         Commissioning studies of
                                         controversial issues

ONDCP     Through the National Drug      Interagency working groups
          Control Strategy, establish
          policies, objectives, and      Review of agency plans for
          priorities for federal drug    implementing the National
          control programs               Drug Control Strategy

          Coordinate federal budget for  Review and certification of
          drug activities                drug control budgets
----------------------------------------------------------------------

         POTENTIAL STRENGTHS
------------------------------------------------------ Chapter 3:2.1.1

One potential advantage of elevating the coordination of nutrition
monitoring to a high-level central authority is the increased
participation of high-level officials in coordination activities. 
This participation could, in turn, increase the ability of the
coordinating body to establish priorities.  Currently, some of the
agencies participating in the Interagency Board are represented by
administrators or directors, while other agencies are represented by
staff members who have no authority to change agency activities or
establish priorities.  In contrast, the political visibility of a
program under White House management could encourage agencies to send
representatives in positions of authority, capable of establishing
priorities and committing resources to support them. 

The political visibility that comes with the participation of a
high-level central authority may also contribute to the effectiveness
of the various coordination tools.  For example, the Interagency
Board compiles budgets obtained from each of the agencies involved in
nutrition monitoring without reviewing them for consistency with the
activities identified in the 10-year comprehensive plan.  In
contrast, the White House offices use the budget review process to
bring activities of the agencies into line with overarching policy
goals.  For example, ONDCP can threaten to decertify an agency's drug
budget if it is not consistent with the National Drug Control
Strategy.  Decertification has no practical ramifications, but it
sends a politically important message about the priority given to
drug control activities by the White House. 

An additional potential advantage of having a central authority
provide coordination is that it can be a central location for
assistance to all data users.  For example, ONDCP has a Bureau of
State and Local Affairs that works with state and local government
agencies involved in drug control activities.  The Bureau serves as a
clearinghouse for information about state and local activities and
uses conferences to increase communication with and among state and
local officials.  In addition, the Bureau can communicate the
concerns of state and local governments and community groups to the
federal agencies involved in drug control programs.  For nutrition
monitoring, such an office could provide a central contact point for
users in a variety of settings, including federal, state, and local
governments, food industry, and health care organizations. 


         POTENTIAL LIMITATIONS
------------------------------------------------------ Chapter 3:2.1.2

Although the increased political visibility of a high-level central
coordination office may facilitate the development of priorities, it
may also increase the potential for political pressure on the data
collection and research.  A conflict between ONDCP and HHS over the
data collection and reporting of drug data illustrates this issue. 
HHS was concerned about the degree of ONDCP's involvement in how the
data were collected and reported, while ONDCP expected HHS to meet
its data needs.  Political influence on the scientific agenda is also
a concern for OSTP, where priorities may change with changing
administrations even if scientific issues remain the same. 

In addition, although White House offices have more influence over
the budgets reported by the different agencies, the budgets developed
for other programs that cut across agency jurisdictions share the
limitation of the coordinated NNMRRP budget.  Specifically, agency
activities may serve multiple purposes, thus making it difficult to
determine how much of the overall costs are dedicated to the
interagency program.  For example, Coast Guard patrol boats serve
drug interdiction purposes, but are also used in search and rescue
missions that are not related to drug control.  Thus, the Coast Guard
can only estimate the portion of its resources dedicated to
supporting national drug control efforts.  Ambiguity about the
portion of a multipurpose program that serves the interagency purpose
makes it difficult to monitor the costs of the program. 

Although central authorities like the White House offices are
financed separately from the agencies they oversee, resource
limitations persist.  For example, HHS staff attributed part of their
conflict with ONDCP to a lack of technical and substantive expertise
among ONDCP staff.  Similarly, a past director of OSTP identified
limited staff resources as a reason why long-term planning received
less attention than short-term problems. 


      COORDINATION THROUGH A LEAD
      AGENCY
-------------------------------------------------------- Chapter 3:2.2

Another possible approach to coordinating the NNMRRP is locating the
responsibility within a single agency.  To investigate this approach,
we discussed the option with our expert panels and used program
documents and evaluations conducted by GAO and the Congressional
Budget Office to examine the experience of the High Performance
Computing and Communications Program (HPCCP).\5 Like the NNMRRP, the
HPCCP involves multiple agencies with different strengths and
missions.  Unlike the NNMRRP, the oversight of the HPCCP is located
in OSTP, which delegated the responsibility for coordinating the
activities across the agencies to a single agency, the National
Library of Medicine.  While not a major player in high-performance
computing, the National Library of Medicine was seen as an
independent, unbiased participant with interest in and knowledge
about the technology.  The Library of Medicine's role has been to
pull together materials for program reports, convene meetings, and
provide a clearinghouse. 


--------------------
\5 The purpose of HPCCP is to further the development and
dissemination of U.S.  supercomputer and high-speed computer network
technologies. 


         POTENTIAL STRENGTHS
------------------------------------------------------ Chapter 3:2.2.1

HPCCP's National Coordination Office shares two of the strengths of
the Interagency Board:  It appears to have facilitated communication
among the agencies and coordination of individual activities.  In
addition, it has provided the Congress with a budget that looks at
the costs of high- performance computing activities across agencies. 
Moreover, the National Coordination Office has the added advantage of
providing the Congress and the public with a central contact point
for information on high-performance computing. 


         POTENTIAL LIMITATIONS
------------------------------------------------------ Chapter 3:2.2.2

Just as the HPCCP shares the NNMRRP's strengths, it also has some of
its limitations.  Specifically, no mechanism is in place to set
priorities across different agency activities.  In a recent review,
GAO suggested that the HPCCP needs

     "an explicit technical agenda, identifying and prioritizing
     specific technology challenges and establishing a framework of
     expected costs and results .  .  .  [to] clarify the program's
     goals and objectives, focus efforts on critical areas, and serve
     as a baseline for measuring program progress and results."\6

In addition, HPCCP did not have uniform guidelines for which research
activities should be included in the budgets submitted by the
different agencies, mirroring the difficulty the other coordination
mechanisms have in tracking funds used for multipurpose programs. 
The National Coordination Office also shares the NNMRRP's lack of
budget and staff resources for coordination. 

An additional concern about the lead agency model was raised by our
expert advisers:  If responsibilities for nutrition monitoring were
located in one agency, nutrition monitoring might become a monopoly,
serving the needs of only one agency.  Its current dispersion across
agencies allows the different components to serve different purposes. 
However, safeguards--such as that used for the HPCCP when it was
located in an agency that did not have a large investment in
high-performance computing, relative to some of the other
agencies--could address this concern. 


--------------------
\6 High Performance Computing and Communications:  New Program
Direction Would Benefit From a More Focused Effort (GAO/AIMD-95-6;
Nov.  4, 1994), p.  22. 


      OTHER COORDINATION
      SUGGESTIONS
-------------------------------------------------------- Chapter 3:2.3

In addition to coordination by an interagency body with permanent
staff and enforcement authority or a single lead agency, other
suggestions for improving coordination of the NNMRRP were to locate
nutrition monitoring in statistical agencies within the user
departments, centralize the congressional appropriations process for
nutrition monitoring activities, and ensure informed review of data
collection plans by qualified staff at the Office of Management and
Budget. 

The first idea--locating nutrition monitoring in statistical agencies
within the departments that use the data--was intended to focus the
program on the quality of the data collected.  However, a concern
about this suggestion was that it could result in decreased
responsiveness to the needs of the data users.  We did not pursue it
because the major surveys are already located in statistical or
research branches of HHS and USDA.  (NHANES is operated by the
National Center for Health Statistics, and NFCS and CSFII were
recently relocated to ARS.)

Similarly, OMB already has responsibility for reviewing the data
collection activities of the agencies, although OMB staff reported
that they rely on the agencies to describe coordination efforts. 
Finally, we did not review the advantages and disadvantages of
centralizing the congressional appropriations process for nutrition
monitoring activities because its relevance to the utility and
efficiency of the data collection activities was not clear. 


   CONCLUSIONS
---------------------------------------------------------- Chapter 3:3

Lack of coordination across program activities has implications for
both the effectiveness and the efficiency of the NNMRRP as a whole. 
The Interagency Board has made progress toward coordinating
activities across the different agencies.  However, the other
approaches to coordination suggest mechanisms that could further
strengthen the work of the Interagency Board. 

First, because its responsibilities are shared by USDA and HHS, the
Interagency Board does not provide a central contact point for users
of nutrition monitoring data.  A central contact could be established
if the Secretaries of USDA and HHS used their option to appoint an
administrator of the program or if responsibility for responding to
requests for information about the program was assigned to a single
agency, as it was for high-performance computing. 

Second, the Interagency Board does not review the agency budgets it
compiles for consistency with the overarching priorities of the
program.  Before such a review could occur, specific objectives and
priority activities would need to be identified.  Then, the
Interagency Board could work with OMB to secure funding of NNMRRP
priorities. 


APPROACHES TO THE CONTINUOUS
COLLECTION OF DATA
============================================================ Chapter 4

Evaluating programs intended to reduce diet-related chronic disease,
tracking progress toward health objectives, and monitoring changes in
our diets are the kinds of activities that require continuous data. 
The advantages and disadvantages of the current and alternate
approaches to collecting continuous data are summarized in table 4.1
and detailed below. 



                                    Table 4.1
                     
                     Approaches to Providing Continuous Data

Approach                   Strength                   Limitation
-------------------------  -------------------------  --------------------------
Current: state-based       Quick availability of      No in-depth food intake or
surveillance systems       data on specific issues    health data

                           Connection between data    Uneven quality and
                           elements and program       completeness of data
                           decisions                  across states

                           Efficient relative to      Limited to populations
                           national surveys because   participating in publicly
                           of the reliance on         funded programs (PedNSS
                           program records (PedNSS    and PNSS) or living in
                           and PNSS) or telephone     households with telephones
                           interviews (BRFSS)         (BRFSS)

                                                      Incomplete participation
                                                      of states in PedNSS and
                                                      PNSS

Potential

A continuous, nationally-  Up-to-date information on  Costs of conducting all
representative survey      the population             survey processes
                                                      concurrently
                           Increased flexibility to
                           respond to new issues by   Difficulties in
                           adding modules to a core   identifying a core set of
                           set of data elements       data elements

                           Increased efficiency as    Inflexibility of core data
                           start-up costs diminish    elements

Addition of nutrition      Efficient means of         Dependent on flexibility
questions to existing      gathering data at          of surveys
ongoing surveys            frequent intervals

--------------------------------------------------------------------------------

   CURRENT APPROACH:  THE
   STATE-BASED SURVEILLANCE
   SYSTEMS
---------------------------------------------------------- Chapter 4:1

Although the provision of continuous data is one of the objectives
identified in the Interagency Board's 10-year plan, not one of the
three major NNMRRP surveys--NFCS, CSFII, or NHANES--is implemented
continuously.  In fact, planned future administrations of two of the
surveys have been postponed, potentially compromising the ability to
monitor trends in diet-related health risks over time and evaluate
the effect of any changes in food assistance policy. 

NFCS has been administered at approximately 10-year intervals since
the 1930s.  Its next implementation was planned to begin in 1996, but
is now tentatively scheduled for 1998, depending on funding.  CSFII,
originally intended to provide continuous data on dietary intake, has
had three separate administrations:  1985-86, 1989-91, and 1994-96. 
After a 1-year pause in 1997, it is expected to resume for another
3-year period in 1998.  Since the National Health Examination Survey
gained a nutrition component in 1971, NHANES has been fielded three
times:  1971-75, 1976-80, and 1988-94.  Like NFCS, its future
implementation is uncertain because of budget constraints.  Its
planned implementation in 1997 is now expected to be postponed. 

Because of the lack of certainty about the implementation of the
national nutrition surveys, the state-based surveillance systems are
currently the primary source of continuous data in the NNMRRP.  As
described in chapter 1, PedNSS and PNSS rely on data from clinic
records from publicly funded health, nutrition, and food assistance
programs and BRFSS collects information through telephone interviews,
with respondents (adults 18 years and over) identified through random
digit dialing.  While a valuable source of quick information for
state and local program managers, the surveillance systems do not
meet the needs of researchers or program decisionmakers who require
either national data or in-depth food intake data. 


      STRENGTHS
-------------------------------------------------------- Chapter 4:1.1

Compared to the national surveys, one of the strengths of the
state-based surveillance systems is that they not only provide data
continuously, but they are also able to process and report the data
relatively quickly.  For PedNSS and PNSS, information is collected as
part of the process of receiving services from WIC and other publicly
funded health, nutrition, and food assistance programs.  Because they
depend on program records, PedNSS and PNSS do not burden respondents
the way surveys dependent on interviews do.  The information is
transmitted from the records of local health and nutrition programs
to the state, which then forwards the records to HHS for analysis. 
Similarly, the data collected by the states for BRFSS are sent to HHS
for processing.  According to HHS officials, all three systems report
data back to the states within a year and generally in less than 9
months.  HHS has also helped states conduct their own analyses by
distributing a standardized software package. 

Another strength of the state-based surveillance systems is that the
data they collect are directly linked to program decisions.  Although
PedNSS and PNSS include only a few indicators of nutritional
deficiencies and behaviors, they are selected to support state data
needs for program planning and management.  For example, the data are
used to target resources for the WIC program.  Similarly, BRFSS data
have been used to inform decisions about nutrition education
programs, such as campaigns to encourage the consumption of five
servings of fruits and vegetables a day. 


      LIMITATIONS
-------------------------------------------------------- Chapter 4:1.2

In contrast to the national surveys, the surveillance systems do not
permit examination of diverse diet-health associations across the
entire population.  Instead of collecting extensive biochemical,
anthropometric, and interview data, they focus on a narrow range of
variables relevant to specific programs or nutritional risks.  For
example, PedNSS collects clinical data on weight and height, monitors
infant feeding practices, and assesses anemia.  PNSS collects
information on anemia and behaviors associated with low-birthweight
babies.  BRFSS asks respondents to report on their consumption of
fat, fruits, and vegetables.  This focus limits the breadth of uses
that can be supported by the data; however, as noted above, it also
limits the burden placed on respondents. 

While the systems are currently limited in the amount of dietary data
they collect, HHS is exploring other methods of gathering these data. 
For example, with HHS support, the University of Texas examined the
use of bar code data to look at dietary patterns.  They concluded
that the technology is not yet ready for use, but through a
partnership with food manufacturers could be a promising method for
the future.  In addition, with USDA, HHS is evaluating the
feasibility of collecting additional dietary data in the clinics that
provide the PedNSS and PNSS records. 

Another concern about surveillance systems is the quality and
completeness of the data across the different states.  For example,
for PedNSS, error can be introduced by variations in practice in
weighing infants, such as with or without the baby's winter clothes,
or by clerical errors in entering the data in states that do not have
automated data systems.  PNSS, which attempts to collect a wider
range of information than PedNSS, suffers from missing data on
several variables, such as the pregnancy risk factors of smoking and
alcohol consumption.  However, HHS provides technical assistance to
help states standardize their data collection procedures.  In fact,
by flagging biologically implausible values for the physical
measures, HHS analyses of the surveillance data help identify clinics
that may have poor procedures. 

The surveillance systems are also limited because, within the
participating states, only certain groups of the population are
covered.  As described earlier, PedNSS and PNSS primarily provide
information on mothers and children participating in the WIC program. 
As a result, data may not be available on other populations that are
potentially at risk, such as homeless people or older children not
eligible for WIC.  In contrast, BRFSS has a wider target population,
collecting data from randomly selected adults 18 years and over. 
However, neither adults in households without telephones nor children
are covered by BRFSS.  Since there is evidence that some health risk
factors, such as smoking, are associated with living in a household
without a telephone, this could affect estimates of the extent of
diet-related risk factors as well. 

Finally, not all states participate in the PedNSS and PNSS
surveillance systems and, therefore, they are not good sources of
national-level data on their populations of interest.  In 1993, 38
states participated in PedNSS and only 20 states participated in
PNSS.  (In contrast, BRFSS has good state coverage; in 1993, only
Wyoming did not participate.)


   POTENTIAL OPTIONS
---------------------------------------------------------- Chapter 4:2

In addition to the state-based surveillance systems, we reviewed two
other approaches to providing continuous data.  The first option is a
national nutrition-related survey that is operated continuously.  To
describe the strengths and weaknesses of this approach, we consulted
with our expert advisers and interviewed managers of the current
national nutrition monitoring surveys about the continuous operation
of the surveys.  The second option is the inclusion of
nutrition-related questions on continuous non-NNMRRP surveys.  The
strengths and weaknesses of this alternative were explored by
examining the NNMRRP's recent experience developing food insecurity
questions for the Current Population Survey (CPS).  In addition, our
expert advisers and the data users who responded to the survey made
other suggestions, which are also briefly presented. 


      A CONTINUOUS NNMRRP SURVEY
-------------------------------------------------------- Chapter 4:2.1

An ongoing national survey is a possible approach to providing
continuously updated information that addresses some of the
limitations of the state-based surveillance systems.  One of the
current national surveys, the CSFII, was developed in response to
calls for the continuous collection of individual data on food
intake.  It has not been implemented regularly in the past; however,
current plans are for the ongoing implementation of CSFII as a 3-year
survey followed by a 1-year pause for planning and development before
the next 3-year period. 


         POTENTIAL STRENGTHS
------------------------------------------------------ Chapter 4:2.1.1

An ongoing survey has the potential to yield continuously updated,
timely data for monitoring the nutritional status of the nation's
population.  The increased timeliness of the data could decrease the
current pressure from collaborating federal agencies to include
components on each implementation of the periodic surveys.  For
example, in contrast to the recent implementation of NHANES, which
attempted to meet as many data needs as possible within a single
survey, the ongoing implementation of NHANES could contain a core set
of data items that would be collected continuously, supplemented at
intervals by rotating modules.  Such a streamlining would also reduce
the burden on respondents. 

Moreover, the surveys could become more flexible by distinguishing
between variables that change rapidly, variables that need regular
but not continuous monitoring, and variables of emerging policy
importance.  The core set of items in continuous implementation could
gather information on rapidly changing variables.  Topical modules on
issues that do not change as rapidly could be included on a regular
schedule.  Finally, as new issues arise, additional questions could
be added. 

In addition to increased timeliness and flexibility, a continuous
survey operation could be more efficient than periodic surveys
because current costly start-up activities, such as planning,
designing sampling strategies, and training interviewers, would be
diminished.  Thus, the data could be collected for less cost per
respondent.  However, as described below, without concomitant
streamlining of the survey, overall costs could increase. 


         POTENTIAL LIMITATIONS
------------------------------------------------------ Chapter 4:2.1.2

While an ongoing survey could save money on start-up costs, it could
increase costs overall because activities that are now funded and
staffed sequentially would require a continuous flow of resources to
be conducted concurrently.  For example, as NHANES is currently
implemented, staff change their activities as the survey moves
through the phases of planning, implementation, analysis, and
dissemination.  If NHANES was implemented continuously, all of these
activities would be going on at the same time.  According to both
USDA and HHS officials, the primary constraints on the continuous
operation of their surveys are the need for dependable funding and
sufficient staff resources.  As described above, the absence of
dependable funding has affected the frequency with which the national
surveys are now implemented. 

The development of a survey that continuously collects data on a core
set of items and intermittently collects data on other issues raises
two difficult issues.  First, the definition of the core items is
complex.  An expert panel convened by the Federation of American
Societies for Experimental Biology was charged with identifying a set
of core indicators to assess the nutritional status of
difficult-to-sample populations.  The report summarizing the work of
the panel noted that (1) the suitability of an indicator changes with
the purposes for the data and (2) information on the determinants and
the consequences of each indicator is also needed.\1 The panel ended
by identifying three sets of indicators--minimal, intermediate, and
comprehensive--without recommending specific measures for the
indicators. 

The second issue with a continuous survey is the potential
inflexibility of the core items once they are selected and
implemented.  While opportunities to test new methods are enhanced
because the survey is in the field continually, making changes to the
data collection procedures can be difficult because of the pressure
to ensure that the measures are consistent over time.  Without such
assurance, changes in an indicator such as obesity may be the result
of changes in how obesity is measured rather than changes in the
prevalence of the condition itself.  Our expert advisers suggested a
survey with built-in periods of transition to allow the survey to
incorporate new methods and new data elements as they emerge. 


--------------------
\1 S.A.  Anderson, ed., "Core Indicators of Nutritional State for
Difficult-to-Sample Populations," Journal of Nutrition, 120:11S (Nov. 
1990), 1559-99. 


      ADDING NUTRITION QUESTIONS
      TO EXISTING CONTINUOUS
      SURVEYS
-------------------------------------------------------- Chapter 4:2.2

A third approach to obtaining frequently up-dated information is the
addition of nutrition-related questions to existing continuous
surveys.  The potential of existing surveys to provide data regularly
on some variables is demonstrated by plans that included NNMRRP
questions on food security in the April 1995 Current Population
Survey conducted by the Census Bureau.\2 Food security is a concept
intended to go beyond the idea of hunger to measure the availability
of food for a family or individual. 

The food security questions were developed by an interagency working
group cochaired by HHS' National Center for Health Statistics and
USDA's Food and Consumer Service.  While the working group developed
questions, USDA reserved space on the CPS for 1995.  The question
development process included determining how the data would be used
and soliciting input from both federal and other data users.  The
food security module contains both core questions and supplemental
ones.  The Census Bureau will include the complete module.  NNMRRP
surveys could include the smaller core set of questions.  If the
initial implementation of the questions yields useful data, USDA
plans to continue to support the inclusion of the food security
questions annually in the CPS. 

The same strategy of piggy-backing nutrition-related questions on
continuous surveys could be used with other surveys, such as the
Survey of Income and Program Participation or the National Health
Interview Survey.  The latter has the added advantage of collecting
health data that could be linked to nutritional indicators. 


--------------------
\2 The CPS is a monthly survey of about 60,000 households across the
country. 


         POTENTIAL STRENGTHS
------------------------------------------------------ Chapter 4:2.2.1

The major advantage of this approach is the efficiency with which
data on specific issues can be gathered.  USDA will pay for the cost
that the food security questions add to the CPS without having the
responsibility or the cost of fielding and managing the survey
itself. 


         POTENTIAL LIMITATIONS
------------------------------------------------------ Chapter 4:2.2.2

A potential disadvantage of this approach is that, just as core
questions on nutritional status could be inflexible, existing
continuous surveys can be hard to change because of their momentum. 
Moreover, existing surveys have their own set of constraints and
limitations.  For example, while CPS could accommodate the addition
of questions about food security, it probably could not accommodate a
module obtaining data on an individual's dietary intake over the last
24 hours.  A 24-hour recall instrument requires considerable training
to administer and adds substantially to the burden on the respondent. 
For these reasons, opportunities to piggy-back nutrition-related
questions on other surveys may be limited. 


      OTHER OPTIONS FOR CONTINUOUS
      DATA
-------------------------------------------------------- Chapter 4:2.3

In addition to the approaches reviewed, other actions were suggested
by our expert advisers and data users who responded to our survey. 
Specifically, longitudinal surveys were suggested as an efficient way
to collect data over time because a new sample does not have to be
selected every time a survey is fielded.  Since original respondents
are followed up in subsequent administrations of the survey,
longitudinal surveys can also be useful for tracking individual-level
changes in food consumption behavior.  However, longitudinal surveys
have their own costs, including the need to collect additional data
so that respondents can be found for later surveys and the likelihood
of attrition as respondents either drop out or cannot be found. 

Another suggestion was the collection of survey data using automated
survey technology.  Because direct entry of the data into an
automated system can speed the processing and aggregation of the
survey, it can also accelerate the release of the data for analysis. 
However, it does not affect the regularity with which the data are
collected in the first place.  HHS already uses automated data
collection for NHANES.  USDA is planning to automate the next
administration of the CSFII. 


   CONCLUSIONS
---------------------------------------------------------- Chapter 4:3

The funding constraints that have caused the postponement of two of
the national surveys jeopardize the availability of periodic data on
the population as a whole.  Although the state-based surveillance
systems provide continuous information, they are inadequate to meet
the need for data on the population at large or on in-depth nutrition
and health status.  Moreover, if approved, proposals to collapse
funds for the WIC program with other food assistance programs into a
block grant for the states could affect the major source of data for
two of the state-based surveillance systems, PedNSS and PNSS. 

Although the NNMRRP has pursued such creative solutions as including
food security questions on the CPS, the availability of up-to-date
information could worsen.  To decide how best to meet the needs for
continuous data in the future, the NNMRRP would first need to analyze
the purposes that require frequently collected data and the current
mechanisms for supporting those purposes.  Within this framework, the
strengths and limitations of the different approaches to increasing
the frequency with which important indicators are measured could be
weighed according to which purposes are supported and which are
diminished. 


APPROACHES TO SUPPORTING RELIABLE
INFERENCES ABOUT SUBPOPULATIONS
============================================================ Chapter 5

Subpopulations can be defined by geographic location as well as by
age and sex (such as infants or elderly women), physiological
characteristics (such as pregnancy), ethnicity or race (such as
Hispanic or Native American), income, and the intersection of any of
these groups (such as low-income children).  As described in chapter
2, information on subpopulations is needed to appropriately target
and evaluate programs that address nutrition-related programs.  As
summarized in table 5.1, this chapter describes current approaches
and some potential options to responding to the calls for better
information on subpopulation groups and small geographic areas in the
NNMRRP.  Other issues of federal assistance to states and localities
for nutrition monitoring are discussed in chapter 6. 



                                    Table 5.1
                     
                       Approaches to Supporting Inferences
                               About Subpopulations

Approach                   Strength                   Limitation
-------------------------  -------------------------  --------------------------
Current

Oversampling of selected   Less costly than a         Appropriate only for
groups in the national     separate survey            special populations that
surveys                                               are covered by the
                           Ability to compare the     national sampling frame
                           subpopulation to the       (such as children in low-
                           general population         income households)

                                                      Primarily appropriate for
                                                      populations that are
                                                      geographically clustered
                                                      or large

                                                      Costs of screening for
                                                      members of the group

State-based surveillance   Collection of data on      No in-depth food intake or
systems                    high-risk populations:     health data
                           PedNSS (low-income
                           children) and PNSS (low-   Uneven quality and
                           income pregnant women)     completeness of data
                                                      across states
                           Collection of data
                           representative of the      Incomplete participation
                           state population of        of states in PedNSS and
                           adults, 18 years or older  PNSS
                           (BRFSS)

                           Potential expansion to
                           include programs that
                           serve other at-risk
                           populations

Potential

Special studies            Ability to tailor data     Inability to compare to
                           collection instruments     the rest of the population
                           and content to the         unless the special study
                           subpopulation              is conducted in tandem
                                                      with a national survey
                           Collection of data on      using comparable
                           subpopulations that are    instruments
                           not adequately covered by
                           the sampling frame used    Costs of tailoring data
                           for national surveys       collection instruments,
                                                      developing the sampling
                                                      frame, and screening

                                                      Difficulties in developing
                                                      a sampling frame that
                                                      allows for generalization

Development of indirect    Efficient means of         Difficult to assess the
estimation programs        generating estimates for   quality of the estimate
                           subpopulations
                                                      Diminished confidence in
                           Historical use in other    the estimates because they
                           areas (estimates of        are based on models rather
                           population, crop yields)   than direct observation

                           Response to limitations
                           of other sources of
                           information, such as cost
                           and relevance
--------------------------------------------------------------------------------

   CURRENT APPROACHES
---------------------------------------------------------- Chapter 5:1

Subpopulation groups are covered in two ways by current NNMRRP
activities.  First, the three national surveys use oversampling of
certain groups to ensure the selection of enough respondents to
support subpopulation estimates.  NHANES focuses on racial and ethnic
subpopulations, while CSFII and NFCS have included subpopulations
defined by income, reflecting USDA's focus on food assistance to
low-income populations.  The second way in which data on
subpopulations are gathered is through the state-based surveillance
systems--PedNSS, PNSS, and BRFSS. 


      OVERSAMPLING
-------------------------------------------------------- Chapter 5:1.1

Oversampling includes members of subpopulation groups in a sample at
a rate greater than their proportion in the population.  The purpose
of oversampling is to ensure that data will be collected on enough
group members to support inferences about the group as a whole. 
Oversampling is already used in the major NNMRRP surveys.  NHANES III
(1988-94)
oversampled non-Hispanic blacks and Mexican-Americans, as well as
persons 60 years or older and children 1-5 years old.  NHANES staff
indicated that two groups (Hispanics and persons 75 years and older)
are likely to be important groups in the next implementation of the
survey because both are growing and have significant health-related
issues to study.  In addition to estimates of the general population,
the current administration of CSFII (1994-96) is expected to produce
estimates for low-income populations through oversampling.  Also,
staff of both USDA and HHS surveys stated that the national surveys
could oversample a state's population, but that the state would have
to finance the added costs. 


         STRENGTHS
------------------------------------------------------ Chapter 5:1.1.1

Oversampling has two major strengths.  First, because it can be used
in conjunction with a national survey, it has efficiencies of scale. 
Specifically, the planning and implementation costs are diminished
because they are part of a larger survey.  Second, data on both the
group and the rest of the population are collected at the same time
and with the same survey procedures, facilitating the comparison of
the two population groups. 


         LIMITATIONS
------------------------------------------------------ Chapter 5:1.1.2

Oversampling implies that the subpopulation group of interest is
included in the sampling frames used to identify participants in the
national surveys.\1 However, not all special populations are well
covered by a national sampling frame.  For example, homeless
individuals, persons who live in institutions, and American Indians
and Alaska Natives living on reservations would not be included in
the national household sampling frame. 

Even for those individuals who are included in the sampling frame,
oversampling may not be appropriate because of the costs incurred in
screening for members of the group.  Screening is the process of
asking questions at sampled households to identify whether they
represent (or include representatives of) the subpopulation of
interest.  Screening adds to the costs of the survey because enough
households have to be sampled and screened to identify the smaller
number of households or individuals that meet the definition of the
subpopulation.  To reduce screening costs, oversampling is most
effective for subpopulations that are geographically clustered or
fairly well represented in the general population, such as persons
with low income who are often clustered by neighborhood.  In
contrast, oversampling is not appropriate for groups that are few in
number or geographically dispersed, such as pregnant women. 

A possible response to some of the limitations of oversampling is the
use of multiple sampling frames.  For some subpopulations,
alternative frames or lists may be available that can be used in
conjunction with the national sampling frame.  Samples can be
selected from both the subpopulation frame and the general population
frame, and weighted estimates can then compensate for the fact that
some group members could be selected from two different sources.  For
example, to oversample for the frail elderly, the elderly individuals
identified in the sample drawn from the national sampling frame could
be supplemented by samples drawn from lists of elderly who
participate in congregate meals programs.  This approach is useful
because, relative to screening in the general population, it is an
inexpensive way to identify members of the subpopulation. 


--------------------
\1 The sampling frame is the group of units from which a sample is
actually drawn.  So, for example, both NHANES and CSFII sample from a
sampling frame of households in the 50 states and the District of
Columbia. 


      STATE-BASED SURVEILLANCE
      SYSTEMS
-------------------------------------------------------- Chapter 5:1.2

Although identified as a limitation in the previous chapter, the
focus of the state-based surveillance systems on particular
subpopulations can also be seen as a strength.  As described in
chapters 1 and 4, states use BRFSS to collect data on the health
behaviors of their adult population.  PedNSS and PNSS are sources of
information on the nutritional status of low-income mothers and
children.  In addition, HHS is exploring opportunities to expand the
program to other populations, such as schoolchildren.  However, the
other disadvantages of the systems, such as the limited amount of
nutritional data they collect and the incomplete participation of the
states, diminish their utility as a source of information on
subpopulations. 


   POTENTIAL OPTIONS
---------------------------------------------------------- Chapter 5:2

The two major alternate approaches to providing subpopulation data
are special studies and indirect estimation.  Special studies are
those that use a separate sampling frame from a national survey and
are not necessarily conducted at the same time as a national survey. 
To describe the strengths and weaknesses of special studies as an
approach to collecting information on subpopulations, we examined
HHS' past experience with the Hispanic Health and Nutrition
Examination Survey (HHANES) and discussed the option with our expert
advisers. 

Indirect estimation uses data that are not direct observations of the
group of interest to develop inferences about the subpopulation.  Our
review of this approach relied on reviews of technical literature,
interviews with USDA and Bureau of Census staff responsible for
indirect estimation programs, and consultation with methodological
experts on our panel of advisers and in HHS.  These and the current
approaches address the suggestions made by the expert advisers and
data users who responded to our survey. 


      SPECIAL STUDIES:  THE HHANES
      MODEL
-------------------------------------------------------- Chapter 5:2.1

An alternate approach to covering subpopulations is conducting a
special study.  A model of this approach is the Hispanic Health and
Nutrition Examination Survey conducted in 1982 by HHS.  HHS developed
HHANES in response to recommendations made by the National Academy
for Public Administration, which identified the Hispanic population
as growing, likely to have low income, and potentially at risk for
health and nutrition problems.  HHANES was conducted as a separate
study rather than integrated into the national survey because NHANES
II was already completed and funding was not available to conduct
HHANES as part of a national survey. 


         POTENTIAL STRENGTHS
------------------------------------------------------ Chapter 5:2.1.1

One of the advantages of conducting HHANES as a separate study was
the opportunity to change the content of the survey instrument to
address issues of special relevance to the Hispanic population.  For
example, unlike NHANES II, HHANES gathered information on health
services use and gallstone disease.  In addition, HHS took steps to
address the cross-cultural issues of applying HHANES to different
Hispanic populations.  The survey instrument was translated into the
idiomatic Spanish of each of the three Hispanic groups surveyed
(Cuban-Americans, Mexican-Americans, and Puerto Ricans) and an
appropriate plan for reaching out to the respondents was developed. 
For example, unlike the regular NHANES, which relies primarily on
press releases and the formal leaders (such as mayors) of the places
they have sampled to communicate the importance of participation in
the survey, HHANES used informal leaders (such as church leaders) and
Spanish-language media.  The opportunity to tailor a special survey
to the population of interest is an advantage of the approach, but it
can be complicated for a population whose members speak multiple
distinct languages and have varying degrees of assimilation into the
U.S.  population. 

Although not demonstrated by HHANES, another advantage of a special
study is the ability to study populations that are not appropriately
studied through oversampling.  As described above, this includes
groups that are not included in the national sampling frame, such as
homeless persons, and groups that are geographically dispersed or not
well represented in the general population, such as pregnant and
lactating women. 


         POTENTIAL LIMITATIONS
------------------------------------------------------ Chapter 5:2.1.2

Unlike oversampling, in which the subpopulation is surveyed at the
same time and usually with the same procedures as the population as a
whole, the data collected by special studies may not be comparable to
the population as a whole.  For example, differences found between
HHANES data on Hispanic groups and NHANES data on the nation could be
the result of national changes in health and nutrition status during
the gap between the two surveys rather than actual differences
between the groups.  Of course, conducting special studies in tandem
with a national survey is possible.  In fact, HHS is considering
conducting special subpopulation surveys at the same time as the next
NHANES. 

Special studies that are conducted in addition to the national
surveys clearly add to the overall cost of data collection.  Specific
costs for surveys of ethnic populations include bilingual
interviewers, the translation of the survey instrument, and outreach
to the group.  Other costs are the development of a separate sampling
frame and screening potential respondents to identify those that
belong to the group of interest.  Finally, if a special study is
conducted at the same time as the national survey, the burden on the
survey support facilities, such as the laboratories that analyze
blood and urine specimens, is increased, which may slow down the
processing and release of the data. 

If a special study is conducted because the group of interest is not
well covered by the national sampling frame, it needs a sampling
frame that allows for generalization to the subpopulation group,
which may be difficult to construct.  For example, to use survey
results to draw conclusions about the population of people who are
without homes, one needs to sample from a complete list of the
members of that group.  For the homeless population, such a list
would be very costly to construct.  As a result, other means--such as
sampling shelters--would have to be used.  The survey results,
though, would probably not be generalizable to the overall population
of homeless people because these other means are likely to be
incomplete. 


      INDIRECT ESTIMATION
-------------------------------------------------------- Chapter 5:2.2

Indirect estimation (also known as small area or synthetic
estimation) refers to procedures that use values of the variable of
interest from an area or time other than the area and time of
interest.  For example, to develop an indirect estimate of the
prevalence of iron-deficiency anemia in a county, the national
estimate of iron-deficiency anemia can be adjusted based on the
county's demographic profile.  Both USDA and HHS have experience with
indirect estimation.  Since the early part of the century, USDA has
had a program to develop indirect estimates of crop yields.  Although
some information is available from state surveys of nonprobability
samples of farmers, the USDA program adjusts these less dependable
estimates so that they aggregate to the more reliable regional and
national estimates that are based on a survey of a national
probability sample of farmers.  HHS has no regular program to produce
indirect state estimates, but since 1968, it has supported the
occasional development and evaluation of indirect estimates from
National Health Interview Survey data.  In addition, HHS produced
state estimates from the National Natality and National Fetal
Mortality Surveys conducted in 1980 using demographic data from the
states to adjust the national estimates.  Indirect estimation models
range from the relatively straightforward adjustments of national or
regional estimates to match local demographic profiles to more
complex models. 


         POTENTIAL STRENGTHS
------------------------------------------------------ Chapter 5:2.2.1

The major strength of indirect estimation is that, compared to
increasing survey sample sizes to obtain data for direct estimates at
state and local levels, it is far less costly.  In addition, it is a
means of extending the usefulness of costly national survey data to
inform decisions made at state or local levels of government. 
Moreover, it is an established approach.  Indirect estimates of such
variables as state and local populations, employment and
unemployment, and crop yields are already used by the federal
government in formulas for determining eligibility and benefit
amounts for federal programs.  Some state governments have also used
indirect estimation to conduct analyses for economic and other types
of programs. 

Indirect estimation also responds to limitations of the data on which
direct estimates might be based.  Program records, such as those used
by PedNSS and PNSS, have the advantage of timeliness since they are
usually collected continuously, but their relevance may be limited
because the data are collected for specific administrative purposes,
not just for nutrition monitoring.  Sample survey data, however, have
the advantage of relevance, but the data can be costly to obtain at a
level of detail that will support estimates for states, localities,
or other subpopulation groups.  In contrast, indirect estimation is
an approach to producing timely, relevant, and detailed information
without a major increase in cost. 


         POTENTIAL LIMITATIONS
------------------------------------------------------ Chapter 5:2.2.2

The major limitation of indirect estimation is the difficulty of
determining the quality of the estimate.  The best way is to compare
it to the true value for the population.  For example, comparing an
indirect estimate of a county's population to census data on the
county's population would enable an assessment of the bias of the
estimate.  However, since indirect estimation is used when data are
not available for direct estimation, such a comparison is usually not
feasible.  Moreover, the quality of the estimates yielded by a model
changes for different populations and for different times.  In other
words, even if a model yields estimates that prove to be unbiased in
comparison to direct observations for the same year, estimates from
the same model for subsequent years may be biased if the
relationships between the variables change over time. 

This limitation has an implication for the use of indirect estimates. 
Specifically, indirect estimates may be difficult to defend in the
political arena because they are based on models rather than direct
observations.  However, in the absence of direct data from other,
more expensive approaches, indirect estimation is preferable to no
information at all. 

Although indirect estimation has been used successfully with other
federal surveys, there are constraints on the development of indirect
estimation programs for the major nutrition monitoring surveys. 
According to USDA and HHS staff, major obstacles include the lack of
staff resources to support the program and the complexity of the
surveys.  Despite these concerns, both agencies have long experience
with indirect estimation in other arenas, which could be applied to
nutrition monitoring. 


   CONCLUSIONS
---------------------------------------------------------- Chapter 5:3

Information on subpopulation groups and small geographic areas is
used to identify nutrition-related problems that are associated with
specific populations and to target programs to those most at risk. 
Different strengths and weaknesses are associated with the four
approaches to meeting this need that we reviewed.  Surveillance
systems and indirect estimation programs are likely to be less
expensive than oversampling or special surveys.  However,
oversampling and special surveys can yield more detailed information
than the surveillance systems that rely on program records and more
dependable estimates than indirect estimation models.  Of the four
approaches, only indirect estimation has not been a part of NNMRRP
activities at one time or another.  From programs in other areas,
this approach appears to be a potentially efficient means of
expanding the information available on subpopulations and small
geographic areas.  However, before the Interagency Board can
determine what priority to give to this promising approach, a
complete picture of what needs would be met by each option is
required. 


APPROACHES TO ASSISTING STATES AND
LOCALITIES
============================================================ Chapter 6

State and local governments are interested not only in the
applicability of federal nutrition monitoring data to their
jurisdictions, but also in having federal assistance in collecting
and interpreting their own data.  This chapter follows on the
previous discussion about NNMRRP support for reliable inferences
about state and local populations to describe current and potential
options for assisting states and localities in their own monitoring
activities. 

The NNMRRP currently assists states through the state-based
surveillance systems, which provide technical and other assistance to
states.  The alternative that we examined, community-based nutrition
monitoring, is a response to states and localities that are
interested in building their capacity to collect their own data.  The
strengths and weaknesses of these two approaches to assisting states
and localities are summarized in table 6.1 and detailed below. 



                                    Table 6.1
                     
                        Approaches to Assisting States and
                                    Localities

Approach                   Strength                   Limitation
-------------------------  -------------------------  --------------------------
Current: state-based       Balance between federal    Limited flexibility for
surveillance systems       need for standardization   state and local
                           and state need for         governments
                           tailoring
                                                      Limited support for local
                           Development of states'     use of the data
                           capacity for data
                           collection and
                           interpretation

                           Potential for
                           improvements on current
                           foundation

Potential: community-      Development of local       Technical and financial
based nutrition            capacity for collecting    resources required to
monitoring                 and using data tailored    assess needs or implement
                           to community concerns      responses to needs

                           Limited federal            Tension between federal
                           investment                 interest in
                                                      standardization and local
                                                      interest in flexibility
--------------------------------------------------------------------------------
Other suggestions by our expert panelists--such as providing
financial and technical assistance, developing standardized modules
of interest for state data collection activities, and assisting state
collection of data on subpopulations--can be implemented as part of
either of the two approaches. 


   CURRENT APPROACH:  THE
   STATE-BASED SURVEILLANCE
   SYSTEMS
---------------------------------------------------------- Chapter 6:1

The strengths and limitations of the surveillance systems as sources
of continuous data and of information on state and other population
groups have been discussed in chapters 4 and 5.  However, they have
additional strengths and limitations related to their ability to meet
state and local needs for assistance. 


      STRENGTHS
-------------------------------------------------------- Chapter 6:1.1

Surveillance systems currently balance the federal interest in
information that is collected in a standard format across states and
the states' needs for flexibility.  For example, with BRFSS, HHS
supplies states with standardized modules of questions, training in
collecting the data, and support in analysis and reporting.  In
addition, changes to the survey content are made in consultation with
state participants in the surveillance system, and states have the
opportunity to add their own questions to the survey. 

The surveillance systems have also played a role in building state
capacity for data collection and analysis.  Specifically, some states
have used their experience with BRFSS to implement their own
telephone surveys using the random digit dialing procedure.  In
addition, standardized software developed by HHS for PNSS has enabled
states to generate their own reports. 

Yet another strength of the surveillance systems is the foundation of
federal-state partnership it provides for future improvements of
federal technical assistance to state and local nutrition monitoring
activities.  For example, as mentioned in previous chapters, HHS is
exploring ways to use the surveillance systems to gather additional
nutritional data, such as more in-depth information on dietary
intake, and to cover new populations, such as schoolchildren. 
Additional technical assistance in data analysis and interpretation
could also be provided through the surveillance system structure. 


      LIMITATIONS
-------------------------------------------------------- Chapter 6:1.2

While the surveillance systems could be expanded and improved to
further respond to the interest of states and localities in receiving
more technical assistance, they are limited in their flexibility
because of the federal interest in standardization across states.  In
addition, users of surveillance system data who responded to our
survey identified some concerns about the surveillance systems. 
Specific issues included the availability of the data to localities,
timeliness of HHS' processing of the data, and the formats of the
reports that HHS provided.  Recommendations for improving the
surveillance systems included increasing local access to data,
reducing the time it takes HHS to process the data, simplifying
reports for local users, and providing additional technical and
financial assistance in data collection and interpretation. 


   POTENTIAL OPTION: 
   COMMUNITY-BASED NUTRITION
   MONITORING
---------------------------------------------------------- Chapter 6:2

The federal-state linkage forged by the surveillance systems could be
further extended to support community-based nutrition monitoring.  To
explore the strengths and limitations of this approach, we reviewed
the literature on two models of community-based programs.  HHS has
funded Planned Approach to Community Health (PATCH) projects, which
used the BRFSS survey instrument as the basis for a community needs
assessment.  The survey data, in combination with interviews with
knowledgeable informants in the community, were used to plan health
promotion programs.  HHS provided technical assistance in the needs
assessment portion of the projects and small awards of a few thousand
dollars for project activities. 

Researchers at Cornell University developed a similar model
specifically for nutrition monitoring that was pilot-tested in three
New York counties with funding from the State Department of Health
and technical assistance from the university.  In their approach, a
coalition of interested community members first articulates potential
information needs.  Then, the group selects specific needs on the
basis of the likelihood that the data will be used by the community. 
To meet the selected information needs, feasible sources of routinely
available data are identified.  According to the model, data
collection should depend on procedures that are already in place,
from such sources as program participation records, patient charts,
and school health screenings.  In its reliance on program records,
this approach is similar to PedNSS and PNSS.  However, there is no
expectation that the same issues will be targeted or the same sources
of data used in each community.  To facilitate the final component of
the system--the communication of monitoring results--a network of
users must be cultivated and the appropriate vehicle for
communicating the results must be used.  Drawing on the experience of
these projects and HHS' PATCH program, the following strengths and
limitations of community-based nutrition monitoring were identified. 


      POTENTIAL STRENGTHS
-------------------------------------------------------- Chapter 6:2.1

Evaluations of both types of community-based programs found evidence
that local capacity for collecting and using data was developed.  By
involving community members in assessing needs, PATCH built skills in
identifying health risks and cultivated community and organizational
supports for health promotion programs.  In the New York counties
that pilot-tested the Cornell model of community-based nutrition
monitoring, data were collected and compiled to describe access to
food and nutrition services and nutritional health of county
residents.  To disseminate the information and guide decisions based
on the data, interagency coalitions were formed.  These coalitions
brought together local nutrition-related professionals, so that
nutrition interventions were better coordinated and information was
shared. 

The limited federal investment in the community-based programs is
another strength.  HHS supported PATCH with small grants and
technical assistance, while the community-based nutrition monitoring
projects were sponsored by the state and received technical
assistance from the state's land grant institution. 

Moreover, there is some evidence that the activities put in place by
these projects will continue.  For example, Cornell reports that

     "At the end of the [3-year] pilot stage, one county obtained
     local funding to continue their monitoring activities, including
     the issue-based coalitions; in the other counties, local
     nutrition councils are coordinating continued monitoring
     efforts."\1

The program has also spread to other counties, some of which have
initiated monitoring activities without outside funding. 


--------------------
\1 "Community-Based Nutrition Monitoring," Division of Nutritional
Sciences, Cornell University, and Division of Nutrition, N.Y.  State
Department of Health, Mar.  1993. 


      POTENTIAL LIMITATIONS
-------------------------------------------------------- Chapter 6:2.2

The experience with community-based programs indicates that
communities require a set of resources--specifically, technical
skills and dedicated personnel--to fully benefit from the projects. 
One evaluation of PATCH found that the projects were most effective
in communities that already had human services and health
professionals who were involved in community health promotion
efforts.  In addition, the projects that had directors seemed to have
the greatest implementation successes.  Both kinds of projects
required considerable time and effort to collect and interpret data. 
Moreover, while HHS provided technical assistance in using the BRFSS
survey in the local needs assessments, additional assistance was
needed to help communities set priorities based on the data. 

The reliance on BRFSS was a limitation for PATCH because BRFSS did
not necessarily include the issues of primary concern for the
community.  For example, one PATCH project was specifically
interested in water quality, which is not addressed in BRFSS.  In
contrast, the Cornell model emphasized local sources of data.  Based
on the initial experience with PATCH, HHS no longer expects
communities to use BRFSS for its needs assessment. 


   CONCLUSIONS
---------------------------------------------------------- Chapter 6:3

Data from the HHS surveillance systems are already extensively used
in state and local program planning.  In addition, the systems are
flexible and are exploring ways to increase the information they
collect on dietary intake.  However, they seem less responsive to the
needs of localities than to the needs of states.  Recent efforts to
build local capacity for data collection and interpretation indicate
that community-based programs are a promising approach to responding
to local needs for nutrition information.  Before the Interagency
Board can decide what priority to place on community-based nutrition
monitoring, however, it must first identify the objectives of the
NNMRRP that would be furthered and the importance of these objectives
relative to others that are competing for program resources. 


LIST OF EXPERTS
=========================================================== Appendix I

This appendix lists the expert advisers who assisted with this
project.  As described in chapter 1, the advisers were organized into
three panels--core policy panel, methodology panel, and data users
panel. 


      CORE POLICY PANEL
------------------------------------------------------- Appendix I:0.1

Johanna Dwyer, D.Sc., R.D., Francis Stern Nutrition Center, New
England Medical Center, and Tufts University Schools of Medicine and
Nutrition

Jean-Pierre Habicht, M.D., Ph.D., Division of Nutritional Sciences,
Cornell University

Catherine Woteki, Ph.D.\1


--------------------
\1 Dr.  Woteki withdrew from the panel when she was appointed to the
Office of Science and Technology Policy in the White House.  During
her participation in our work, she was the Director of the Food and
Nutrition Board of the National Academy of Sciences. 


      METHODOLOGY PANEL
------------------------------------------------------- Appendix I:0.2

Norman Bradburn, Ph.D., Director, National Opinion Research Center

Marilyn Buzzard, Ph.D., Director, Nutrition Coordinating Center,
University of Minnesota

Ricardo O.  Castillo, M.D., M.P.H., Co-Director, Pediatric
Gastroenterology, Stanford University Medical Center

Alan R.  Kristal, Dr.P.H., Fred Hutchinson Cancer Research Center and
Department of Epidemiology, University of Washington

James Lepkowski, Ph.D., Institute for Social Research, University of
Michigan

Cheryl Ritenbaugh, Ph.D., Department of Family and Community
Medicine, University of Arizona

Laura Sims, Ph.D., Department of Nutrition and Food Science,
University of Maryland


      DATA USERS PANEL
------------------------------------------------------- Appendix I:0.3

Elizabeth Barnett, Ph.D., North Carolina Department of Environment,
Health, and Natural Resources

Doris Disbrow, Dr.P.H., R.D., Center for Health Education

Pamela Haines, Dr.P.H., R.D., Department of Nutrition, University of
North Carolina

Jay Hirschman, M.P.H., Senior Analyst, Food and Consumer Service,
U.S.  Department of Agriculture

Karen J.  Morgan, Ph.D., Senior Director, Nutrition and Consumer
Affairs, Nabisco Brands

Barbara Petersen, Ph.D., Technical Assessment Systems


SOURCES FOR THE MODEL FEATURES
========================================================== Appendix II

The model features that are the focus of this report were derived
from four sources.  First, we reviewed previous evaluations, from
which 13 historical criticisms of federal nutrition monitoring were
identified.  Second, we considered the objectives and related
activities outlined in the 10-year comprehensive plan developed by
the Interagency Board.  Third, through an iterative process of panel
meetings and review of written materials, a number of suggestions for
improvements to the NNMRRP were generated by the expert advisers. 
Fourth, the data users who responded to our survey provided written
comments describing what changes to specific data collection
activities would increase their use of the data. 


   PAST EVALUATIONS OF FEDERAL
   NUTRITION MONITORING
-------------------------------------------------------- Appendix II:1

Table II.1 lists the criticisms identified in past evaluations and
categorizes them by the features of a model program that they
suggest.  (See Nutrition Monitoring (GAO/PEMD-94-23), pp.  5-6.)




                                    Table II.1
                     
                     Features of a Model Program Suggested by
                                 Past Evaluations

Feature                                  Past criticism
---------------------------------------  ---------------------------------------
Coordinated system that responds         Lack of coordination among nutrition
efficiently to the diverse needs of the  monitoring activities
data users

                                         Information needs of users not
                                         systematically determined

                                         Lack of compatibility in methods for
                                         assessing dietary intake

                                         Core set of standardized measures not
                                         yet developed for major surveys

                                         Compatible sampling techniques not used
                                         for national surveys

                                         Reporting by national surveys not
                                         integrated

Continuous or more frequent collection   Data not collected continuously
of data

Support for reliable inferences about    Specific population groups not covered
subpopulation groups and small           by major surveys
geographic areas

                                         Specific geographic areas not
                                         represented by major surveys

Assistance to states and localities      State and local data needs not fully
                                         addressed

Improved methodology for assessing       Improvement needed in methodology for
dietary intake and nutritional status    assessing dietary intake and
                                         nutritional status

Low respondent burden and high response  Sampling problems with NFCS
rates

Timely processing and dissemination of   Need for more timely dissemination of
data                                     survey information
--------------------------------------------------------------------------------

   THE 10-YEAR COMPREHENSIVE PLAN
-------------------------------------------------------- Appendix II:2

Table II.2 provides examples from the 68 activities listed in the
10-year comprehensive plan that correspond to the desirable features
identified from the review of previous evaluations of federal
nutrition monitoring.  (Other desirable features suggested by the
10-year plan activities but not addressed in this report focus on
strengthening the food composition and food supply data systems.)



                                    Table II.2
                     
                      Examples of Planned NNMRRP Activities
                          Associated With Model Features

Feature                                  Planned activity\a
---------------------------------------  ---------------------------------------
Coordinated system that responds         Coordinate the planning for coverage,
efficiently to the diverse needs of the  tracking, and reporting of findings
data users                               from surveys and surveillance systems

                                         Identify ways to increase comparability
                                         within a dietary method to improve the
                                         quality and usefulness of data

                                         Establish a mechanism for improved
                                         coordination among federal agencies
                                         that collect and use survey information
                                         about knowledge, attitudes, and
                                         behavior to assess gaps and
                                         duplications in existing surveys

Continuous or more frequent collection   No specific activities listed\b
of data

Support for reliable inferences about    Develop and implement a plan for
subpopulation groups and small           improved coverage of subgroups of the
geographic areas                         population at nutritional risk or at
                                         risk for underconsumption or
                                         overconsumption of nutrients and food
                                         components

                                         Evaluate the feasibility of alternate
                                         statistical methodologies for creating
                                         state and local estimates

Assistance to states and localities      Provide assistance for the development
                                         and maintenance of state structure,
                                         staff, and programs to support their
                                         participation in NNMRRP

                                         Expand the coverage of current state
                                         and local nutrition monitoring
                                         activities in selected population
                                         groups through technical assistance and
                                         grant awards

Improved methodology for assessing       Conduct research to develop, improve,
dietary intake and nutritional status    and validate laboratory measures of
                                         nutritional status

                                         Develop and evaluate procedures for
                                         determining usual intakes of foods and
                                         nutrients from surveys employing 24-
                                         hour recall measures of dietary intake

Low respondent burden and high response  No specific activities listed\c
rates

Timely processing and dissemination of   No specific activities listed\d
data
--------------------------------------------------------------------------------
\a See the Ten-Year Comprehensive Plan, 58 Fed.  Reg.  111 (June 11,
1993), pp.  32752-806. 

\b Although no specific activities are planned to ensure the
continuous collection of data, two of the six objectives include
continuous data and one of the activities mentions coordinating the
timing of surveys. 

\c USDA has separated the household and individual portions of the
NFCS survey in order to reduce respondent burden and improve response
rates.  In addition, USDA signed interagency agreements with the
Bureau of Census for assistance in designing and conducting the
individual and household surveys, as well as for research on
improving methods for collecting household food use data. 

\d Although none of the activities identified under the objectives
specifically address the timeliness of data processing and
dissemination, part of the overarching goal stated for the plan is
"efficiently disseminating and exchanging information with data
users." In addition, the plan discusses possible mechanisms for
increasing awareness and dissemination of the data and states that
the Interagency Board plans to establish a central clearinghouse. 


   CONSULTATION WITH EXPERT
   ADVISERS
-------------------------------------------------------- Appendix II:3

The expert advisers to the project (listed in appendix I) provided
guidance by reviewing materials and participating in panel meetings. 
Through this process, several suggestions were generated, and their
advantages and disadvantages were discussed.  The suggestions
outlined below in table II.3 do not indicate consensus among the
panelists. 



                                    Table II.3
                     
                      Advisory Panel Suggestions Associated
                               With Model Features

Feature                                  \Suggested approach\
---------------------------------------  ---------------------------------------
Coordinated system that responds         A single lead agency
efficiently to the diverse needs of the
data users

                                         An interagency body with permanent
                                         staff and enforcement authority

                                         Location in statistical agencies within
                                         the user departments or within a
                                         central statistical agency

                                         Centralization of the congressional
                                         appropriations process for nutrition
                                         monitoring activities

                                         Informed review of data collection
                                         plans by qualified people at OMB

Continuous or more frequent collection   Continuous national nutrition surveys
of data

                                         Addition of modules to existing surveys

                                         Reliance on program data that are
                                         already collected

                                         Collection of longitudinal data

Support for reliable inferences about    Surveys of population subgroups instead
subpopulation groups and small           of national-level surveys
geographic areas

                                         Different sampling strategies for
                                         different populations (for example,
                                         list-based sampling for populations
                                         that are rare in a broad-based sample)

                                         Contracts with states and localities to
                                         gather information on geographically-
                                         based populations

                                         Indirect estimation to support
                                         inferences about population subgroups
                                         and small geographic areas

Assistance to states and localities\a    Financial assistance to states to
                                         determine their data needs

                                         A federal-state partnership in which
                                         states can provide funds for some extra
                                         sampling or extra questions on federal
                                         surveys

                                         Federal development of standardized
                                         modules of interest for state data
                                         collection activities

                                         Federal assistance to state collection
                                         of data on subpopulations

                                         Technical assistance in data
                                         interpretation

Improved methodology for assessing       Continued or increased research on
dietary intake and nutritional status    nutrition monitoring methods

                                         Postmarket surveillance data (such as
                                         that gathered through bar code
                                         scanners)

                                         Automated collection of dietary data

                                         Reliance on program records

Low respondent burden and high response  Provide more feedback to respondents
rates                                    about survey results

Timely processing and dissemination of   Assistance to data users in
data                                     interpreting and analyzing the data

                                         Development of core analysis data sets
                                         that are focused on specific issues or
                                         groups

                                         A data clearinghouse

                                         Automated data collection
--------------------------------------------------------------------------------
\a The panelists disagreed about the responsibility states should
have for data collection, with some arguing for state-based data
collection that feeds into a federal system and others arguing for
less state responsibility for data collection, but increased
consideration of state needs in federal data collection activities. 

In addition to comments that could be categorized by the model
features suggested by the historical criticisms, the expert advisers
also had other suggestions for improving the system, including

  -- develop an ongoing evaluation of the system's content and
     methods, including a review of the information needs;

  -- continue to collect both household and individual level data;

  -- have a system to maintain data comparability over time; and

  -- use longitudinal designs to measure change. 


   SUGGESTIONS FROM THE SURVEY OF
   DATA USERS
-------------------------------------------------------- Appendix II:4

The comments that users of nutrition monitoring data provided in
response to our survey were analyzed to identify major themes for
each of three groups of data collection activities--USDA surveys, HHS
surveys, and HHS state-based surveillance systems.  Table II.4
identifies the themes associated with the features of a model
program.  More detailed summaries can be found in Nutrition
Monitoring (GAO/PEMD-95-15). 



                                    Table II.4
                     
                     Major Themes in the Suggestions Made by
                                    Data Users


                                                            HHS surveillance
Model feature       USDA surveys        HHS surveys         systems
------------------  ------------------  ------------------  --------------------
Coordinated system  \a                  \a                  \a
that responds
efficiently to the
diverse needs of
the data users

Continuous or more  Continuous or more  Continuous or more  \b
frequent            frequent data       frequent data
collection of data  collection          collection

Support for         Increased sample    More detail on      More detail on
reliable            sizes and           racial, ethnic,     subpopulation groups
inferences about    broadened           and age groups      in the reporting of
subpopulations      coverage                                the data
geographic areas                        Data that can
                    More detail on      support estimates   Increased ability to
                    racial, ethnic,     for small           look at substate
                    age, and income     geographic areas    geographic divisions
                    groups

                    Refined geographic
                    coverage,
                    specifically state
                    and substate data

Assistance to       \c                  \c                  Simplified reports
states and                                                  that are more
localities                                                  readily used at the
                                                            local level

                                                            Additional technical
                                                            and financial
                                                            assistance in data
                                                            collection and
                                                            interpretation

                                                            Better controls on
                                                            the quality of the
                                                            data collected

Improved            More specificity    More information    More data on dietary
methodology for     and detail about    on health habits    intake
assessing dietary   foods and better    and outcomes
intake and          data on
nutritional status  food composition    More detailed data
                                        on food
                    Improved questions  consumption
                    about dietary
                    behavior            Improved dietary
                                        intake methods
                    More information
                    about health and
                    demographic
                    variables

                    Higher quality
                    dietary recall
                    data

Reduced respondent  Reduced respondent  \d                  \d
burden and          burden and
improved response   improved response
rates               rates

Timely processing   Improved            Improved            Improved timeliness
and dissemination   timeliness and      timeliness and      of HHS' return of
of data             documentation of    documentation of    the data to states
                    the data            the data

                    Dissemination of    Dissemination of
                    the data in         the data in
                    formats that        formats that
                    facilitate access   facilitate access
                    and analysis        and analysis
--------------------------------------------------------------------------------
\a Coordination was not a major theme of the comments provided in
response to our survey of data users probably because the survey
queried respondents about changes to specific data collection
activities rather than NNMRRP as a whole. 

\b Users of surveillance system data were unlikely to identify
continuous data as a major theme since these systems collect data
continuously. 

\c Because our survey asked for comments about the two data
collection systems most frequently used by the respondent, state and
local users were more likely to comment on the surveillance systems. 
As a result, their interest in changing the national surveys to
better meet their needs was not gauged. 

\d The HHS systems have not suffered the same response rate problems
that USDA's 1987-88 NFCS experienced. 

The themes reflect the strengths and weaknesses of the different data
collection activities.  For example, response rates were probably a
major issue to users of USDA surveys because of the response rate
problems of NFCS.  Similarly, assistance to states and localities was
a theme from the users of the surveillance systems, many of whom
manage state and local programs.  In addition to the themes that
correspond to the features of a model program, other major issues in
the suggestions from the data users were the collection of
longitudinal data (from users of USDA survey data) and the collection
of both individual and household data (from users of data from USDA
and HHS surveys). 




(See figure in printed edition.)Appendix III
COMMENTS FROM THE DEPARTMENT OF
AGRICULTURE
========================================================== Appendix II



(See figure in printed edition.)



(See figure in printed edition.)



(See figure in printed edition.)



(See figure in printed edition.)


The following are GAO's comments in response to the June 13, 1995,
letter from the Department of Agriculture. 

1.  What USDA finds inconsistent with the report reflects the
inconsistency of NNMRRP.  The 10-year plan, a natural location for a
statement of priorities within expected funding levels, specified
neither priorities nor expected funding levels.  As noted in our
report, the later priority-setting exercise was an improvement but
was also limited in its utility because it did not specify whether
data needs and feasibility were considerations in ranking the
objectives. 

2.  The central contact for data users that is suggested by the
alternate approaches to coordination entails much more than a
clearinghouse for data products.  For example, the Office of National
Drug Control Policy acts not only as a source of information for
state and local drug enforcement interests, but also as a conduit of
information about state and local concerns to the federal agencies. 

3.  USDA makes several recommendations on how the Board can be
improved.  These recommendations are consistent with the strengths
that we identified in the other approaches to coordination. 

4.  The Consumer Expenditure Survey also provides information about
the cost of food used at the household level.  However, because of
the detail about the expenditures on specific foods gathered by the
NFCS, we have amended the description of the NFCS. 

5.  Following the classification provided by the NNMRRP document, The
Directory of Federal and State Nutrition Monitoring Activities, CSFII
and DHKS focus on food and nutrient consumption.  Table 1.1 indicates
that data on food and nutrient consumption are used for health- as
well as food-related purposes. 

6.  The sentence has been changed to reflect this information. 

7.  Because the coordinated NNMRRP budget reports amounts dedicated
to nutrition monitoring by agency rather than by purpose, the
accounting system used by ARS is not relevant.  All NNMRRP
expenditures by ARS will be reported together, whether they are
dedicated to surveys or related research. 

8.  The fact that the plan was intentionally vague does not diminish
the degree to which the lack of detail limits its utility as a
statement about the specific objectives of the program, the
activities planned to meet the objectives, and the resources needed
to support the plans. 

9.  While technical expertise can be supplied by staff, only members
with organizational authority can make decisions about agency
priorities and resources.  USDA's comment regarding p.  ES-7
(responded to in comment 3 above) suggests the importance of having
members with budget decision-making authority. 

10.  The statement has been changed to reflect this information. 




(See figure in printed edition.)Appendix IV
COMMENTS FROM THE PUBLIC HEALTH
SERVICE
========================================================== Appendix II



(See figure in printed edition.)



(See figure in printed edition.)



(See figure in printed edition.)


The following are GAO's comments on the June 16, 1995, letter from
the Public Health Service. 

1.  As PHS states, the 10-year plan does plan to conduct evaluations
in 1997 and 2002.  However, neither these evaluations nor the Board's
annual summaries of progress are a strategy for weighing the costs
against the benefits of current or new activities.  A framework for
comparing existing and potential monitoring activities in such terms
as their feasibility, ability to meet data needs, and other
considerations is still needed. 

2.  The paragraph referred to (now on pp.  34-35) is not about the
progress of the working groups, but about the guidance provided by
the 10-year plan for the program as a whole.  While the progress
reports indicate specific examples of improvements within working
groups, the 10-year plan does not constitute a comprehensive
proposal.  Such a proposal would include specific objectives,
detailed plans for how to meet the objectives, and cost estimates and
funding requests to support the plans.  The availability of resources
affects the ability to meet an objective, but is not a reason for not
developing specific plans in the first place. 

3.  In contrast to such coordinating bodies as the Interagency Board
and the National Library of Medicine, the central authorities that we
reviewed have personnel who are not detailed from the agencies. 
While the potential disadvantage that PHS suggests (that is,
unfamiliarity with NNMRRP) can be easily overcome, the advantage of
independence from the interests of any single agency is difficult to
achieve in the other approaches. 

4.  The sentence has been changed to clarify that, with current
levels of state participation, the PedNSS and PNSS can not be used to
provide nationally representive data of the low-income mothers and
children receiving assistance through the WIC program. 

5.  This section has been changed to reflect the lack of coverage of
American Indians and Alaska Natives living on reservations by the
national household sampling frame. 


MAJOR CONTRIBUTORS TO THIS REPORT
=========================================================== Appendix V


   PROGRAM EVALUATION AND
   METHODOLOGY DIVISION
--------------------------------------------------------- Appendix V:1

John Oppenheim, Assistant Director
Leslie Riggin, Project Manager
Lï¿½ Xuï¿½n Hy, Adviser
Venkareddy Chennareddy, Referencer
Elizabeth W.  Scullin, Communications Analyst




RELATED GAO PRODUCTS
============================================================ Chapter 1

Nutrition Monitoring:  Data Serve Many Purposes; Users Recommend
Improvements (GAO/PEMD-95-15; June 20, 1995). 

High Performance Computing and Communications:  New Program Direction
Would Benefit From a More Focused Effort (GAO/AIMD-95-6; Nov.  4,
1994). 

Nutrition Monitoring:  Progress in Developing a Coordinated Program
(GAO/PEMD-94-23; May 27, 1994). 

Food Nutrition:  Better Guidance Needed to Improve Reliability of
USDA's Food Consumption Data (GAO/RCED-94-30; Oct.  25, 1993). 


*** End of document. ***