Agricultural Conservation: USDA Should Improve Its Methods for
Estimating Technical Assistance Costs (30-NOV-04, GAO-05-58).
The U.S. Department of Agriculture's (USDA) Natural Resources
Conservation Service (NRCS), working with state and local
partners, provides landowners with technical assistance for
multiple programs to plan and implement conservation measures
that protect soil, water, and wildlife. For years, the Congress
has been seeking detailed cost information on this assistance as
it examined USDA budget requests. In part, because NRCS's
financial system was not designed for estimating future budgets,
in 1998 NRCS began developing additional cost data and a computer
model for estimating future technical assistance costs. GAO was
asked to (1) review NRCS's technical assistance cost estimates
and (2) identify causes of any differences between the estimates
and actual costs ultimately reported by NRCS.
-------------------------Indexing Terms-------------------------
REPORTNUM: GAO-05-58
ACCNO: A13996
TITLE: Agricultural Conservation: USDA Should Improve Its
Methods for Estimating Technical Assistance Costs
DATE: 11/30/2004
SUBJECT: Administrative costs
Agricultural programs
Budget outlays
Comparative analysis
Computer modeling
Conservation
Cost analysis
Financial management
Financial management systems
Future budget projections
Technical assistance
Work measurement
Program costs
Workloads
Cost estimates
******************************************************************
** This file contains an ASCII representation of the text of a **
** GAO Product. **
** **
** No attempt has been made to display graphic images, although **
** figure captions are reproduced. Tables are included, but **
** may not resemble those in the printed version. **
** **
** Please see the PDF (Portable Document Format) file, when **
** available, for a complete electronic file of the printed **
** document's contents. **
** **
******************************************************************
GAO-05-58
United States Government Accountability Office
GAO Report to the Committee on Agriculture, House of Representatives
November 2004
AGRICULTURAL
CONSERVATION
USDA Should Improve Its Methods for Estimating Technical Assistance Costs
a
GAO-05-58
Highlights of GAO-05-58, a report to the Committee on Agriculture, House
of Representatives
The U.S. Department of Agriculture's (USDA) Natural Resources Conservation
Service (NRCS), working with state and local partners, provides landowners
with technical assistance for multiple programs to plan and implement
conservation measures that protect soil, water, and wildlife. For years,
the Congress has been seeking detailed cost information on this assistance
as it examined USDA budget requests. In part, because NRCS's financial
system was not designed for estimating future budgets, in 1998 NRCS began
developing additional cost data and a computer model for estimating future
technical assistance costs. GAO was asked to (1) review NRCS's technical
assistance cost estimates and (2) identify causes of any differences
between the estimates and actual costs ultimately reported by NRCS.
To improve NRCS's cost estimates, GAO recommends that the Secretary of
Agriculture direct the Chief of NRCS to identify the estimated costs
incurred by partners, ensuring that estimates are more comparable with
actual costs when tested, and modify the assumptions for estimating the
time that tasks take to better reflect actual work conditions. NRCS
generally agreed with the findings and recommendations and indicated it
would use them as the basis for making improvements to its estimation
methods.
November 2004
AGRICULTURAL CONSERVATION
USDA Should Improve Its Methods for Estimating Technical Assistance Costs
In 2003, NRCS started testing its computer model by comparing estimates of
technical assistance costs for 10 Farm Bill conservation programs, with
actual costs reported by NRCS. GAO's analysis of these comparisons shows
that NRCS's model made estimates, program-by-program, which varied
considerably from the agency's actual costs. For fiscal year 2003, for
example, NRCS's model estimated that the technical assistance costs for
seven Farm Bill programs would be higher by 9 to 50 percent, than NRCS
ultimately incurred. For three other Farm Bill programs, the estimates
were lower than the agency incurred by 16 to 60 percent. Most of the
estimates fell outside NRCS's goal of estimating to within 10 percent of
the agency's actual costs. In addition, for the 10 Farm Bill conservation
programs combined, NRCS estimated its technical assistance costs at $295
million for fiscal year 2003, which is about 15 percent more than the $257
million that NRCS incurred. NRCS officials generally agreed with this
analysis.
GAO identified several reasons for the differences between the cost
estimates and the actual costs.
o First, some of NRCS's technical assistance work was delayed, occurring
later than NRCS assumed when it estimated its costs. This contributed to
some overestimation by the model, according to NRCS officials.
o Second, NRCS's estimates include costs incurred by NRCS's partners.
Such costs are generally not included in the actual costs reported by
NRCS.
o Third, some data NRCS uses in its model are based on inaccurate
assumptions. For example, when developing estimates about the time it
takes NRCS staff to perform technical assistance tasks for use in the
model, NRCS assumes, among other things, that its staff are fully trained
and perform technical assistance work without interruption. These
assumptions do not reflect actual workplace conditions and lead to
underestimates. NRCS officials said they would reconsider these and other
assumptions.
www.gao.gov/cgi-bin/getrpt?GAO-05-58.
To view the full product, including the scope and methodology, click on
the link above. For more information, contact Lawrence J. Dyckman, (202)
512-3841, [email protected].
Contents
Letter
Results in Brief
Background
NRCS's Technical Assistance Cost Estimates Differ from Actual
Costs Reported by NRCS Differences in Estimated and Actual Costs Have
Several Causes Conclusions Recommendations Agency Comments and Our
Evaluation
1 4 5
9 11 16 17 17
Appendixes
Appendix I: Objectives, Scope, and Methodology 19
Appendix II: GAO Contacts and Staff Acknowledgments 22
GAO Contacts 22
Acknowledgments 22
Table 1: Comparison of Technical Assistance
Tables Costs Estimated by
NRCS's Model and Actual Costs Reported by NRCS,
Fiscal
Years 2002 and 2003 10
Table 2: Costs Estimated by NRCS's Model, with
and without
Partner Costs, Compared to Actual Costs
Reported by
NRCS, Fiscal Year 2002 13
Figure 1: Steps NRCS Has Taken to Develop a
Figures Method for Estimating Future Technical
Assistance Costs and
Budgets 7
Figure 2: Estimates and Data Used in NRCS's
Cost of Programs
Model and Discussed in This Report 8
Contents
Abbreviations
CRP Conservation Reserve Program
NRCS Natural Resources Conservation Service
USDA U.S. Department of Agriculture
WRP Wetlands Reserve Program
This is a work of the U.S. government and is not subject to copyright
protection in the United States. It may be reproduced and distributed in
its entirety without further permission from GAO. However, because this
work may contain copyrighted images or other material, permission from the
copyright holder may be necessary if you wish to reproduce this material
separately.
A
United States Government Accountability Office Washington, D.C. 20548
November 30, 2004
The Honorable Bob Goodlatte
Chairman
The Honorable Charles W. Stenholm
Ranking Minority Member
Committee on Agriculture
House of Representatives
The U.S. Department of Agriculture (USDA) conducts various conservation
programs that help protect our nation's soil, water, and wildlife. In
partnership with state and local agencies, USDA's Natural Resources
Conservation Service (NRCS) provides technical assistance to farmers,
ranchers, and others to help them take part in these conservation
programs. For example, through the Conservation Reserve Program (CRP),
the largest program for retiring farmland for conservation purposes, USDA
and its partners provide technical and financial assistance to eligible
farmers who contract with USDA to take such conservation actions as
planting native grasses in place of crops to prevent erosion.1 This
technical
assistance generally consists of making on-site visits to identify the
conservation needs of the land, advising landowners on selection, design,
and application of conservation measures, determining landowners'
eligibility to participate in the program, and developing and monitoring
contracts the agency signs with landowners to implement conservation
measures. In fiscal year 2003-the latest year for which data are
available-NRCS reported that it spent $257 million on technical assistance
to implement this and nine other farm-related conservation programs. In
future years, NRCS officials are anticipating growth in its conservation
programs, and that this growth will sharply increase the agency's
technical
assistance costs.2
1The Farm Service Agency administers this program, and NRCS and its
partners provide technical assistance.
2According to USDA, the Farm Security and Rural Investment Act of 2002
(the 2002 Farm Bill) authorized a $17.1 billion increase for conservation
over a 10-year period.
For years, the Congress has been seeking more detailed and accurate
technical assistance cost information to better understand NRCS's costs
and how program growth might affect the agency's budget. Although NRCS has
had historical and anecdotal information on its technical assistance
expenditures, its financial system has not contained the detailed
technical assistance cost information by program and contract that is
needed for estimating future program budgets. In response to congressional
concerns about this lack of information about the cost of NRCS technical
assistance work, NRCS began developing a method and the cost data needed
for estimating its future technical assistance costs and budgets in 1998.3
These efforts include
o a workload analysis process, which provides estimates of the time
required for technical assistance tasks, and
o a computer model, called the cost of programs model, which uses time
estimates from the workload analysis and other cost data about NRCS
salaries and expenses, for estimating NRCS's costs and future budgets.
In 2003, NRCS began using cost estimates from its model to provide USDA
and congressional decision makers with detailed information on the cost of
providing technical assistance.4 In 2004, NRCS also used its model to
estimate the agency's budget requirements. When technical assistance funds
exceeded actual costs, the excess funds were converted to and used as
financial assistance to landowners, according to NRCS officials. In
addition, to assess and improve the quality of the estimates made by the
model, NRCS compared the model's program cost estimates for fiscal years
2002 and 2003 to program costs reported by its information systems for
those fiscal years.
3Both congressional and executive officials need timely, accurate, and
consistent financial information to make informed decisions. The Congress
uses program cost estimates to choose among policies and alternative
program designs, and the administration uses these estimates to develop
and adjust program implementation plans. For example, see GAO, Budget
Issues: The Importance of Increased Accuracy of Budget Outlay Estimates,
GAO/AIMD-99-235R (Washington D.C.: Aug. 30, 1999).
4For example, NRCS gave the Congress estimates of the average cost to
provide technical assistance over the life of CRP and WRP contracts: about
$1,400 and $15,000, respectively. Technical assistance for WRP contracts
is more expensive than for CRP contracts because wetlands conservation
work is more complex and time consuming than farmland conservation work.
Because of your continuing interest in ensuring the development of
accurate information about the costs of NRCS programs, you asked us to (1)
review the technical assistance cost estimates produced by NRCS's model
and (2) identify the causes of any differences between the estimated costs
and the actual costs ultimately reported by NRCS.
To review the technical assistance cost estimates produced by NRCS's
model, we assessed NRCS's comparisons of its model results with costs
reported by NRCS for 10 Farm Bill conservation programs for fiscal years
2002 and 2003. The 10 conservation programs authorized or reauthorized by
the Farm Security and Rural Investment Act of 2002 (the 2002 Farm Bill)
and discussed in this report are the Agricultural Management Assistance
Program, the Conservation Reserve Program,5 the Environmental Quality
Incentives Program, the Farm and Ranch Lands Protection Program, the
Ground and Surface Water Conservation Program, Klamath Basin, the Wetlands
Reserve Program, the Wildlife Habitat Incentive Program, the Conservation
Security Program, and the Grassland Reserve Program. The Ground and
Surface Water Conservation Program, Klamath Basin, Conservation Security
Program, and Grassland Reserve Program are new programs authorized by this
legislation.
Concerning one key source of the cost information, NRCS's financial
system, we noted, among other things, that USDA obtained an unqualified
opinion on its financial management activities in fiscal years 2002 and
2003. This opinion covers, in part, the salary cost data that NRCS relies
on when reporting its costs. To identify causes of differences between the
estimated costs and the costs reported by NRCS, we reviewed NRCS's
assumptions and data used in the cost estimates. As part of this effort,
we analyzed NRCS's model and related documentation to determine whether
the model appeared to be a logical and reasonable method for estimating
technical assistance costs. To determine the reliability of workload data
used in the model, we evaluated NRCS's instructions for developing them,
and obtained the views of NRCS officials and staff who led the development
of the workload analyses in the 10 states that performed over half of the
agency's CRP technical assistance and over 40 percent of its Wetlands
Reserve Program (WRP) in fiscal year 2002.6 We performed our work
5NRCS includes estimates for the Conservation Reserve Enhancement Program
within the Conservation Reserve Program estimates.
6The 10 states are Kansas, Illinois, Iowa, Louisiana, Minnesota,
Mississippi, Missouri, New York, North Dakota, and Wisconsin.
between September 2003 and October 2004 in accordance with generally
accepted government auditing standards.
Results in Brief The technical assistance cost estimates produced by
NRCS's model for 10 Farm Bill conservation programs for fiscal years 2002
and 2003 varied considerably from NRCS's actual costs.
o For fiscal year 2002, NRCS's model estimated technical assistance costs
for 8 Farm Bill programs that were then operating. Estimates for 3 of the
8 programs-1 large, established program and 2 small, new programs- ranged
from 48 to 302 percent above actual costs. Estimates for the 5 other
programs ranged from 5 to 36 percent lower than actual costs. Overall, for
the 8 programs, NRCS estimated its technical assistance costs at $254
million, about 19 percent more than the $213 million ultimately reported
by NRCS.
o For fiscal year 2003, NRCS's model estimated technical assistance costs
for the 10 Farm Bill programs. For 7 of these programs, the estimates were
higher than the actual costs by 9 to 50 percent, and for 3 of the
programs, the estimates were lower by 16 to 60 percent. Overall, of the 10
programs, NRCS estimated its technical assistance costs at $295 million,
about 15 percent more than the actual $257 million reported by NRCS.
These results fall outside the agency's goal of achieving estimates that
are within 10 percent (higher or lower) of NRCS's actual costs. NRCS
officials generally agreed with our analysis and continue to work on
improving their estimates.
We identified several reasons for the differences between the estimates
and actual costs. First, because of funding issues, some NRCS technical
assistance work was delayed and occurred later in the year than NRCS
assumed it would when it made its estimates. As a result, NRCS's actual
costs for technical assistance were less than NRCS had estimated. Such
differences are to be expected because the timing of funding is dependent
on a number of congressional and executive actions, which vary from year
to year and cannot be precisely forecast. Second, NRCS's technical
assistance cost estimates include costs incurred by NRCS's partners, which
are generally not reimbursed or otherwise incurred by NRCS. Including
these costs contributed to the model's overestimates. While partner costs
are needed to understand the full costs of programs, these costs should
not
be included when NRCS reports its costs, and should be identified when it
compares its cost estimates with actual costs. Third, some data NRCS uses
in its model are based on inaccurate assumptions. One such assumption is
that there is one "typical" type of technical assistance work in each area
for which time estimates were developed for the model. NRCS staff had
difficulty describing their "typical" work for each area because of the
significant variations in topography, land uses, size of operations, and
conservation practices in many areas. Such problems could cause either
over-or underestimates. Additionally, we found when developing data about
the time it takes NRCS staff to perform technical assistance tasks for use
in the model, NRCS assumes, among other things, that its staff are fully
trained and work without interruption. These assumptions do not reflect
actual workplace conditions and lead to underestimates. NRCS officials
said that they would reconsider these assumptions.
To improve NRCS's cost estimates, we are recommending that the Secretary
of Agriculture direct the Chief of NRCS to clearly identify the portion of
its estimated costs that is incurred by partners and not paid for by NRCS;
modify assumptions about how long it takes to complete tasks to better
reflect actual workplace conditions; and pilot test any new methods,
including those developed to better estimate typical work in areas of the
country with diverse conditions. In commenting on a draft of this report,
NRCS said that they generally agreed with our findings and recommendations
and these would serve as the basis for improvements they plan to make in
their methods.
Background In 1994, the Congress established NRCS and gave it jurisdiction
over programs of the former Soil Conservation Service, as well as other
USDA financial or technical assistance programs for natural resource
conservation and rural development.7 With more than 12,000 employees
nationwide-about three-fourths located in its state offices and 2,500
field offices-NRCS focuses primarily on private and other nonfederal
lands. NRCS staff regularly work in partnership with state, local, and
private entities, using the same case files and technical assistance
tools. NRCS's primary partners are state conservation agencies and the
approximately 3,000 conservation districts nationwide. These conservation
districts are
7Federal Crop Insurance Reform and Department of Agriculture
Reorganization Act of 1994, Pub. L. No. 103-354 (1994).
units of local government organized to support local conservation efforts
with their own programs and staff.
When asked to do so by farmers and others, NRCS staff assess farmers'
needs-a process that generally involves traveling to the site. NRCS staff
work with the landowner to develop a conservation plan that describes the
strategy to be used, the schedule of activities, and estimated costs. In
some instances, plans are revised several times until the landowner
selects a final alternative. If the landowner applies to implement the
conservation plan under a Farm Bill program and the land meets the
eligibility criteria and is accepted, NRCS develops a contract. After a
contract is developed and signed, NRCS staff complete paperwork for
payments to the landowner. NRCS staff assist with installation of
practices, for example, by surveying land, providing practice standards
and specifications, and ensuring contractors have carried out the terms of
the contract.8 In addition, NRCS continues to document activities
throughout the life of the contract, which may be years or decades. Staff
also periodically certify that the participant is complying with the
contract terms, depending on the program requirements.
In 1998, the Chief of NRCS called for a new agencywide effort to improve
NRCS's accountability by providing better information and analyses on how
the agency uses its resources and what it achieves with its funds. As part
of these efforts, NRCS has taken steps to estimate its future technical
assistance costs and budgets, as described in figure 1.
8NRCS staff provide some technical assistance to landowners even if the
landowner does not apply or qualify for USDA Farm Bill programs.
Figure 1: Steps NRCS Has Taken to Develop a Method for Estimating Future
Technical Assistance Costs and Budgets
Developing a workload analysis process: In a workload analysis, teams of
NRCS staff and their partners estimate the time they take to perform
technical assistance tasks-time per task estimates. These estimates are
particularly important because NRCS uses them to determine its estimated
technical assistance cost for each conservation program. NRCS completed a
national workload analysis in 2000, using the following steps:
0M NRCS divided the country into 218 areas where teams of staff were
formed to make time estimates.
0M NRCS divided its conservation work into 28 types, such as conservation
work on pastureland, irrigation systems development, and wetlands
restoration.
0M NRCS then identified 356 tasks associated with these 28 types of work.
0M In each area, as a basis for developing time per task estimates, the
NRCS teams developed descriptions of the typical technical assistance work
they perform. For example, a team from one area might have described one
type of its typical work as planting grass to stop erosion on a 150-acre
dairy farm.
0M After developing descriptions of the types of typical work in each
area, each team then estimated the time required to perform the tasks
associated with them.
NRCS plans to perform a similar nationwide workload analysis after it
completes revisions to its methodology.
Developing a computer model for estimating current and future costs: NRCS
calls this its cost of programs model. The model incorporates (1) the time
per task estimates; (2) salary and benefit costs based, in part, on data
from NRCS's time and attendance system; and (3) other data, such as
overhead costs from the agency's financial system. NRCS developed the cost
of programs model in 2002.
Developing a quality assurance process: This process involves, in part,
(1) testing the results of the workload analyses and (2) testing the
estimates made by the computer model. Before NRCS staff completed their
1999 workload analysis, they analyzed their preliminary results for field
and state offices by several means including comparing their estimates
with those made by the staff from neighboring areas, and reassessed and
adjusted the estimates when they thought they were either too high or too
low. In addition, NRCS tested cost estimates from its model for fiscal
years 2002 and 2003, by comparing them with the actual costs. NRCS
officials said that their goal in these tests is to have their technical
assistance cost estimates fall within 10 percent of the costs reported by
NRCS.
Source: GAO analysis of NRCS's documents.
NRCS's model for estimating the cost of programs consists of an Excel
spreadsheet that makes program-by-program calculations using such data as
time per task estimates and salary benefits and support costs, as well as
assumptions about the average length of a contract and the proportion of
work that will be performed during each year of the contract. Figure 2
illustrates the information used in NRCS's cost of programs model.
Figure 2: Estimates and Data Used in NRCS's Cost of Programs Model and
Discussed in This Report
Source: GAO analysis based on NRCS documents.
Note: NRCS's salary data relies, in part, on data from the time and
attendance system. The model also relies on other data, such as program
participation and number of contracts. The source of some of these data is
the Farm Service Agency, which administers CRP.
Time Per Task Estimates NRCS state offices have overall responsibility for
developing the time per task estimates used in the model. NRCS divided the
country into 218 areas and assigned teams of about 7 to 12 NRCS and
partner staff to develop time per task estimates for the work to be done
in each area. According to NRCS's process for developing time per task
estimates, these teams first define the typical work in their area-that
is, the sizes and types of land they work with and the conservation
practices they plan for this land. For example, a Texas team determined
that one type of typical conservation work in its area was planting grass
cover on a 150-acre dairy farm. Using this description, the team discussed
and agreed on estimates of time required for each of the 29 tasks
associated with performing this work, such as the time it would take them
to design a grass-planting plan. Generally, after developing the time per
task estimates, the team leader submits them to the state time per task
leader to review and input into an NRCS database. The methods of review
the state offices use include
comparing the areas' estimates with each other. If the state leader
encounters substantial unexplained differences among estimates, he or she
generally contacts the team leader for an explanation and to have the
estimate changed, if appropriate. The state leader then submits final time
per task estimates to NRCS headquarters, which also reviews them. If
questions arise, headquarters may ask the state office to perform
additional review, and make changes, if warranted.
Since the quality of data entered into a model affects the estimates it
produces, NRCS has worked to increase the reliability of its time per task
estimates by developing new estimates on three separate occasions and
asking for staff feedback on the quality of the estimating process. In
addition, NRCS plans to update its time per task estimates again because
the tasks and the time required to perform them have changed, along with
program requirements and policies, over the past 5 years.
Salary Costs NRCS's cost model also includes data on average salary costs,
which are based, in part, on data from NRCS's time and attendance system.
NRCS staff enter hours that they work by program, and by the activities
associated with those programs, biweekly in the time and attendance
system. For example, they might record working 20 hours on CRP and 20
hours on WRP. For the 20 CRP hours, they may record 10 hours to determine
land eligibility for multiple landowners and 10 hours working with
landowners on conservation plans. These data are reviewed first by
supervisors and then at the state office level.
NRCS's Technical NRCS tested cost estimates from its model for fiscal
years 2002 and 2003 by
comparing them with the agency's actual costs. Our analysis of
theseAssistance Cost comparisons shows that program-by-program estimates
from NRCS's Estimates Differ from model vary considerably from the
agency's actual costs as shown in table 1. Actual Costs Reported These
results do not meet the agency's goal of achieving a difference of no
more than 10 percent between estimates and reported costs.
by NRCS
Table 1: Comparison of Technical Assistance Costs Estimated by NRCS's
Model and Actual Costs Reported by NRCS, Fiscal Years 2002 and 2003
Dollars in thousands
2002 2003 Difference that estimates are over or (under) reported costs
Difference that estimates are over or (under) reported costs Farm Bill Programs
Costs Costs
Costs reported by reported by
estimated NRCS NRCS
by NRCS information information
model systems Number Percent systems Number Percent
Costs estimated by NRCS model
Environmental
Quality
Incentives
Program $162,220 $109,885 $52,335 48 $175,932 $146,675 $29,257
Conservation
Reserve 60,963 64,378 (3,415) (5) 71,424 58,730 12,694
Program
Wetlands
Reserve
Program 16,619 21,422 (4,803) (22) 20,873 24,797 (3,924) (16)
Wildlife
Habitat
Incentives 6,171 9,477 (3,306) (35) 6,396 11,388 (4,992) (44)
Program
Ground and
Surface
Water
Conservation
Program 3,093 769 2,324 302 7,811 5,292 2,519
Grassland Reserve
a aaa
Program 5,206 4,447 759
Agricultural
Management
Assistance 3,025 4,753 (1,728) (36) 1,436 1,314 122
Program
Farmland
Protection
Program 1,562 1,939 (377) (19) 3,180 2,387 793
Klamath 529 196 333 170 2,142 1,432 710
Basin
Conservation
Security a a a a 125 313 (188) (60)
Program
Total $254,182 $212,819 $41,363 19 $294,525 $256,775 $37,750 15
Source: NRCS and GAO analysis of NRCS cost estimates.
Note: Klamath Basin and the Ground and Surface Water Conservation Program
are new programs also. Conservation Reserve Enhancement Program estimates
are included in those for CRP. Fiscal years 2002 and 2003 are the only
year for which NRCS had complete data at the time of our review. Numbers
and percentages in the table may not equal totals because of rounding.
aThe Grasslands Reserve Program and the Conservation Security Program had
no technical assistance costs in 2002 because NRCS was in the process of
setting up program rules.
o For fiscal year 2002, the CRP estimate was closest to the actual
costs-it was 5 percent lower than the actual costs reported. Of the
remaining program estimates for 2002, three were higher than the actual
cost data by 48 percent to 302 percent, and four were lower by 19 percent
to 36 percent. Altogether, NRCS estimated that its technical assistance
costs for eight of the Farm Bill conservation programs would be about $254
million, 19 percent higher than its actual costs of about $213 million.
NRCS did not estimate costs for two programs that had not yet been
implemented.
o For fiscal year 2003, the estimate for the Agricultural Management
Assistance Program, a program that provides financial assistance to
producers in 15 states to, among other things, construct or improve
irrigation structures and plant trees, was closest to the reported costs-
it was 9 percent greater than the actual costs. Of the remaining program
estimates for 2003, six were higher than the actual cost data by 17
percent to 50 percent, and three were lower by 16 percent to 60 percent.
Altogether, NRCS estimated its technical assistance costs for 10 Farm Bill
conservation programs would be about $295 million, about 15 percent higher
than its actual costs of about $257 million. This estimate was 4 percent
closer to total actual costs than in fiscal year 2002.
o For the three largest programs-the Environmental Quality Incentives
Program, CRP, and WRP-the estimates varied from the actual cost data
somewhat less in fiscal year 2003 than 2002. In fiscal year 2002, the
estimates had a spread of 70 percent-from a 22 percent underestimate to a
48 percent overestimate. In fiscal year 2003, the estimates had a spread
of 38 percent-from a 16 percent underestimate to a 22 percent
overestimate.
Differences in Estimated and Actual Costs Have Several Causes
We identified several reasons for the differences between NRCS's estimated
and actual costs. First, for fiscal years 2002 and 2003, the model
estimated costs based on the assumption that programs would be fully
funded in the beginning of the fiscal year. This did not happen, however,
because the 2002 Farm Bill was enacted later than expected and because
USDA operated under a continuing resolution for a good portion of fiscal
year 2003. According to NRCS officials, as a result, less technical
assistance work was performed than the estimates reflected. Second, NRCS's
model includes costs for work performed by partners' staff and paid for by
the partner organizations, while the actual cost data generally contains
only the costs for the work of NRCS's staff. Third, NRCS's model uses some
data
that are based on inaccurate assumptions. This is likely to have
contributed to differences between estimates and actual costs reported by
NRCS.
Actual Timing of NRCS Work Differed from Timing Assumed When Estimating
Costs
In several instances, NRCS performed technical assistance work at
different times than NRCS originally assumed it would when estimating
technical assistance costs. First, NRCS estimates have assumed that full
funding would be available for new contracts at the start of the fiscal
year, but in practice, this has not occurred. For example, in 2002, the
Farm Bill was enacted in May, later than NRCS expected, and OMB
apportioned funds to USDA for implementing the Farm Bill programs in
July-about three quarters of the way through the fiscal year. Since only
2-1/2 months of the fiscal year remained, different work-and in some
cases, less work- was performed under the Farm Bill than NRCS had
anticipated, according to NRCS officials. The general 2002 sign-up period
for CRP contracts did not start until August 2002, limiting the amount of
work performed on new CRP contracts. Moreover, in fiscal year 2003, USDA
operated under a continuing resolution until receiving fiscal year 2003
appropriations in February and an OMB apportionment in March-about half
way through the fiscal year. While NRCS can adjust its assumptions, it is
not possible to eliminate uncertainties and variances related to the
timing of funding approvals that cause differences between the estimated
and actual program costs.9 NRCS officials said that they have been
studying the model's estimates and modifying some assumptions about
workloads to improve the estimates.
NRCS Model Included Costs for Work Performed by Partners
Because NRCS's partners' efforts are a relatively important part of
overall technical assistance efforts, NRCS has included its partners'
costs in its model so that it is in a position to estimate total technical
assistance costs to carry out the programs. For example, according to 1999
NRCS staff estimates, the most recent available, NRCS's partners were
responsible for about 17 percent of total CRP costs and 15 percent of
total WRP costs that year. Using those percentages, we calculated that
NRCS's partners might have added about $15 million to these two programs'
fiscal year 2002 technical assistance costs.
9For example, see Congressional Budget Office, Estimating the Costs of
One-Sided Bets: How CBO Analyzes Proposals with Asymmetric Uncertainties,
(Washington D.C.: Oct. 1999).
To further illustrate the effects of including partners' costs, we
conducted two comparisons. First, we compared NRCS's fiscal year 2002
technical assistance cost estimates, which include partners' costs, for
two programs-CRP and WRP-with the costs reported by NRCS. We then made the
same comparison but with partners' costs excluded. (See table 2.) The
results show that the differences between the estimated and actual costs
increase when partner costs are excluded. For the CRP program, for
example, the difference between the estimated and reported costs increases
from a 5 percent underestimate to a 22 percent underestimate when partner
costs are excluded. For WRP, the difference between the estimated and
reported costs also increases from a 22 percent underestimate to a 34
percent underestimate when partner costs are excluded.
Table 2: Costs Estimated by NRCS's Model, with and without Partner Costs,
Compared to Actual Costs Reported by NRCS, Fiscal Year 2002
Dollars in millions
CRP WRP
Percentage
Costs Costs over or Costs Costs
estimated reported by (under) estimated reported by Percentage
over or
Cost using information reported using information (under)
comparisons model systems costs model systems reported
costs
Including
partners' $61.0 $64.4 (5) $16.6 $21.4 (22)
costs in
model
Excluding
partners' $50.3 $64.4 $14.1 $21.4
costs from
model (22) (34)
Source: NRCS and GAO analysis of NRCS costs.
However, NRCS officials believe that the differences between their
estimates, which include partners' costs, and the reported actual costs
are less significant because their partner costs have decreased-they could
not say by how much. This may not be the case, however. An NRCS briefing
document reported in 2000 that NRCS expected its partners to perform more
of its workload than in 1999. Similarly, most of the officials we spoke
with from conservation districts-which are key partner organizations-
reported that conservation districts have increased the amount of
technical assistance work they have performed in the past few years.
Finally, partner's costs should not be included when NRCS reports its
costs, and when it compares its estimated and actual costs. In commenting
on a draft of this report, NRCS officials noted that it is important to
include partner costs when estimating the full costs of the programs. They
also stated that
estimated costs are used for budgetary purposes and actual costs are the
only costs charged to the government and used in final reports.
Time Per Task Data Used in the Model Are Based on Inaccurate Assumptions
Descriptions of Typical Work
NRCS's estimates of technical assistance costs for 10 Farm Bill
conservation programs are developed, in part, using time per task data
that are based on inaccurate assumptions. First, NRCS's basic assumption
used for developing its estimates-descriptions of typical technical
assistance work-oversimplifies the situations field staff encounter to the
extent that the resulting estimates do not accurately represent the time
it takes staff to do the work. Second, NRCS's time per task estimates are
based on some assumptions that do not reflect actual workplace conditions.
Finally, some NRCS staff who provide technical assistance are uncertain
about how to allocate the time they spend traveling to and from field
locations among NRCS's programs.
NRCS uses descriptions of typical work as the basis for its time per task
estimates. NRCS guidance tells teams to develop descriptions of their
typical work, but does not tell them how to do this for areas in which
conditions are diverse. In the absence of more specific guidance, staff
have encountered difficulty with the concept of typical work. According to
one team leader, for example, the guidance does not tell teams how to
determine what is typical when there are significantly varying land sizes,
types of farms, or conservation practices, and several team leaders told
us that such variations made identifying typical work difficult or
impossible. For example, a North Dakota team leader said that the
conservation work in his area varies considerably, ranging from installing
pipelines to creating ponds to building fences. Moreover, a Wisconsin team
leader told us that at least three typical descriptions would be needed to
represent the variety of farms in his area: one each for dairy, vegetable,
and cranberry farms. Other NRCS staff commented that it may not be
possible to describe what is typical for western rangeland, where
operations vary from hundreds of acres to tens of thousands of acres.
Finally, another team leader told us that NRCS's guidance is vague, and as
a result, his team interpreted the term "typical" to mean average, while
another understood it to be the median or middle value. NRCS staff raised
these concerns in 2000, and NRCS officials are aware of the difficulties
staff have encountered in describing their typical work. According to
these officials, the scope of this estimating problem is nationwide and
resolving this concern is important for making more accurate estimates.
NRCS is considering whether it is possible to resolve difficulties staff
face in describing typical work by enlarging the number of areas for which
estimates are developed or collecting data for
more than one "typical" unit per area, and thereby reducing the extent of
diversity within each. Additional information will be needed to determine
whether such an approach will succeed.
Assumptions about the NRCS has directed its staff to base its time per
task estimates on three
Workplace assumptions that we believe do not reflect actual workplace
conditions. First, estimating staff are to assume that all NRCS staff are
fully trained. This is not the case, however. About 10 percent of current
NRCS staff were hired between 2001 and 2004. According to one NRCS
official, staff need 1 to 1.5 years to become able to independently
perform most technical assistance for CRP and 3 to 5 years for WRP.
Because not all staff are fully trained, assuming that they are is likely
to inappropriately lower the time per task estimates. The second
assumption is that staff are not interrupted during their workday. Under
normal conditions, staff regularly experience interruptions that decrease
productivity. Assuming that this is not the case is also likely to
contribute to inappropriately lowering the time per task estimates. A
third assumption, which would likely lead to raising the time per task
estimates, is that NRCS staff completely follow NRCS's policies,
procedures, and guidance in performing work. Actually, however, staff
sometimes take shortcuts that do not comply with all policies, procedures,
and guidance-thereby completing tasks faster than expected.
In contrast, NRCS's reported actual costs are based on actual work
conditions. That is, these costs reflect the additional time taken by new
and partially trained staff, the added time caused by interruptions that
staff regularly face, and the timesaving shortcuts that staff sometimes
take. Although we could not determine the precise effects of these
assumptions, some information is available to indicate that they warrant
reexamination. For example, NRCS staff reported that by using shortcuts,
their CRP and WRP work took 24 percent and 31 percent less time,
respectively, than the time they had estimated in their workload analysis.
NRCS officials said that they have adjusted the model to take into account
some policy and procedural changes that reduce the workloads of NRCS staff
and added that they would also reconsider the assumptions that we
identified.
Allocation of Travel Time NRCS staff also reported some confusion about
how to allocate their travel time. The guidance directs staff to divide
travel time, which constitutes an important portion of field staff work
time, among different program activities when necessary.10 Staff usually
drive to meet with landowners and view land, often traveling to distant
locations and working on several program activities with several farmers
during a single trip. When this occurs, they must determine how much
travel time they should assign to each program. The guidance states that
travel time should be "prorated" among different program activities but
does not explain how to do this. This lack of guidance results in
reporting inconsistencies. For example, staff often visit several farms on
a single trip making it difficult to determine how to prorate this time
among multiple program activities. NRCS officials said that they are aware
of this problem but have not yet developed a solution.
Conclusions While we recognize that only 2 years of comparative cost data
are available and that NRCS has been striving to improve its technical
assistance cost estimates, NRCS's cost estimates differ enough from actual
costs reported by NRCS to be of concern to those who use these estimates.
NRCS's overall technical assistance cost estimate for 10 Farm Bill
conservation programs for fiscal year 2003 is closer to the reported cost
than the estimate was for fiscal year 2002, but too much variation is
evident on a program-by-program basis in both years. Until improvements in
NRCS's technical assistance cost estimating are demonstrated through tests
of the model's results, we believe NRCS cost estimates should be used with
caution. Without identifying costs incurred by its partners when assessing
the reasonableness of the estimates made by its model, NRCS cannot ensure
the validity of its cost comparisons. Also, unless NRCS modifies its
assumptions to better reflect actual workplace conditions, its technical
assistance cost estimates will not be as precise as they could be.
Finally, without pilot testing its plans for improving descriptions of
typical work or other changes in data development, NRCS cannot be assured
that its investment in its next nationwide workload analysis will be well
spent. As NRCS improves the quality of its workload analysis, including
its time per task estimates, and the assumptions used in the model, we
believe more accurate technical assistance cost estimates will be
developed. Moreover,
10Staff travel is an important task-we performed a sensitivity analysis
that suggests travel time has a larger impact on CRP cost estimates than
most other tasks associated with CRP contracts.
when these improvements have been made, NRCS will be in a better position
to evaluate the overall quality of its estimating. Further testing in the
years ahead may well be needed to gain a better understanding of the
causes of variations in the program-by-program cost estimates.
Recommendations To improve the accuracy, and therefore the usefulness of
NRCS's program cost estimating, we are recommending that the Secretary of
Agriculture direct the Chief of NRCS to take the following three actions:
o clearly identify nonreimbursable costs incurred by NRCS's partners when
presenting estimates of NRCS's costs, ensuring that its model's estimates
are comparable with actual data;
o change the assumptions used for developing time per task data for the
model so that they better reflect actual work conditions; and
o pilot test the feasibility of proposed changes in the development of
the time per task data, including changes in development of typical work
descriptions in several diverse areas of the country before proceeding
with another nationwide workload analysis.
Agency Comments and Our Evaluation
We provided a draft of this report to USDA for review and comment. We
received oral comments from NRCS, which are summarized below. We also
received technical comments, which are incorporated in this report as
appropriate. NRCS accepted our findings and said they would develop
actions in response to our recommendations. The agency stated that the
report provides the basis for updating the agency's workload analysis,
more accurately estimating partner contributions to NRCS's programs, and
making other necessary adjustments to the assumptions in cost estimates.
NRCS also stated that our report rightly points out that some assumptions
used in estimation were inaccurate, but that portions of the report had an
unnecessarily, negative tone. They noted for example that the estimates
would have been closer to actual costs if funding had been available at
the beginning of fiscal years 2002 and 2003. We agree that earlier funding
would have likely helped close the gap between estimated and actual costs.
NRCS also stated that partner contributions could be excluded from the
model's estimates, but that NRCS wanted to acknowledge the full cost of
programs using partner costs. We agree that partners' costs can and should
be excluded from the model's estimated costs when cost estimates are used
for budgetary purposes or for comparison with actual costs. Lastly, NRCS
commented that we found no problems with the logic of the model. We
disagree. The model's inclusion of costs that the agency did not incur,
such as partners' costs, is inappropriate when comparing estimates to
NRCS's actual cost data.
As agreed with your offices, unless you publicly announce its contents
earlier, we plan no further distribution of this report until 30 days from
the date of this letter. We will then send copies to interested
congressional committees, the Secretary of Agriculture, the Chief of NRCS,
and other interested parties. We will also make copies available to others
on request. In addition, this report will be available at no charge on the
GAO Web site at http://www.gao.gov.
If you or your staff have any questions about this report, please call me
at (202) 512-3841. Key contributors to this report are listed in appendix
II.
Lawrence J. Dyckman Director, Natural Resources
and Environment
Appendix I
Objectives, Scope, and Methodology
Our objectives were to (1) review the technical assistance cost estimates
produced by the model and (2) identify the sources of differences that may
occur between the estimates and NRCS's reported costs.
To review the estimates, we assessed the differences between the NRCS's
model results compared with the actual costs reported by NRCS. We compared
NRCS's fiscal years 2002 and 2003 technical assistance cost estimates by
program with the actual costs reported by NRCS, the only years for which
these two sets of costs were available. To identify sources of difference
between these costs, we assessed assumptions and data used in the cost
estimates. To assess NRCS's cost model, we analyzed the model and related
documentation to determine whether the model appeared to be a reasonable
method for estimating program costs. In addition, we checked whether the
model formulas, contained in a MS Excel file, used the appropriate data,
and we reviewed the formulas to ensure that they were logical. In
addition, we replicated the model formulas using the proper data and
ensured that the resulting figures matched those shown in the model. We
interviewed NRCS officials responsible for developing the cost model to
gain an understanding of the model and its development. These officials
were staffed in NRCS's Budget Planning and Analysis Division, Operations
Management and Oversight Division, and in field offices.
We conducted sensitivity analyses to illustrate the possible importance of
different variables in the model. These sensitivity analyses were
conducted using Monte Carlo simulation, which uses random numbers to
measure the effects of uncertainty on model output-in this case, the
technical assistance cost estimates. Our analysis was based on general
assumptions about the probability distributions characterizing some of the
variables in the model.
Also, we interviewed and requested documentation from 20 officials of
state government agencies and conservation district associations to assess
whether the nonreimbursed contributions of conservation districts
increased or decreased in the past several years. We did so for each of
the 10 states with the most CRP and WRP contracts. The 10 states are
Illinois, Iowa, Kansas, Louisiana, Minnesota, Mississippi, Missouri, New
York, North Dakota, and Wisconsin. Of the 20 officials we contacted, 13
responded to our request and provided some information about partners'
contributions.
To assess NRCS's means for ensuring the reliability of the data used in
the cost model, we traced the time per task and other data to their
respective
Appendix I
Objectives, Scope, and Methodology
sources, which included agency reports and databases. Since information
regarding data sources was not always readily available in writing, we met
with NRCS officials who described and provided the sources of the model's
data. Once we identified the source, we verified that the data had been
correctly transferred from the source to the model. In addition, we
performed limited reliability tests, primarily tests for omitted entries
and outliers, of the available source data. Furthermore, to obtain NRCS's
field staff views on the reliability of the time per task estimates and
time and attendance data used in the model, we used a semistructured
interview guide to interview all 10 officials leading estimate development
efforts in the 10 states that had the most CRP and WRP contracts in 2002.
The 10 states had over half of NRCS's total CRP contracts and over 40
percent of its WRP contracts. Using another semistructured interview
guide, we interviewed 10 randomly selected NRCS team leaders (out of 44
leaders) who each led a team developing time per task estimates in one
area in each of the 10 states. However, we could not assess the quality of
NRCS's reviews of its workload analysis at state offices because NRCS
state officials retained insufficient documentation of the reasons for
changes in data made during their reviews.
To assess the reliability of other data used in NRCS's model, we reviewed
NRCS's method for developing overhead cost data and salary cost data,
which relies on the time and attendance system. Overall, we noted that
USDA obtained an unqualified opinion on its financial management
activities in fiscal years 2002 and 2003.1 This opinion covers, in part,
the salary cost data that NRCS relies on when reporting its costs. In
addition, we reviewed NRCS's development of overhead cost information that
is based on Office of Management and Budget budget object classifications,
which include such costs as rent, utilities, equipment, and supplies. In
addition, we reviewed an NRCS draft report about the quality of NRCS time
and attendance data. That 2001 NRCS draft report found that about half of
NRCS field offices had deficiencies in documenting their use of time, but
the report did not provide sufficient detail to reveal the precise extent
of the problems. Since then, NRCS has implemented corrective actions,
according to agency officials, and has verified on a limited basis that
improvement has occurred.
1GAO, Department of Agriculture: Status of Efforts to Address Major
Financial Management Challenges, GAO-03-871T, June 10, 2003. USDA, Office
of Inspector General, U.S. Department of Agriculture's Consolidated
Financial Statements for Fiscal Years 2003 and 2002, # 50401-51-FM,
January 2004.
Appendix I
Objectives, Scope, and Methodology
Our issues with the reliability of data are discussed throughout the
report. We performed our work between September 2003 and October 2004 in
accordance with generally accepted government auditing standards.
Appendix II
GAO Contacts and Staff Acknowledgments
GAO Contacts Lawrence J. Dyckman (202) 512-3841 Charles M. Adams (202)
512-8010
Acknowledgments In addition to the individuals named above, Charles W.
Bausell Jr., Nancy Crothers, Anne E. Dilger, Beverly A. Peterson, Lynne M.
Dunlay, and Judy K. Pagano made key contributions.
GAO's Mission The Government Accountability Office, the audit, evaluation
and investigative arm of Congress, exists to support Congress in meeting
its constitutional responsibilities and to help improve the performance
and accountability of the federal government for the American people. GAO
examines the use of public funds; evaluates federal programs and policies;
and provides analyses, recommendations, and other assistance to help
Congress make informed oversight, policy, and funding decisions. GAO's
commitment to good government is reflected in its core values of
accountability, integrity, and reliability.
Obtaining Copies of The fastest and easiest way to obtain copies of GAO
documents at no cost
is through GAO's Web site (www.gao.gov). Each weekday, GAO postsGAO
Reports and newly released reports, testimony, and correspondence on its
Web site. To Testimony have GAO e-mail you a list of newly posted products
every afternoon, go to
www.gao.gov and select "Subscribe to Updates."
Order by Mail or Phone The first copy of each printed report is free.
Additional copies are $2 each. A check or money order should be made out
to the Superintendent of Documents. GAO also accepts VISA and Mastercard.
Orders for 100 or more copies mailed to a single address are discounted 25
percent. Orders should be sent to:
U.S. Government Accountability Office 441 G Street NW, Room LM Washington,
D.C. 20548
To order by Phone: Voice: (202) 512-6000 TDD: (202) 512-2537 Fax: (202)
512-6061
To Report Fraud, Contact:
Waste, and Abuse in Web site: www.gao.gov/fraudnet/fraudnet.htm
E-mail: [email protected] Programs Automated answering system: (800)
424-5454 or (202) 512-7470
Congressional Gloria Jarmon, Managing Director, [email protected] (202)
512-4400 U.S. Government Accountability Office, 441 G Street NW, Room 7125
Relations Washington, D.C. 20548
Public Affairs Susan Becker, Acting Manager, [email protected] (202)
512-4800 U.S. Government Accountability Office, 441 G Street NW, Room 7149
Washington, D.C. 20548
*** End of document. ***