[Final Audit Report on Improvements Needed in Developing and Reporting on GPRA Performance Indicators, Department of the Interior ]
[From the U.S. Government Printing Office, www.gpo.gov]

Report No. S-IN-MOA-0088-2002

Title: Final Audit Report on Improvements Needed in Developing
       and Reporting on GPRA Performance Indicators, Department
       of the Interior 

  


Date: March 31, 2004

****************************************DISCLAIMER**********************
This file contains an ASCII representation of an OIG report. No attempt has been made to display graphic images or illustrations. Some tables may be included, but may not resemble those in the printed version. A printed copy of this report may be obtained by referring to the PDF file or by calling the Office of Inspector General, Division of Acquisition and Management Operations at (202) 219-3841.
************************************************************************


Memorandum

To:		Assistant Secretary, Policy, Management and Budget

From:		William J. Dolan, Jr.  
		Regional Audit Manager, Eastern Region

Subject:	Final Audit Report on Improvements Needed in Developing and Reporting on GPRA Performance Indicators, Department of the Interior 
(No. S-IN-MOA-0088-2002)

      This report presents the results of our audit of the Department of the Interior (DOI) reporting of its fiscal year 2001 Government Performance and Results Act (GPRA) strategic goal 1, which relates to protecting the environment and preserving our nationï¿½s natural and cultural resources.  This strategic goal reports on DOIï¿½s efforts to protect the health of public lands, maintain healthy natural systems, protect and recover imperiled species, and protect and restore cultural resources.  

      We concluded that improvements were needed to DOIï¿½s development of and reporting on GPRA performance indicators.  In the March 16, 2004 response to the draft report, the Assistant Secretary for Policy, Management and Budget concurred with the reportï¿½s two recommendations.  Based on the corrective actions discussed in the response and additional information subsequently provided, we consider the recommendations to be resolved and implemented, and no further action by the Department is required.  

      The legislation, as amended, creating the Office of Inspector General, (5 U.S.C. App. 3) requires semiannual reporting to Congress on all audit reports issued, actions taken to implement audit recommendations, and recommendations that have not been implemented.

      We appreciate the cooperation we received from DOI and bureau managers and staff at all locations.  If you have any questions regarding this report, please call me at (703) 487-8011.


Executive Summary

Improvements Needed in Developing and Reporting on
GPRA Performance Indicators





The 1993 enactment of the Government Performance and Results Act (GPRA) requires each federal department and agency to prepare annual performance reports comparing planned accomplishments for mission-related activities with the results actually achieved.  The Department of the Interior (DOI) prepared the fiscal year 2001 Departmental Overview based on its Strategic Plan for fiscal year 2000 to fiscal year 2005.  The overview presents selected bureau and office goals and measures that achieve DOIï¿½s goals.  Strategic goal 1, protect the environment and preserve our nationï¿½s natural and cultural resources, recognizes that the interdependence between people and their environment requires healthy lands, waters, and other resources.



Our objective was to determine whether DOI (1) developed appropriate performance indicators for strategic goal 1 and its related performance measures and (2) accurately reported performance data and related information for these indicators.



We concluded that DOI should revise several of its performance indicators and measures to improve performance information provided to DOI managers, Congress, and the Office of Management and Budget.  In addition, DOI staff and management should ensure that the data used to report DOI performance are accurate.  During our review, we noted that DOI is revising its strategic planning model and instituting a new data verification policy.  These actions address many of the issues disclosed by our audit.



DOI concurred with the reportï¿½s two recommendations and has already taken corrective actions to implement them
(Appendix 4).   Based on the response and other information subsequently provided, we consider the recommendations to be resolved and implemented, and no further action by the Department is required.  



Abbreviations

BIA 	 Bureau of Indian Affairs
BLM 	 Bureau of Land Management
BOR 	 Bureau of Reclamation
CLI 	 Cultural Landscapes Inventory
DOI 	 Department of the Interior
FWS 	 U.S. Fish and Wildlife Service
GAO 	 General Accounting Office
GPRA 	 Government Performance and Results Act
LCS 	 List of Classified Structures
NAWCA 	 North American Wetlands Conservation Act
NPS 	 National Park Service
NRDAR 	 Natural Resource Damage Assessment and Restoration
OIG 	 Office of Inspector General
OMB 	 Office of Management and Budget
OSM 	 Office of Surface Mining Reclamation and Enforcement
PMDS 	 Performance Management Data System
STA 	 Storm Water Treatment Area


Introduction




Background
Congress enacted the 1993 Government Performance and Results Act (GPRA) to ï¿½help Federal managers improve service delivery, by requiring that they plan for meeting program objectives and by providing them with information about program results and service quality.ï¿½  GPRA holds federal departments accountable for achieving program results by requiring annual performance plans with specific, measurable goals and annual performance reports, which compare actual performance with these goals.  The Office of Management and Budget (OMB) Circular No. A-11 also provides guidance on GPRA implementation.





The Department of the Interiorï¿½s (DOI or the Department) eight bureaus prepared individual fiscal year 2001 annual performance reports.  A Department-level summary or ï¿½overviewï¿½ of bureau and departmental office efforts also provided a Departmentwide perspective.  DOIï¿½s performance report for fiscal year 2001 included six GPRA strategic goals.



DOI GPRA strategic goal 1 addresses the protection of the environment and the preservation of our nationï¿½s natural and cultural resources.  DOI established 13 performance indicators to evaluate its success in meeting the 7 measures for goal 1 (Table 1).
































Objectives and Scope
The objective of our audit was to determine whether DOI (1)ï¿½developed appropriate annual performance indicators for strategic goal 1 and its related performance measures and (2)ï¿½accurately reported performance data and related information for these indicators.  We selected GPRA strategic goal 1 because it included performance indicators for multiple DOI offices and bureaus.  To determine the appropriateness of the performance indicators, we reviewed all 13 indicators for general compliance with GPRA and OMB requirements.  In addition, we tested the performance data and narrative information for four indicators (nos. 1, 6, 7, and 12) to determine reporting accuracy.  We discuss the results of our audit in two sections accordingly.  The Appendices contain detailed scope and methodology information, sites visited, and prior audit coverage.




Results of Audit




Performance Indicators Should Be Revised
The fiscal year 2001 indicators did not provide sufficient useful information for decision-making by DOI managers, Congress, or OMB.  Nine of the 13 performance indicators were not appropriate or were not complete.  That is, they did not include all relevant bureaus, did not have annual targets, were not under the direct control of federal managers, or were not outcome-oriented.  We attributed this inadequacy to weaknesses in the Departmentï¿½s strategic planning process. 





DOI managers have recognized the need to improve indicators to achieve greater integration of purpose and function throughout the Department and to provide a GPRA document that represents the entire Department.  The Department is in the process of revising its strategic and performance plans to improve its reporting of Departmentwide accomplishments.


Basis for Determining Appropriate Indicators
To determine whether the Department developed appropriate annual performance indicators, we reviewed all 13 indicators for general compliance with GPRA and OMB requirements.  Specifically, we assessed whether they were:



> Objective, quantifiable, and measurable;
> Clear, meaningful1, reasonable, and relevant;
> Related to the strategic goal;
> Representative of Departmental accomplishments;
> Outcome-oriented; and
> Subject to sufficient Departmental control and oversight.



We found that 9 of the 13 performance indicators reviewed did not meet these criteria as identified in Table 2.






Public Lands Restored

We determined that the indicator for the public lands restored performance measure was not representative of the Departmentï¿½s accomplishments.  The objective of the restoration of public lands measure is to restore lands managed by, or under the regulatory jurisdiction of, DOI from previous uses such as mining, farming, timber harvesting, and other land disturbing activities.  The Department utilizes the number of acres restored from four bureaus as the performance indicator for this measure:



> Office of Surface Mining Reclamation and Enforcement (OSM) reclaimed abandoned mine lands2 
> U.S. Fish and Wildlife Service (FWS) restored acres in the National Wildlife Refuge System
> National Park Service (NPS) restored disturbed lands in National Parks
> Bureau of Indian Affairs (BIA) reforested acres through tree planting and thinning



This indicator was not representative because it included data not related to this goal and excluded data from bureaus that did perform restoration.  For example, acreage reported by BIA consisted of forest lands prepared for commercial harvesting that would have been more appropriately reported under a DOI goal related to resource use, rather than resource protection.  Concurrently, the acres restored by Bureau of Land Management (BLM) and Bureau of Reclamation (BOR) were not reported.  BLM manages 264 million acres of public lands and performs activities such as watershed restoration, remediation of abandoned mine sites, forest restoration and habitat improvement.  BOR restored wetlands and riparian habitats in 2001.  Without the BLM and BOR acreage this indicator does not include all of DOIï¿½s restoration accomplishments.


Damaged Lands and Resources Restored
We determined that the two performance indicators for the damaged lands and resources restored performance measure were not outcome-oriented (not a good characteristic of what was actually accomplished) and were not subject to DOI control.  The Department uses the Natural Resource Damage Assessment and Restoration (NRDAR) Program to assess damages to natural resources resulting from spills or hazardous substance releases, obtain settlements from responsible parties, and provide recovered monies to natural resource trustees to fund repairs.



The number of projects and the amount of settlement recoveries deposited into the Restoration Fund are output (an activity performed) and not outcome indicators (measures of how well a program performed).  The NRDAR manager agreed that the amount of resources restored would be a more appropriate indicator because it is outcome-oriented.  Prior to our audit, the manager requested development of a database that would collect this data.



Secondly, DOI managers had limited control over the accomplishments of these indicators.  The Department cannot initiate or complete restoration projects through NRDAR, and project funding is dependent upon court awards (settlements) and not annual appropriations.  For settlement recovery, individual DOI bureaus determine the amount of damage to the resource and seek compensation from responsible parties.


South Florida Natural Systems Restored
We determined that the South Florida restoration indicators were not appropriate3 because DOI managers did not set a target for fiscal year 2001 and lacked oversight authority for the project.  The Department reported on the cumulative acres of storm water treatment areas (STA)4 constructed and acres of land acquired for habitat protection for the South Florida natural systems restored performance measure.  Through the South Florida Ecosystem Restoration Program, the DOI seeks to restore, preserve, and protect the South Florida ecosystem while maintaining a sustainable South Florida economy.



DOI managers stated that they did not set a target for the number of acres of construction or acquisition for fiscal year 2001 due to the long-term nature of the South Florida restoration project.  In addition, they wanted to coordinate GPRA reporting with the status report provided to Congress every 2 years.  However, GPRA requires annual reporting.  Another DOI program, Endangered Species, has similar reporting requirements and long-term objectives, but still complied with the annual requirement.



Also, DOI managers were not directly responsible for the management of the project. Although the DOI leads the South Florida Ecosystem Restoration Task Force, it does not have any oversight or project authority.  The State of Florida carried out much of the STA construction and land acquisition, with the State estimated to contribute over half of the overall budget for the restoration.  The Army Corps of Engineers has the largest federal role in the project.  The Task Force acts as a facilitator between federal, state, and local agencies and other stakeholders involved in this $14.8 billion project.


Species Protected
We determined that two of the four indicators for the species protected performance measure are duplicative and should not be reported at the Department level. Under the authority of the Endangered Species Act of 1973, the Department is responsible for the protection and recovery of species listed as threatened or endangered.  The Departmentï¿½s GPRA Report contained four endangered species indicators: two for NPS and two for FWS.5



The NPS indicators for stable and improved endangered species within park boundaries duplicate the FWS data, which already includes NPS data in the nationwide total.  While an NPS endangered species indicator may be appropriate in the NPS GPRA Report, it is not appropriate in the DOI Report.



Cultural Properties Restored
We determined that the indicators for the cultural properties restored performance measure were not representative of the Departmentï¿½s accomplishments.  In 2001, the Department reported on two performance indicators for NPS cultural properties:  the number of historic structures in good condition and the number of cultural landscapes in good condition.



Both the historic structures and cultural landscapes indicators were not representative because they did not include the cultural properties under the management of four other DOI bureaus - BLM, FWS, BOR, and BIA.  



For example, BLM has estimated that it is responsible for protecting and restoring over 4 million archeological and historical properties.  Although FWS, BOR, and BIA have considerably fewer properties, they all have cultural properties listed on or eligible for the National Register of Historic Places.



Department officials stated that the full universe of cultural properties from all bureaus was not used because baseline data was not available for all bureaus.  However, at a minimum, the role of BLM, FWS, BOR, and BIA in protecting cultural properties should have been recognized in the DOI GPRA Report.  Also, an indicator to develop cultural property baselines for the other bureaus should have been included in the Departmentï¿½s performance plan.



In addition, the cultural landscapes indicator did not provide relevant information.  Although the historic structures indicator did compare structures in good condition to the total universe, the cultural landscapes in good condition were not measured against all NPS landscapes in the inventory.  Including the total universe would provide a more accurate picture of the overall condition of cultural landscapes under DOI protection.  The DOI response indicated that this correction has been made.



We also noted that the objective for the cultural properties measure included a reference to archaeological sites in addition to historic structures and cultural landscapes.  However, DOI did not include an indicator for archaeological sites in its GPRA Report.  For consistency, either the objective or the indicators need to be revised.


Weaknesses in DOI Strategic Planning Processes
We believe that weaknesses in DOIï¿½s strategic planning processes contributed to inappropriate GPRA performance indicators.  In fiscal year 2001, the DOI GPRA Report did not provide a comprehensive picture of the Departmentï¿½s accomplishments.  The performance data in the Report was extracted from the performance reports of its bureaus, offices, and departmental programs.  The bureaus often did not have comparable GPRA performance indicators, although they performed activities that were similar in nature or supported the same departmental strategic goal.  This occurred because each bureau and office developed its own strategic plans and reports, and the Department did not adequately coordinate performance planning and reporting between bureaus.  As a result, the Department did not have the relevant data necessary to fully report on its overall accomplishments.



The Department is currently revising its Strategic and Performance Plans and processes.  The new strategic planning model could potentially resolve some of these weaknesses because it is intended to:



> Clearly define performance measures; 
> Identify all contributors to performance measures and establish common measures to the extent possible; 
> Provide guidance for ensuring that data collected is valid and verifiable;
> Require that goals and measures be as outcome oriented as possible;
> Minimize the use of performance information over which managers have limited control; 
> Ensure that indicators are appropriately placed in each mission area, are not duplicative or overlapping, and that individual programs are adequately represented; and 
> Provide for both long-term goal and target setting and the use of incremental performance targets to establish annual performance expectations.












Recommendation



We recommend that the Department revise GPRA indicators for the strategic goal 1 as needed to ensure that all indicators:

1. Represent the Departmentï¿½s accomplishments and include accomplishments from all bureaus and offices with activity relating to the indicators, 

2. Are outcome-oriented, 

3. Are subject to oversight and control of DOI managers, 

4. Include annual targets, 

5. Eliminate duplicate data, and 

6. Provide relevant information.


Agency Response and Office of Inspector General Reply
DOI concurred with this recommendation.  The new Departmental Strategic Plan was completed and issued in September 2003.  The Departmentï¿½s response to the draft report stated that that the new planï¿½s highly integrated approach and outcome-orientation will resolve the weaknesses in the old plan identified by this report.  We have reviewed the new Strategic Plan and additional information provided related to corrective actions taken, and we agree that the new goals and measures for resource protection are more appropriate, representative, relevant, and outcome-oriented.  We consider this recommendation implemented, and no further action is required by the Department 






Improvements Needed for Accurate Reporting
Data reported in the fiscal year 2001 GPRA Report were inaccurate, unsupported or incomplete for the four performance indicators tested (Table 3).  These data errors indicated a lack of management controls over data accuracy.  In addition, narrative information in the report lacked sufficient disclosure or was incorrect.



Accurate data and information will improve program accountability of managers and allow Congress and OMB to better evaluate DOIï¿½s performance.  The data problems were the result of limited data verification at all levels of the organization and insufficient procedures to collect and maintain supporting documentation for GPRA performance data.  In January 2003, DOI distributed guidance to all bureaus and offices designed to improve and/or establish data validation and verification processes.  































Basis for Determining Accurate Indicators
To determine the accuracy of data and information reported for the four performance indicators, we assessed whether:



> The data reported the correct accomplishment amount and included all bureaus with activities related to the indicators.
> The data was consistently defined, reported, and verified by all levels and locations.
> The GPRA Report narrative and tables accurately and sufficiently described the data. 
> The data was supported by evidence and accurately collected and reported by organizations. 


Public Lands Restored
Our review of a sample of the acres reported by FWS and NPS as restored found inaccurate and unsupported data by both bureaus.  In addition, total DOI acreage is inaccurate and incomplete because it excludes any BLM and BOR restoration efforts and includes BIA acreage that should have been omitted.



Fish and Wildlife Service.  In fiscal year 2001, FWS reported 86,030 acres restored on refuge lands and 19,571 acres through 10 North American Wetland Conservation Act (NAWCA) grant projects.  We found that of 12,291 acres reported by FWS for seven refuges in our sample, 5,268 acres were misstated because two refuges reported incorrect data.6  In addition, we identified 1,769 enhanced NAWCA acres that should not have been included in acres restored.7  We could not determine how much of an additional 3,243 NAWCA acres were misclassified as restored because FWS did not separate enhanced acres from restored acres for three projects.



National Park Service.  NPS reported 7,500 acres restored in fiscal year 2001.  However, we found that the 5,878 acres reported by NPS for eight parks in our sample was misstated by 2,328 acres because four of the parks reported incorrect data.  These errors were the result of reporting the incorrect activity (300 acres), miscalculations (1,729 acres), and reporting prior year accomplishments (299 acres).



We also identified an inconsistency that allowed some NPS sites to report natural restoration as an accomplishment.  One of the national parks we visited counted 1,700 acres of land restored (22.7 percent of the NPS total) by prohibiting vehicular access and allowing the land to naturally restore itself.8  We believe NPS guidance defining restoration leaves room for interpretation and needs to be clarified.



Other Bureaus.  Total DOI restoration acreage is also incomplete and inaccurate because BLM and BOR restoration efforts were excluded, and BIA acreage should have been omitted.  We noted that although the BLM and BOR GPRA Reports included restoration goals, they did not collect similar data.  BLM did not present data in acres and BOR mixed restoration acreage with other activities, such as habitat protection and enhancement.  In addition, the 50,600 acres reported by BIA (28.5 percent of DOIï¿½s total) should not have been included under this goal because BIA was preparing forest lands for commercial harvesting and not performing restoration.


Natural Processes Restored Through Fire Management
Our review of DOI fuel treatment projects at 14 locations identified several inaccuracies in the total number of acres reported in fiscal year 2001.  During that year, the Department reported 728,000 acres treated by four bureaus (BIA, BLM, FWS, and NPS) for this performance measure.  Fuels are removed or modified to reduce the potential for wildfires, lessen post-fire damage, and limit the spread and proliferation of invasive species and diseases.  Methods used include prescribed fire, mechanical activities (cutting), herbicides, biological treatments, or combinations of these methods.  



The data problems we identified included reporting projects twice (1,827 acres by FWS), not reporting eligible acres (2,603 acres by BLM and FWS), lack of support for accomplishments (113 acres by BLM and NPS), and discrepancies between national, state, regional, and field location project information (9,319 acres by BIA, FWS, and NPS).  BLM, FWS, and NPS offices at various levels could not provide an accurate listing by project of acres receiving fuels treatment that matched the acreage reported to DOI for inclusion in the 2001 GPRA Report.  Without a listing by project, data verification is difficult to perform.  Another impact on data accuracy is that the bureaus and their field locations are not using the same criteria for calculating burned acres and for reporting multiple treatment of the same acreage.  Overall, the errors indicated a lack of control over data accuracy.  


Cultural Properties Restored
Our review of cultural properties included performance indicators for NPS historic structures and cultural landscapes.  We found NPS did not have sufficient documentation to support the GPRA performance data for historic structures, and we could not determine if condition assessments were performed timely.  Also, DOIï¿½s reported accomplishments were incomplete because the cultural properties data did not include the historic structures and cultural landscapes of BLM, FWS, BOR, and BIA.



Historic Structures.  We could not verify the accuracy of the data reported for the condition of NPS historic structures because hard copies of condition assessments were not maintained once the data was entered into the national List of Classified Structures (LCS).  In addition, data in the LCS and the Performance Management Data System (PMDS), which should theoretically be the same, did not match.



Our interpretation of NPS technical guidance is that the number and condition of historic structures in the LCS, certified PMDS, and uncertified PMDS should agree.  However, Table 4 below shows the discrepancies between the total number of structures and the condition of structures in 2001 in the three sources of historic structure data.9  The discrepancies between condition data for individual parks and regions and the LCS data were even higher, in some cases.  We believe it is important that parks and the national program office agree on the number and condition of historic structures to ensure that all resources under park management are accounted for and protected.






















































































Also, we could not determine if the condition assessments for historic structures were current.  In 1999, a data field was added to the LCS for the date of the last condition assessment.  To populate this data field, NPS staff sometimes used the year the record was last edited.  As a result, the actual date of the last condition assessment may be older than the date in the LCS, the condition of some structures may have changed, and the LCS data may no longer be reliable.



The Departmentï¿½s response to the draft report stated that NPS has taken steps to resolve discrepancies between LCS and PMDS data.  To address conflicting requirements, NPS has established a separate database for tracking National Historic Preservation Section 110 structures that are not part of GPRA requirements.  



Other Bureaus.  The total number and condition of cultural properties reported by DOI in 2001 are also incomplete because historic structures and cultural landscapes for BLM, BIA, BOR, and FWS were not included in the data.  As discussed in the measure appropriateness section, these other bureaus have a significant number of cultural properties under their management.



Report Narrative
In addition to performance result data, the performance report includes narrative information for each of the performance measures10 related to objectives, background information, baselines, data verification and validation, and data sources and limitations.  We found inaccurate and incomplete information in the narrative for the four indicators.  Specifically:



> The total DOI effort needed to restore all lands from previous uses was not disclosed.  Also, land restoration performance data was not verified and statistical analysis and random sampling were not performed.
> The fuel treatment indicator did not contain data on the total acreage needing treatment.
> The historical structures indicator did not disclose the baseline and data verification process.
> The cultural landscapes indicator did not disclose in the DOI narrative that this indicator only measured landscapes that had received condition assessments.11  Using the total number of landscapes would decrease the accomplishment from 31 percent to 5 percent in good condition.  The DOI response indicated that this calculation has been corrected.


Data Were Not Systematically Verified
Many of the problems with the accuracy of data and related information in the fiscal year 2001 GPRA Report occurred because there were no processes in place requiring bureau managers at all levels to verify data.  DOI GPRA managers stated that they relied on the bureaus to verify their own data.  Headquarters program managers and some managers at state/regional and field offices said that they compared targets and accomplishments for reasonableness, but this is not adequate to ensure that reported data are accurate.  Performance data should be verified from the bottom up by staff most familiar with day-to-day program activities, but we determined such verification is not being done.



In addition, in fiscal year 2001, DOI and its bureaus did not have adequate procedures to accumulate sufficient information to support the data included in the departmental GPRA Report for three12 of the four indicators.  Field offices should maintain a list of the accomplishments reported into GPRA, which would allow state/regional and national offices to match and verify this data.  Managers could use this information to readily identify their bureausï¿½ accomplishments for other purposes as well.



New DOI Data Verification Policy
A January 2003 memorandum from DOIï¿½s Assistant Secretary for Policy, Management and Budget (PMB) required upper level DOI and bureau managers to ensure that their organizations have proper performance data protocols in place within 90 days.  The organizations have discretion in establishing protocols based on guidance provided in the Data Validation and Verification Assessment Matrix attached to the memorandum.  The Matrix includes standards and procedures for defining source data, maintaining supporting documentation, and training staff in data collection.  It also provides criteria for data entry procedures, data security, defining data limitations, and oversight and certification.  The effective implementation of this new policy would address the data accuracy issues raised in this report.  In addition, the Departmentï¿½s new strategic planning process should address concerns over misclassification of performance data and incomplete or inaccurate reporting. 


Improving the Usefulness of the GPRA Report to Decision Makers
DOI needs appropriate goals and indicators and accurate information to achieve the major GPRA objective of helping Congress develop a clearer understanding of achievements in relation to expenditures.  Congress can then determine whether progress is being made and whether the level of performance is sufficient to justify federal resources and effort.



Establishing appropriate goals and indicators that focus on mission results is key to the performance management process.  DOIï¿½s choice of GPRA indicators should identify its highest program priorities to its own managers and employees, Congress, and OMB.  To make DOIï¿½s GPRA Reports more useful to decision makers, indicators should provide a more comprehensive picture of its mission, performance, and results.  In addition, accurate, sufficient, timely, and relevant information on program performance would provide DOI with better control over resources used and improve accountability for results by program managers.


Recommendation



We recommend that the Department establish an oversight process to ensure bureaus and offices implement the requirement for data verification set forth in the January 2003 memorandum issued by the Assistant Secretary, PMB.


Agency Response and Office of Inspector General Reply
DOI concurred with this recommendation.  The Departmentï¿½s response and subsequent information provided indicated that GPRA-related performance measures have been incorporated into senior manager performance development plans to foster greater accountability for and investment in performance data, data credibility, and results. We consider this recommendation implemented and no further action is required by the Department 




Appendix 1
Page 1 of 1
Audit Scope and Methodology



Government Auditing Standards
We conducted our audit in accordance with the Government Auditing Standards, issued by the Comptroller General of the United States.  Accordingly, we included such tests of records and other auditing procedures considered necessary to accomplish our objectives.  As part of our audit, we reviewed the Department of the Interiorï¿½s Annual Departmental Report on Accountability and the individual bureau reports for fiscal year 2001, which include information required by the Federal Managersï¿½ Financial Integrity Act.  Based on that review, we determined that none of the weaknesses reported for the Department or bureaus were within the objectives and scope of our review.


Scope and Methodology
The scope of our audit included a review of DOI compliance with GPRA and OMB requirements establishing performance goals and indicators, reporting performance accomplishments, and verifying and validating performance data.  To accomplish our objectives, we reviewed DOI performance goals, indicators, and accomplishments for fiscal year 2001 and performance plans for fiscal years 2001 and 2002.  Our fieldwork was performed from May 2002 through January 2003.







Appendix 2
Page 1 of 2
Audit Sites Visited




We visited or contacted DOI headquarters offices and five bureaus--BIA, BLM, BOR, FWS, and NPS.  We visited several regional offices and field offices judgmentally selected throughout the United States, as identified below.



Sites Visited
Location




Department of the Interior:


Office of Planning and Performance Management
Washington, DC

Office of Financial Management 
Washington, DC

Natural Resource Damage Assessment & Restoration Program, Office of Wildland Fire Coordination
Washington, DC

South Florida Ecosystem Restoration Task Force
Miami, FL




Bureau of Indian Affairs:


Strategic Planning Office, Division of Forestry
Washington, DC

Fire and Aviation Management
Boise, ID

Northern California Agency & Pacific Regional Office
Redding, CA

Northwest Regional Office
Portland, OR




Bureau of Land Management:


Office of Business and Fiscal Resources - Budget
Washington, DC

Office of Fire and Aviation
Boise, ID

Redding Field Office & California State Office
Redding, CA

Colorado State Office
Lakewood, CO

Oregon/Washington State Office
Portland, OR

Salem District Office
Salem, OR




U.S. Fish and Wildlife Service:


Planning and Evaluation Staff
Washington, DC

National Wildlife Refuge (NWR) System
Arlington, VA

Division of Bird Habitat Conservation
Arlington, VA

Division of Endangered Species
Arlington, VA

Fire Management Branch
Boise, ID




Mountain-Prairie Regional Office
Denver, CO

	Arapaho NWR, Rocky Mountain Arsenal NWR
Colorado




Pacific Regional Office
Portland, OR

	Willamette Valley NWR Complex
Oregon




Southeast Regional Office
Atlanta, GA

	A.R.M. Loxahatchee NWR, Lake Woodruff NWR, South Florida Field Office
Florida






Appendix 2
Page 2 of 2



Southwest Region:


	Sacramento NWR Complex
California

	San Francisco Bay NWR Complex
California




National Park Service:


Historic Preservation, Recreation & Partnerships
Washington, DC

Park Historic Structures & Cultural Landscapes
Washington, DC

Natural Resource Stewardship & Science
Washington, DC

Office of Strategic Planning
Denver, CO

Fire Management Program Center
Boise, ID




Intermountain Regional Office
Denver, CO

	Rocky Mountain National Park
Colorado




Pacific West Regional Office
Oakland, CA

	Whiskeytown-Shasta-Trinity NRA
California




North East Regional Office
Philadelphia, PA

	Delaware Water Gap NRA
Pennsylvania

	Gettysburg National Military Park
Pennsylvania

	Independence National Historic Park
Pennsylvania

	Valley Forge National Historic Park
Pennsylvania




Southeast Regional Office
Atlanta, GA

	Big Cypress National Preserve
Florida

	Biscayne National Park, 
       Everglades National Park
Florida




We also contacted additional offices via telephone, e-mail, and/or facsimile.





Appendix 3
Page 1 of 2
Prior Audit Coverage



Prior Audit Coverage
The following Office of Inspector General (OIG) and General Accounting Office (GAO) reports are related to either GPRA reporting or challenges identified in the program areas we reviewed.


GAO-03-104
> In its January 2003 performance and accountability report, GAO identified specific performance and management challenges facing DOI.13  According to GAO, although DOI has taken steps to better manage the ecosystem restoration efforts, more work is needed to address wildfire threats caused by excessive fuels buildup and to complete actions that will improve the South Florida ecosystem restoration effort.  Additionally, better data-gathering techniques are needed to accurately assess the condition of NPSï¿½ cultural properties.


2002-I-0047
> In 2002, we reported that BLM needed to improve its development and reporting on GPRA goals and measures by revising them to focus on its mission and performance.14  We also recommended the establishment of a data verification and validation process to ensure the accuracy of the GPRA data and the development of procedures to accumulate information supporting reported GPRA data.


GAO-01-872T
> In June 2001 GAO testified that GPRA is the centerpiece of a management infrastructure that can provide information to assist Congress and decision makers assess what the federal government should do in the 21st century and how.15  Performance information can help build a better-equipped government to deliver 






Appendix 3
Page 2 of 2



economical, efficient and effective programs.  Among the major challenges identified were the:



o need for a results orientation,
o assurance that daily operations contribute to results, and
o capacity to gather and use performance information.


00-I-533
> In 2000, we reported on our review of the Departmentï¿½s and its bureaus Fiscal Year 1999 Performance Reports and 2001 Annual Performance Plans.16  Departmentwide suggestions for improvements in the performance measures included eliminating or modifying goals that are not program or function-related, establishing goals for all significant programs and functions, describing the method used to establish target levels of accomplishment, indicating the relevance of performance measures, and establishing appropriate measures as performance indicators.  Concerning data reliability and verification we found that the issue of data validation and verification were not addressed by those bureaus that had significantly revised their performance plans and had not established performance baselines.  Additionally, although data collection shortcomings and difficulties encountered in obtaining reliable data were disclosed, only FWS described data limitations for each performance measure.






A



Appendix 5
Page 1 of 1
Status of Audit Recommendations




Recommendation
Status
Action Required



1 and 2
Implemented
No further action by the Department is required.





3 According to DOI managers, the new performance plan will not include program specific indicators such as the South Florida Ecosystem Restoration Program.
4 STAï¿½s are man-made wetlands constructed to improve water quality by treating urban and agricultural run off before it is discharged to the natural areas.
5 In Table 1, indicators 8 and 9 pertain to FWS, and indicators 10 and 11 pertain to NPS.
6 Staff at one refuge agreed that they should have reported 2,693 acres instead of 8,000.  The other refuge had project documentation for an additional unreported 39 acres.
7 FWS distinguishes between habitat management and enhancement and habitat restoration actions.  Habitat management and enhancement includes activities such as water level management, grazing, haying, farming, forest management, and invasive plant control.
8 The Park reported 1,700 acres, or 1 percent, of the total 170,000-acre project.
9 The LCS is maintained by the national program office and should be the source of certified PMDS data.  National parks also report the number and condition of their historic structures into the PMDS as uncertified data.
10 There are seven performance measures that include the 13 performance indicators.  See Table 1.
11 The NPS performance report accurately disclosed this information.
12 Acres of mined land, refuges, park lands, and forests restored; acres treated to restore natural processes; and number of historic structures in good condition.
13 Major Management Challenges and Program Risks:  Department of the Interior, Report No. GAO-03-104, January 2003.
14 Improvements Needed in Developing and Reporting on GPRA Goals and Measures:  Reducing Threats to Public Health, Safety, and Property, OIG Report No. 2002-I-0047.
15 Managing for Results:  Using GPRA to Assist Oversight and Decisionmaking, Report No. GAO-01-872T, 
June 19, 2001.
16 Review of the Fiscal Year 1999 Performance Reports and Fiscal Year 2001 Performance Plans for the U.S. Department of the Interior, OIG Report No. 00-I-533, June 2000.