Performance Budgeting: OMB's Program Assessment Rating Tool Presents Opportunities and Challenges For Budget and Performance Integration

Performance Budgeting: OMB's Program Assessment Rating Tool
Presents Opportunities and Challenges For Budget and Performance
Integration (04-FEB-04, GAO-04-439T).

The Office of Management and Budget's (OMB) Program Assessment
Rating Tool (PART) is meant to provide a consistent approach to
evaluating federal programs during budget formulation. The
subcommittee asked GAO to discuss our recent report, Performance
Budgeting: Observations on the Use of OMB's Program Assessment
Rating Tool for the Fiscal 2004 Budget (GAO-04-174) and
strategies for improving PART and furthering the goals envisioned
by the Government Performance and Results Act of 1993 (GPRA).
-------------------------Indexing Terms-------------------------
REPORTNUM: GAO-04-439T
ACCNO: A09213
TITLE: Performance Budgeting: OMB's Program Assessment Rating
Tool Presents Opportunities and Challenges For Budget and
Performance Integration
DATE: 02/04/2004
SUBJECT: Congressional/executive relations
Evaluation methods
Performance measures
Planning programming budgeting
Program evaluation
OMB Program Assessment Rating Tool

******************************************************************
** This file contains an ASCII representation of the text of a **
** GAO Product. **
** **
** No attempt has been made to display graphic images, although **
** figure captions are reproduced. Tables are included, but **
** may not resemble those in the printed version. **
** **
** Please see the PDF (Portable Document Format) file, when **
** available, for a complete electronic file of the printed **
** document's contents. **
** **
******************************************************************
GAO-04-439T

United States General Accounting Office

GAO Testimony

Before the Subcommittee on Government Efficiency and Financial Management Committee on Government Reform, House of Representatives

For Release on Delivery Expected at 2:00 p.m. EST Wednesday February 4, 2004

PERFORMANCE BUDGETING

OMB's Program Assessment Rating Tool Presents Opportunities and Challenges For Budget and Performance Integration

Statement of Paul L. Posner
Managing Director, Federal Budget Issues
Strategic Issues

GAO-04-439T

Highlights of GAO-04-439T, a testimony before the Subcommittee on
Government Efficiency and Financial Management, Committee on Government
Reform, House of Representatives.

In our recent report on PART we recommended that the Director of OMB (1)
address the capacity demands of PART, (2) strengthen PART guidance, (3)
address evaluation information scope and availability issues, (4) focus
program selection on critical operations and crosscutting comparisons, (5)
expand the dialogue with Congress, and (6) articulate and implement a
complementary relationship between PART and GPRA.

OMB generally agreed with our findings, conclusions, and recommendations
and said it is already taking actions to address many of our
recommendations.

We also suggested that Congress consider the need for a structured
approach to articulating its perspective and oversight agenda on
performance goals and priorities for key programs.

www.gao.gov/cgi-bin/getrpt?GAO-04-439T.

To view the full product, including the scope and methodology, click on the link
above. For more information, contact Paul Posner at (202) 512-9573 or
[email protected].

February 2004

PERFORMANCE BUDGETING

OMB's Program Assessment Rating Tool Presents Opportunities and Challenges For
Budget and Performance Integration

PART helped structure OMB's use of performance information for internal
program and budget analysis and stimulated agency interest in budget and
performance integration. Moreover, it illustrated the potential to build
on GPRA's foundation to more actively promote the use of performance
information in budget decisions. OMB deserves credit for inviting scrutiny
of its federal program performance reviews and sharing them on its Web
site.

Much of PART's potential value lies in its program recommendations but
follow through will require sustained commitment by agencies and OMB. OMB
devoted considerable effort to developing PART, but diagnosing problems
and rating programs are only the beginning of PART's ambitious agenda.
Implementing change and providing oversight takes time; OMB needs to be
mindful of this as it considers capacity and workload issues in the PART.

As is to be expected in the first year of any reform, PART is a work in
progress and we noted in our report where OMB might make improvements. Any
tool that is sophisticated enough to take into account the complexity of
the U.S. government will require exercising some judgment. Therefore it is
not surprising that we found inconsistencies in OMB staff interpreting and
applying PART.

PART provides an opportunity to more efficiently use scarce analytic
resources, to focus decision makers' attention on the most pressing policy
issues, and to consider comparisons and trade-offs among related programs
by more strategically targeting PART assessments based on such factors as
the relative priorities, costs, and risks associated with related clusters
of programs and activities. PART assessments underscored long-standing
gaps in performance and evaluation information throughout the federal
government. By reaching agreement on areas in which evaluations are most
essential, decision makers can help ensure that limited resources are
applied wisely.

The relationship between PART and the broader GPRA strategic planning
process is still evolving. Although PART can stimulate discussion on
programspecific performance measurement issues, it is not a substitute for
GPRA's strategic, longer-term focus on thematic goals, and department-and
governmentwide crosscutting comparisons. Although PART and GPRA serve
different needs, a strategy for integrating the two could help strengthen
both.

Federal programs are designed and implemented in dynamic environments
where competing program priorities and stakeholders' needs must be
balanced continually and new needs addressed. PART clearly serves OMB's
needs but questions remain about whether it serves the various needs of
other key stakeholders. If PART results are to be considered in the
congressional debate it will be important for OMB to (1) involve
congressional stakeholders early in providing input on the focus of the
assessments; (2) clarify any significant limitations in the assessments
and underlying performance information; and (3) initiate discussions with
key congressional committees about how they can best leverage PART
information in congressional authorization, appropriations, and oversight
processes.

Mr. Chairman and Members of the Subcommittee:

I am pleased to be here today to discuss performance budgeting and the Office of Management and Budget's (OMB) Program Assessment Rating Tool (PART). Since the 1950s, the federal government has attempted several governmentwide initiatives designed to better align spending decisions with expected performance-what is commonly referred to as "performance budgeting." The consensus is that prior efforts-including the Hoover Commission, the PlanningProgrammingBudgetingSystem (PPBS), Management by Objectives, and ZeroBased Budgeting (ZBB)-did not succeed in significantly shifting the focus of the federal budget process from its longstanding concentration on the items of government spending to the results of its programs. However, the persistent attempts reflect a longstanding interest in linking resources to results.

In the 1990s, Congress and the executive branch laid out a statutory and management framework that provides the foundation for strengthening government performance and accountability, with the Government Performance and Results Act of 19931 (GPRA) as its centerpiece. GPRA is designed to inform congressional and executive decision making by providing objective information on the relative effectiveness and efficiency of federal programs and spending. A key purpose of the act is to create closer and clearer links between the process of allocating scarce resources and the expected results to be achieved with those resources. We have learned that this type of integration is critical from prior initiatives that failed in part because they did not prove to be relevant to budget decision makers in the executive branch or Congress.2 GPRA requires both a connection to the structures used in congressional budget presentations and consultation between the executive and legislative branches on agency strategic plans; this gives Congress an oversight stake in GPRA's success.3

1 Pub. L. No. 10362 (1993).

2 U.S. General Accounting Office, Performance Budgeting: Past Initiatives
Offer Insights for GPRA
Implementation, GAO/AIMD9746 (Washington, D.C.: Mar. 27, 1997).

3 See Pub. L. No. 10362 S: 2 (1993), 5 U.S.C. S: 306 (2003), and 31 U.S.C. S:S: 11151116 (2003).

This administration has made the integration of performance and budget information one of five governmentwide management priorities under the President's Management Agenda (PMA).4 Central to this initiative is the PART. OMB developed PART as a diagnostic tool meant to provide a consistent approach to evaluating federal programs and applied it in formulating the President's fiscal year 2004 budget request. PART covers four broad topics for all "programs"5 selected for review: (1) program purpose and design, (2) strategic planning, (3) program management, and (4) program results (i.e., whether a program is meeting its longterm and annual goals) as well as additional questions that are specific to one of seven mechanisms or approaches used to deliver the program.6

GPRA expanded the supply of performance information generated by federal agencies, although as the PART assessments demonstrate, more must be done to develop credible performance information. However, improving the supply of performance information is in and of itself insufficient to sustain performance management and achieve real improvements in management and program results. Rather, it needs to be accompanied by a demand for that information by decision makers and managers alike. PART may mark a new chapter in performancebased budgeting by more successfully stimulating demand for this information- that is, using the performance information generated through GPRA's planning and reporting processes to more directly feed into executive branch budgetary decisions.

My statement today focuses on six points:

o
" PART helped structure OMB's use of performance information for its internal program and budget analysis, made the use of this information

4 In addition to budget and performance integration, the other four priorities under the PMA are strategic management of human capital, expanded electronic government, improved financial performance, and competitive sourcing.

5 There is no standard definition for the term "program." For purposes of PART, OMB described the unit of analysis (program) as (1) an activity or set of activities clearly recognized as a program by the public, OMB, or Congress; (2) having a discrete level of funding clearly associated with it; and (3) corresponding to the level at which budget decisions are made.

6 The seven major categories are competitive grants, block/formula grants, capital assets and service acquisition programs, credit programs, regulatorybased programs, direct federal programs, and research and development programs. Tax programs were not addressed for the fiscal year 2004 PART process.

more transparent, and stimulated agency interest in budget and performance integration. Moreover, it illustrated the potential to build on GPRA's foundation to more actively promote the use of performance information in budget decisions.

o
" Much of the potential value of PART lies in the related program recommendations and associated improvements, but follow through will require sustained commitment by agencies and OMB. OMB's efforts in developing PART have been considerable, but diagnosing problems and applying ratings are the beginning not the end of PART's ambitious agenda. Identifying solutions, implementing changes, and providing oversight takes time, and OMB needs to be mindful of this as it considers the capacity and workload issues in the PART.

o
" As is to be expected in the first year of any reform, PART is a work in progress and we noted in our report where OMB might make improvements. Any tool that is sophisticated enough to take into account the complexity of the U.S. government will require some exercise of judgment. Therefore it is not surprising that we found some inconsistencies in OMB staff interpreting and applying PART.

o
" PART provides an opportunity to more efficiently use scarce analytic resources, to focus decision makers' attention on the most pressing policy issues, and to consider comparisons and tradeoffs among related programs by more strategically targeting PART assessments based on such factors as the relative priorities, costs, and risks associated with related clusters of programs and activities. The first year PART assessments underscored the longstanding gaps in performance and evaluation information throughout the federal government. By reaching agreement on areas in which evaluations are most essential, decision makers can help ensure that limited resources are applied wisely.

o
" The relationship between PART and its process and the broader GPRA strategic planning process is still evolving. Although PART can stimulate discussion on programspecific performance measurement issues, it is not a substitute for GPRA's strategic, longerterm focus on thematic goals and departmentand governmentwide crosscutting comparisons. Although PART and GPRA serve different needs, a strategy for integrating the two could help strengthen both.

o
" Federal programs are designed and implemented in dynamic environments where competing program priorities and stakeholders'

needs must be balanced continually and new needs must be addressed. While PART clearly serves the needs of OMB in budget formulation, questions remain about whether it serves the various needs of other key stakeholders. If the President or OMB wants the PART and its results to be considered in the congressional debate, it will be important for OMB to (1) involve congressional stakeholders early in providing input on the focus of the assessments; (2) clarify any significant limitations in the assessments as well as the underlying performance information; and (3) initiate discussions with key congressional committees about how they can best take advantage of and leverage PART information in congressional authorization, appropriations, and oversight processes. Moreover, Congress needs to consider ways it can articulate its oversight priorities and performance agenda.

My statement is based on our recently published report on OMB's PART.7 This subcommittee along with several other requesters asked GAO to review the application of PART in its first year, its relationship to GPRA planning and reporting requirements, and its strengths and weaknesses as an evaluation tool. We conducted our work in accordance with generally accepted government auditing standards.

Strengths and Weaknesses of PART in Its First Year of Implementation

Through its development and use of PART, OMB has more explicitly infused performance information into the budget formulation process; increased the attention paid to evaluation and to performance information; and ultimately, we hope, increased the value of this information to decision makers and other stakeholders. By linking performance information to the budget process, OMB has provided agencies with a powerful incentive for improving both the quality and availability of performance information. The level of effort and involvement by senior OMB officials and staff clearly signals the importance of this strategy in meeting the priorities outlined in the PMA. OMB should be credited with opening up for scrutiny-and potential criticism-its review of key areas of federal program performance and then making its assessments available to a potentially wider audience through its Web site.

7 U.S. General Accounting Office, Performance Budgeting: Observations on
the Use of OMB's Program Assessment Rating Tool for the Fiscal Year 2004
Budget, GAO04174 (Washington, D.C.: Jan. 30, 2004).

As OMB and others recognize, performance is not the only factor in funding decisions. Determining priorities-including funding priorities-is a function of competing values and interests. Accordingly, we found that while PART scores were generally positively related to proposed funding changes in discretionary programs, the scores did not automatically determine funding changes. That is, for some programs rated "effective" or "moderately effective" OMB recommended funding decreases, while for several programs judged to be "ineffective" OMB recommended additional funding in the President's budget request with which to implement changes. In fact, the more important role of PART was not its use in making resource decisions, but in its support for recommendations to improve program design, assessment, and management.

As shown in figure 1, we found that 82 percent of PART's recommendations addressed program assessment, design, and management issues; only 18 percent of the recommendations had a direct link to funding matters.8

Figure 1: Fiscal Year 2004 PART Recommendations

8 The 234 programs assessed for fiscal year 2004 contained a total of 612 recommendations.

OMB's ability to use PART to identify and address future program improvements and measure progress-a major purpose of PART-depends on its ability to oversee the implementation of PART recommendations. As OMB has recognized, following through on these recommendations is essential for improving program performance and ensuring accountability. Currently, OMB plans to assess an additional 20 percent of all federal programs annually. As the number of recommendations from previous years' evaluations grows, a system for monitoring their implementation will become more critical. However, OMB does not have a centralized system to oversee the implementation of such recommendations or evaluate their effectiveness.

The goal of PART is to evaluate programs systematically, consistently, and transparently. OMB went to great lengths to encourage consistent application of PART in the evaluation of government programs, including pilot testing the instrument, issuing detailed guidance, and conducting consistency reviews. Although there is undoubtedly room for continued improvement, any tool is inherently limited in providing a single performance answer or judgment on complex federal programs with multiple goals.

Performance measurement challenges in evaluating complex federal programs make it difficult to meaningfully interpret a bottomline rating. OMB published both a single, bottomline rating for PART results and individual section scores. It is these latter scores that are potentially more useful for identifying information gaps and program weaknesses. For example, one program that was rated "adequate" overall got high scores for purpose (80 percent) and planning (100 percent), but poor scores in being able to show results (39 percent) and in program management (46 percent). In a case like this, the individual section ratings provided a better understanding of areas needing improvement than the overall rating alone. In addition, bottomline ratings may force raters to choose among several important, but disparate goals and encourage a determination of program effectiveness even when performance data are unavailable, the quality of those data is uneven, or they convey a mixed message on performance.

Any tool that is sophisticated enough to take into account the complexity of the U.S. government will always require some interpretation and judgment. Therefore it is not surprising that OMB staff were not fully consistent in interpreting complex questions about agency goals and results. In addition, the limited availability of credible evidence on program

results also constrained OMB's ability to use PART to rate programs' effectiveness.

Many PART questions contain subjective terms that are open to interpretation. Examples include terminology such as "ambitious" in describing soughtafter performance measures. Because the appropriateness of a performance measure depends on the program's purpose, and because program purposes can vary immensely, an ambitious goal for one program might be unrealistic for a similar but more narrowly defined program. Without further guidance, it is unclear how OMB staff can be expected to be consistent.

We found inconsistencies in how the definition of acceptable performance measures was applied. Our review surfaced several instances in which OMB staff inconsistently defined appropriate measures-outcome versus output-for programs. Agency officials also told us that OMB staff used different standards to define measures as outcomeoriented. Outputs are the products and services delivered by the program whereas outcomes refer to the results of outputs. For example, in the employment and training area, OMB accepted shortterm outcomes, such as obtaining high school diplomas or employment, as a proxy for longterm goals for the Department of Health and Human Services' Refugee Assistance program, which aims to help refugees attain economic selfsufficiency as soon as possible. However, OMB did not accept the same employment rate measure as a proxy for longterm goals for the Department of Education's Vocational Rehabilitation program because it had not set longterm targets beyond a couple of years. In other words, although neither program contained longterm outcomes, such as participants gaining economic selfsufficiency, OMB accepted shortterm outcomes in one instance but not the other.

The yes/no format employed throughout most of the PART questionnaire resulted in oversimplified answers to some questions. Although OMB believes it helped standardization, the yes/no format was particularly troublesome for questions containing multiple criteria for a "yes" answer. Agency officials have commented that the yes/no format is a crude reflection of reality, in which progress in planning, management, or results is more likely to resemble a continuum than an on/off switch. We found several instances in which some OMB staff gave a "yes" answer for successfully achieving some but not all of the multiple criteria, while others gave a "no" answer when presented with a similar situation. For example, OMB judged the Department of the Interior's (DOI) Water Reuse and Recycling program "no" on whether a program has a limited number of

ambitious, longterm performance goals, noting that although DOI set a longterm goal of 500,000 acrefeet per year of reclaimed water, it failed to establish a time frame for when it would reach the target. However, OMB judged the Department of Agriculture's and DOI's Wildland Fire programs "yes" on this question even though the programs' longterm goals of improved conditions in highpriority forest acres are not accompanied by specific time frames.

The lack of program performance information also creates challenges in effectively measuring program performance. According to OMB, about half of the programs assessed for fiscal year 2004 lacked "specific, ambitious longterm performance goals that focus on outcomes" and nearly 40 percent lacked sufficient "independent, quality evaluations." Nearly 50 percent of programs assessed for fiscal year 2004 received ratings of "results not demonstrated" because OMB decided that program performance information, performance goals, or both were insufficient or inadequate. While the validity of these assessments may be subject to interpretation and debate, our previous work9 has raised concerns about the capacity of federal agencies to produce evaluations of program effectiveness as well as credible data.

The Relationship between GPRA and PART

PART was designed for and is used in the executive branch budget preparation and review process. As a result, the goals and measures used in PART must meet OMB's needs. By comparison, GPRA-the current statutory framework for strategic planning and reporting-is a broader process involving the development of strategic and performance goals and objectives to be reported in strategic and annual plans and reports. OMB said that GPRA plans were organized at too high a level to be meaningful for programlevel budget analysis and management review. OMB acknowledges that GPRA was the starting point for PART, but as I will explain, it appears that OMB's emphasis is shifting such that over time the performance measures developed for PART and used in the budget process may also come to drive agencies' strategic planning processes.

9 U.S. General Accounting Office, Program Evaluation: Agencies Challenged
by New Demand for Information on Program
Results, GAO/GGD9853 (Washington, D.C.: Apr. 24, 1998).

The fiscal year 2004 PART process came to be a parallel competing structure to the GPRA framework as a result of OMB's desire to collect performance data that better align with budget decision units. OMB's most recent Circular A11 guidance clearly requires both that each agency submit a performance budget for fiscal year 2005 and that this should replace the annual GPRA performance plan.10 These performance budgets are to include information from the PART assessments, where available, including all performance goals used in the assessment of program performance done under the PART process. Until all programs have been assessed using PART, the performance budget will also include performance goals for agency programs that have not yet been assessed. OMB's movement from GPRA to PART is further evident in the fiscal year 2005 PART guidance stating that while existing GPRA performance goals may be a starting point during the development of PART performance goals, the GPRA goals in agency GPRA documents are to be revised, as needed, to reflect OMB's instructions for developing the PART performance goals. Lastly, this same guidance states that GPRA plans should be revised to include any new performance measures used in PART and that unnecessary measures should be deleted from GPRA plans.

Although there is potential for complementary approaches to GPRA and PART, the following examples clearly illustrate the importance of carefully considering the implications of selecting a unit of analysis, including its impact on the availability of performance data. They also reveal some of the unresolved tensions between the President's budget and performance initiative-a detailed budget perspective-and GPRA-a more strategic planning view. Experience with the PART highlighted the fact that defining a "unit of analysis" useful for both programlevel budget analysis and agency planning purposes can be difficult. For example, disaggregating programs for PART purposes could ignore the interdependence of programs recognized by GPRA by artificially isolating programs from the larger contexts in which they operate. Agency officials described one program assessed with the PART-Projects for Assistance in Transition from Homelessness-that was aimed at a specific aspect of homelessness, that is, referring persons with emergency needs to other agencies for housing and needed services. OMB staff wanted the agency to produce longterm outcome measures for this program to support the PART review process. Agency officials argued that chronically homeless people require many services, and that this federal program often supports only some of

10 OMB Circular A11, Preparation, Submission, and Execution of the
Budget.

the services needed at the initial stages of intervention. GPRA-with its focus on assessing the relative contributions of related programs to broader goals-is better designed to consider crosscutting strategies to achieve common goals. Federal programs cannot be assessed in isolation. Performance needs also to be examined from an integrated, strategic perspective.

One way of improving the links between PART and GPRA would be to develop a more strategic approach to selecting and prioritizing areas for assessment under the PART process. Targeting PART assessments based on such factors as the relative priorities, costs, and risks associated with related clusters of programs and activities addressing common strategic and performance goals not only could help ration scarce analytic resources but also could focus decision makers' attention on the most pressing policy and program issues. Moreover, such an approach could facilitate the use of PART assessments to review the relative contributions of similar programs to common or crosscutting goals and outcomes established through the GPRA process.

The Importance of Congressional and Other Stakeholder Involvement

We have previously reported11 that stakeholder involvement appears critical for getting consensus on goals and measures. In fact, GPRA requires agencies to consult with Congress and solicit the views of other stakeholders as they develop their strategic plans.12 Stakeholder involvement can be particularly important for federal agencies because they operate in a complex political environment in which legislative mandates are often broadly stated and some stakeholders may strongly disagree about the agency's mission and goals.

The relationship between PART and its process and the broader GPRA strategic planning process is still evolving. As part of the executive branch budget formulation process, PART must clearly serve the President's interests. Some tension about the amount of stakeholder involvement in the internal deliberations surrounding the development of PART measures and the broader consultations more common to the GPRA strategic

11 U.S. General Accounting Office, Agencies' Strategic Plans Under GPRA:
Key Questions to Facilitate Congressional Review (Version 1),
GAO/GGD10.1.16 (Washington, D.C.: May 1997).

12 5 U.S.C. S: 306(d) (2003).

planning process is inevitable. Compared to the relatively openended GPRA process, any budget formulation process is likely to seem closed.

Yet, we must ask whether the broad range of congressional officials with a stake in how programs perform will use PART assessments unless they believe the reviews reflect a consensus about performance goals among a community of interests, target performance issues that are important to them as well as the administration, and are based on an evaluation process that they have confidence in. Similarly, the measures used to demonstrate progress toward a goal, no matter how worthwhile, cannot serve the interests of a single stakeholder or purpose without potentially discouraging use of this information by others. Accordingly, if PART is to be accepted as other than one element in the development of the President's budget proposal, congressional understanding and acceptance of the tool and analysis will be important.

Congress has a number of opportunities to provide its perspective on performance issues and performance goals, such as when it establishes or reauthorizes a new program, during the annual appropriations process, and in its oversight of federal operations. In fact, these processes already reflect GPRA's influence. Reviews of language in public laws and committee reports show an increasing number of references to GPRArelated provisions. What is missing is a mechanism to systematically coordinate a congressional perspective.

In our report, we have suggested steps for both OMB and the Congress to take to strengthen the dialogue between executive officials and congressional stakeholders. We have recommended that OMB reach out to key congressional committees early in the PART selection process to gain insight about which program areas and performance issues congressional officials consider warrant PART review. Engaging Congress early in the process may help target reviews with an eye toward those areas most likely to be on the agenda of the Congress, thereby better ensuring the use of performance assessments in resource allocation processes throughout government. We have also suggested that Congress consider the need to develop a more systematic vehicle for communicating its top performance concerns and priorities; develop a more structured oversight agenda to prompt a more coordinated congressional perspective on crosscutting performance issues; and use this agenda to inform its authorization, appropriations, and oversight processes.

Concluding Observations

The PART process is the latest initiative in a longstanding series of reforms undertaken to improve the link between performance information and budget decisions. Although each of the initiatives of the past appears to have met with an early demise, in fact, subsequent reforms were strengthened by building on the legacy left by their predecessors. Prior reforms often failed because they were not relevant to resource allocation and other decision making processes, thereby eroding the incentives for federal agencies to improve their planning, data, and evaluations.

Unlike many of those past initiatives, GPRA has been sustained since its passage 10 years ago, and evidence exists that it has become more relevant than its predecessors. PART offers the potential to build on the infrastructure of performance plans and information ushered in by GPRA and the law's intent to promote the use of these plans in resource allocation decision making. GPRA improved the supply of plans and information, while PART can prompt greater demand for this information by decision makers. Potentially, enhancing interest and use may bring about greater incentives by agencies to devote scarce resources to improving their information and evaluations of federal programs as well.

Increasing the use and usefulness of performance data is not only important to sustain performance management reforms, but to improve the processes of decision making and governance. Many in the U.S. believe there is a need to establish a comprehensive portfolio of key national performance indicators. This will raise complex issues ranging from agreement on performance areas and indicators to getting and sharing reliable information for public planning, decision making, and accountability. In this regard, the entire agenda of management reform at the federal level has been focused on shifting decision making and agency management from process to results. Although the PART is based on changing the orientation of budgeting, other initiatives championed by Congress and embodied in the PMA are also devoted to improving the accountability for performance goals in agency human capital management, financial management, competitive sourcing, and other key management areas.

In particular, we have reported that human capital-or people-is at the center of any serious change management initiative. Thus, strategic human capital management is at the heart of government transformation. Highperforming organizations strengthen the alignment of their GPRA strategic and performance goals with their daily operations. In that regard,

performance management systems can be a vital-but currently largely unused tool-to align an organization's operations with individual daytoday activities. As we move forward to strengthen government performance and accountability, effective performance management systems can be a strategic tool to drive internal change and achieve desired results.

The question now is how to enhance the credibility and use of the PART process as a tool to focus decisions on performance. In our report, we make seven recommendations to OMB and a suggestion to Congress to better support the kind of collaborative approach to performance budgeting that very well may be essential in a separation of powers system like ours. Our suggestions cover several key issues that need to be addressed to strengthen and help sustain the PART process. We recommend that the OMB Director take the following actions:

o
" Centrally monitor agency implementation and progress on PART recommendations and report such progress in OMB's budget submission to Congress. Governmentwide councils may be effective vehicles for assisting OMB in these efforts.

o
" Continue to improve the PART guidance by (1) expanding the discussion of how the unit of analysis is to be determined to include tradeoffs made when defining a unit of analysis, implications of how the unit of analysis is defined, or both; (2) clarifying when output versus outcome measures are acceptable; and (3) better defining an "independent, quality evaluation."

o
" Clarify OMB's expectations to agencies regarding the allocation of scarce evaluation resources among programs, the timing of such evaluations, as well as the evaluation strategies it wants for the PART, and consider using internal agency evaluations as evidence on a casebycase basis-whether conducted by agencies, contractors, or other parties.

o
" Reconsider plans for 100 percent coverage of federal programs and, instead, target for review a significant percentage of major and meaningful government programs based on such factors as the relative priorities, costs, and risks associated with related clusters of programs and activities.

o
" Maximize the opportunity to review similar programs or activities in the same year to facilitate comparisons and tradeoffs.

o
" Attempt to generate, early in the PART process, an ongoing, meaningful dialogue with congressional appropriations, authorization, and oversight committees about what they consider to be the most important performance issues and program areas that warrant review.

o
" Seek to achieve the greatest benefit from both GPRA and PART by articulating and implementing an integrated, complementary relationship between the two.

In its comments on our report, OMB outlined actions it is taking to address several of these recommendations, including refining the process for monitoring agencies' progress in implementing the PART recommendations, seeking opportunities for dialogue with Congress on agencies' performance, and continuing to improve executive branch implementation of GPRA plans and reports.

Our recommendations to OMB are partly directed at fortifying and enhancing the credibility of the PART itself and the underlying data used to make the judgments. Decision makers across government are more likely to rely on PART data and assessments if the underlying information and the rating process are perceived as being credible, systematic, and consistent. Enhanced OMB guidance and improved strategies for obtaining and evaluating program performance data are vital elements.

The PART process can be made more sustainable if the use of analytic resources at OMB and the agencies is rationalized by reconsidering the goal of 100 percent coverage of all federal programs. Instead, we suggest a more strategic approach to target assessments on related clusters of programs and activities. A more targeted approach stands a better chance of capturing the interest of decision makers throughout the process by focusing their attention on the most pressing policy and program issues on how related programs and tools affect broader crosscutting outcomes and goals. Unfortunately, the governmentwide performance plan required by GPRA has never been engaged to drive budgeting in this way.

Improving the integration of inherently separate but interrelated strategic planning and performance budgeting processes can help support a more strategic focus for PART assessments. GPRA's strategic planning goals could be used to anchor the selection and review of programs by providing a foundation to assess the relative contribution of related programs and tools to broader performance goals and outcomes.

Finally, refining the PART questionnaire and review process, and improving the quality of data are important, but the question of whose interests drive the process is perhaps paramount in our system. Ultimately, the impact of PART on decision making will be a function not only of the President's decisions, but of congressional decisions as well.

Much is at stake in the development of a collaborative performance budgeting process. Not only might the PART reviews come to be disregarded absent congressional involvement, but more important, Congress will lose an opportunity to use the PART process to improve its own decision making and oversight processes.

This is an opportune time for the executive branch and Congress to carefully consider how agencies and committees can best take advantage of and leverage the new information and perspectives coming from the reform agenda under way in the executive branch. Ultimately, the specific approach or process is not important. We face a longterm fiscal imbalance, which will require us to reexamine our existing policies and programs. It is all too easy to accept "the base" as given and to subject only new proposals to scrutiny and analysis. The norm should be to reconsider the relevance or "fit" of any federal program, policy, or activity in today's world and for the future.

Mr. Chairman, this concludes my prepared statement. I would be pleased to answer any questions you or the other members of the Committee may have at this time.

For future contacts regarding this testimony, please call Paul L. Posner, Managing Director, Federal Budget Issues, at (202) 5129573. Individuals making key contributions to this testimony included Denise M. Fantone and Jacqueline Nowicki.

This is a work of the U.S. government and is not subject to copyright
protection in the United States. It may be reproduced and distributed in
its entirety without further permission from GAO. However, because this
work may contain copyrighted images or other material, permission from the
copyright holder may be necessary if you wish to reproduce this material
separately.

GAO's Mission
The General Accounting Office, the audit, evaluation and investigative arm of Congress, exists to support Congress in meeting its constitutional responsibilities and to help improve the performance and accountability of the federal government for the American people. GAO examines the use of public funds; evaluates federal programs and policies; and provides analyses, recommendations, and other assistance to help Congress make informed oversight, policy, and funding decisions. GAO's commitment to good government is reflected in its core values of accountability, integrity, and reliability.

Obtaining Copies of GAO Reports and Testimony

The fastest and easiest way to obtain copies of GAO documents at no cost is through the Internet. GAO's Web site (www.gao.gov) contains abstracts and fulltext files of current reports and testimony and an expanding archive of older products. The Web site features a search engine to help you locate documents using key words and phrases. You can print these documents in their entirety, including charts and other graphics.

Each day, GAO issues a list of newly released reports, testimony, and correspondence. GAO posts this list, known as "Today's Reports," on its Web site daily. The list contains links to the fulltext document files. To have GAO email this list to you every afternoon, go to www.gao.gov and select "Subscribe to email alerts" under the "Order GAO Products" heading.

Order by Mail or Phone
The first copy of each printed report is free. Additional copies are $2 each. A check or money order should be made out to the Superintendent of Documents. GAO also accepts VISA and Mastercard. Orders for 100 or more copies mailed to a single address are discounted 25 percent. Orders should be sent to:

U.S. General Accounting Office 441 G Street NW, Room LM Washington, D.C. 20548

To order by Phone: Voice: (202) 5126000 TDD: (202) 5122537 Fax: (202) 5126061

To Report Fraud, Contact: Web site: www.gao.gov/fraudnet/fraudnet.htm

Waste, and Abuse in Email: [email protected]

Federal Programs Automated answering system: (800) 4245454 or (202) 5127470

Public Affairs
Jeff Nelligan, Managing Director, [email protected] (202) 5124800 U.S. General Accounting Office, 441 G Street NW, Room 7149 Washington, D.C. 20548
*** End of document. ***