Managing for Results: Federal Managers' Views Show Need for Ensuring Top
Leadership Skills (Letter Report, 10/20/2000, GAO/GAO-01-127).

Congress has sought to improve federal management and instill a greater
focus on results by enacting a results oriented statutory framework with
the Government Performance and Results Act (GPRA) as its centerpiece.
Congress and the executive branch realize, of course, that performance
improvements do not take place merely because a set of management
requirements has been put in place. GAO conducted surveys in 1997 and
2000 on agencies' progress towards establishing a focus on results. GAO
observed that progress has been uneven in building the organizational
cultures to create and sustain a focus on results governmentwide. A
significantly higher percentage of managers in 2000 than in 1997
reported that their agencies had provided, arranged, or paid for
training that would help them accomplish two results-oriented
management-related tasks--setting performance goals and implementing the
requirements of GPRA. Overall, the survey results indicate that, in some
keys areas, agencies may be losing ground in their efforts towards
building organizational cultures that support a focus on results. GAO's
2000 survey results indicate that the Senate confirmation process must
ensure that political nominees have the appropriate management and
leadership skills needed to continue to transform federal agencies into
high-performing organizations.

--------------------------- Indexing Terms -----------------------------

 REPORTNUM:  GAO-01-127
     TITLE:  Managing for Results: Federal Managers' Views Show Need
	     for Ensuring Top Leadership Skills
      DATE:  10/20/2000
   SUBJECT:  Productivity in government
	     Performance measures
	     Surveys
	     Strategic planning
	     Agency missions
	     Accountability
	     Presidential appointments
	     Human resources training
IDENTIFIER:  Senior Executive Service
	     GPRA
	     Government Performance and Results Act
	     SES

******************************************************************
** This file contains an ASCII representation of the text of a  **
** GAO Testimony.                                               **
**                                                              **
** No attempt has been made to display graphic images, although **
** figure captions are reproduced.  Tables are included, but    **
** may not resemble those in the printed version.               **
**                                                              **
** Please see the PDF (Portable Document Format) file, when     **
** available, for a complete electronic file of the printed     **
** document's contents.                                         **
**                                                              **
******************************************************************

GAO-01-127

MANAGING FOR RESULTS

Federal Managers' Views Show Need for Ensuring Top Leadership Skills kills

United States General Accounting Office

GAO Report to the Chairman, Subcommittee on Oversight of Government

Management, Restructuring, and the District of Columbia, Committee on
Governmental Affairs, U. S. Senate

October 2000 GAO- 01- 127

United States General Accounting Office Washington, D. C. 20548

Page 1 GAO- 01- 127 Ensuring Management Skills

October 20, 2000 The Honorable George V. Voinovich Chairman, Subcommittee on
Oversight of

Government Management, Restructuring, and the District of Columbia Committee
on Governmental Affairs United States Senate

Dear Mr. Chairman: During the past decade, Congress has sought to improve
federal management and instill a greater focus on results by enacting a
resultsoriented statutory framework with the Government Performance and
Results Act (GPRA) as its centerpiece. In establishing this framework,
Congress was taking an important step to help improve federal management,
accountability, and effectiveness. Congress and the executive branch
realize, of course, that performance improvements do not take place merely
because a set of management requirements has been put in place. Rather,
performance improves if and when managers effectively implement those
requirements and use them as a basis for more informed decisionmaking.

At your request, in 2000, we surveyed managers across the federal government
on their experiences with and perceptions about performance and management
issues. This survey was a follow- up to a survey we did in 1996- 1997 (1997
survey). 1 Because of the subcommittee's interest in ensuring that political
appointees in the next administration have the requisite leadership and
management skills to successfully create and sustain high- performing
federal agencies, you asked us to suggest questions to assist the Senate in
its constitutional role of confirming nominees. In our August 2000 report to
you, we provided a set of suggested questions that the Senate can draw from
during the confirmation process. 2

As agreed, the specific objective of this report was to discuss the
information our 1997 and 2000 surveys provide regarding the progress
agencies are making in establishing a focus on results. This report on the
initial results of our 2000 survey is one component of our response to your

1 The Government Performance and Results Act: 1997 Governmentwide
Implementation Will Be Uneven (GAO/ GGD- 97- 109, June 2, 1997). 2
Confirmation of Political Appointees: Eliciting Nominees' Views on
Leadership and Management Issues (GAO/ GGD- 00- 174, Aug. 11, 2000).

Page 2 GAO- 01- 127 Ensuring Management Skills

request that we survey federal managers. In the coming months, we will
continue to analyze the survey data in combination with other agency
information to explore possible reasons for hurdles agencies face in
becoming high- performing organizations, and to look for solutions. As
agreed, early in 2001, we will provide you with a report containing a more
comprehensive analysis of our 2000 survey results.

Progress has been uneven in building the organizational cultures to create
and sustain a focus on results governmentwide. Our 1997 survey results
suggested that building those cultures was a work in progress across the
federal government. We concluded that agencies' performance could be
enhanced if federal managers' experiences with and more positive perceptions
about results- oriented management practices became more widespread and
pervasive. Managers' experiences and perceptions in our 2000 survey
indicated some positive changes since our 1997 survey; however, there are
still many areas where such changes have not occurred.

Moreover, in several areas, it appears that there has been some stagnation
or retrenchment in managers' perceptions. For example, top leadership's
commitment to achieving results- a critical ingredient in managing for
results- remained stagnant. About 53 percent of managers perceived a strong
commitment to achieving results to a great or very great extent in 2000,
while 57 percent had this perception in 1997- not a statistically
significant change. 3

In another important area- use of performance information for program
management activities- a significantly lower percentage of managers reported
that they were using such information to a great or very great extent in
2000 than in 1997 for five out of eight key management activities we asked
about. These included setting program priorities and adopting new program
approaches or changing work processes. On the other hand, there was a
significant increase in the percentage of managers reporting that they had
performance measures for the programs they were involved with. This
percentage was statistically significantly higher in 2000 than in 1997.
Specifically, managers reported statistically significant increases in 2000
for three of five types of performance measures- outcome, output, and
efficiency measures. Although having performance measures is an important
step in managing for results, the benefit of collecting performance
information is only fully realized when this information is actually used by
managers.

3 In this report, the term “statistical significance” is defined
as a difference that is significant at the .05 or lower probability level.
Results in Brief

Page 3 GAO- 01- 127 Ensuring Management Skills

Successful human capital management strategies are key to maintaining a
highly skilled, energized, and empowered workforce that is focused on
results. Managers' responses to human capital- related questions were
discouraging. About 36 percent of managers responded in 2000 that managers
or supervisors at their level had the decisionmaking authority they needed
to a great or very great extent to help their agencies accomplish their
strategic goals, not a statistically significant improvement over the 31
percent who responded in this way in 1997. The percentage of managers who
reported that, to a great or very great extent, they were held accountable
for the results of the programs for which they were responsible increased
from 55 percent in 1997 to 63 percent in 2000, approaching statistical
significance. 4 However, the differences between this level of
accountability and the level of authority managers perceived was great in
both years. Such an imbalance can inhibit the development of an environment
conducive to achieving results.

Further, a significantly higher percentage of managers in 2000 than in 1997
reported that their agencies had provided, arranged, or paid for training
that would help them accomplish two results- oriented management- related
tasks- setting performance goals and implementing the requirements of GPRA.
However, no significant differences emerged concerning four other tasks,
including using program performance information to make decisions and
developing performance measures. Finally, 31 percent of managers in 2000
said that employees in their agencies received positive recognition to a
great or very great extent for helping their agencies accomplish their
strategic goals. This was not a statistically significant improvement over
the 26 percent responding in this way 3 years ago.

In addition, in many cases, significant gaps existed between the views of
Senior Executive Service (SES) and non- SES managers in 2000, as they did in
1997. In these cases, the views of SES managers were more positive than the
views of non- SES managers. This suggests that the underlying principles
necessary to achieve a focus on results throughout the federal government,
while gaining some acceptance among higher- level managers, are not
filtering down within federal agencies.

Overall, the survey results indicate that, in some key areas, agencies may,
in fact, be losing ground in their efforts towards building organizational
cultures that support a focus on results. Our 2000 survey results underscore
the importance of the subcommittee's efforts to work with

4 In this report, the phrase “approaching statistical
significance” is defined as a difference that is significant at the
.06 probability level.

Page 4 GAO- 01- 127 Ensuring Management Skills

other Senate committees to bring greater attention to nominees' leadership
capabilities and to ensure that federal agency leaders have the appropriate
management and leadership skills needed to continue to transform the
cultures of federal agencies so that they operate as high- performing
organizations.

In 1997, we surveyed managers across the federal government to obtain
information on their experiences with results- oriented management practices
and related challenges. This survey was done as part of our response to the
requirement under GPRA that we report to Congress on the implementation of
the act, including prospects for compliance by agencies governmentwide. At
the time, implementation of GPRA governmentwide was about to begin. At your
request, using the baseline provided by our 1997 survey results, we again
surveyed managers in 2000 in order to measure agencies' progress in
improving federal management.

A critical issue in making such progress, and one which is addressed in
several ways by our surveys, is whether agency leaders have the skills and
knowledge necessary to implement the basic tenets of performance- based
management. Because of the importance of these critical management skills to
successfully tackling the challenges facing federal agencies, it is
essential that top leaders- especially political appointees- have them.
Early next year, Senate committees will be holding confirmation hearings for
many of the next administration's nominees to senior positions; our report
to you on eliciting nominees' views on leadership and management issues
provides suggested questions that the committees can use. Rather than a
template intended to be uniformly or comprehensively applied to every
nomination, these questions are intended to be a resource from which Senate
committees can select those questions that are most relevant and appropriate
for a given situation or nominee. The information from our surveys of
federal managers on performance and management issues can illustrate the
usefulness of our suggested questions in helping to ensure that political
appointees have the skills and the leadership and management experience
required to successfully run federal agencies. Background

Page 5 GAO- 01- 127 Ensuring Management Skills

The results of our 1997 survey, as well as its scope and methodology, 5 were
included in our June 1997 report on the governmentwide implementation of
GPRA. Our 2000 survey used a stratified random sample of 3,816 fulltime mid-
and upper level civilian managers and supervisors in 28 executive branch
agencies. 6 These agencies represent about 97 percent of the executive
branch full- time workforce. 7 The sample was drawn from the Office of
Personnel Management's Central Personnel Data File (CPDF) as of March 30,
1999, using file designators indicating performance of managerial and
supervisory functions. Our overall response rate was about 70 percent of the
resulting eligible sample, ranging from about 54 percent to 76 percent at
the 28 agencies.

The questionnaire was designed to obtain the observations and perceptions of
respondents on such results- oriented management topics as the presence,
use, and usefulness of performance measures; hindrances to measuring and
using performance information; agency climate; information technology;
program evaluation; and various aspects of GPRA. About half the items on the
questionnaire were asked in our 1997 survey. This earlier survey, although
done with a smaller sample size of 1,300 managers, covered the same
agencies. In contrast to our 1997 survey, the sample for the 2000 survey was
larger to allow for individual agency analyses. Similar to our earlier
survey, the sample for the 2000 survey included both SES and non- SES
managers in General Schedule (GS), General Management (GM), or equivalent
schedules, at levels comparable to GS/ GM- 13 through career SES or
equivalent levels of executive service.

The overall survey results are generalizable to the CFO Act agencies. A more
complete description of the scope and methodology for our 2000 survey is
included in appendix II. We conducted our work between January 1999 and
August 2000 in accordance with generally accepted government auditing
standards. Because this report provides summary

5 The methodology used in our 1997 survey was comparable to the methodology
used in our 2000 survey, although the 2000 methodology was augmented to
allow for several additional analyses. (See appendix II.)

6 The 28 agencies include the 24 agencies covered by the Chief Financial
Officers (CFO) Act of 1990 (see appendix I) and 4 selected subagencies
within 4 of those 24 agencies. The four subagencies are the Federal Aviation
Administration at the Department of Transportation, the Forest Service at
the Department of Agriculture, the Health Care Financing Administration at
the Department of Health and Human Services, and the Internal Revenue
Service at the Department of the Treasury. In reporting the questionnaire
data, when we use the term “governmentwide” and the phrase
“across the federal government,” we are referring to these 24
CFO Act executive branch agencies, and when we use the term “federal
managers,” we are referring to both managers and supervisors.

7 Excludes employees of the United States Postal Service, an independent
establishment of the executive branch of the U. S. government. Scope and

Methodology

Page 6 GAO- 01- 127 Ensuring Management Skills

results of our survey across the 28 agencies included in our sample and does
not discuss the survey results or implications for any individual agency, we
did not obtain agency comments on a draft of the report.

Successfully addressing the challenges that federal agencies face requires
leaders who are committed to achieving results, who recognize the importance
of using results- oriented goals and quantifiable measures, and who
integrate performance- based management into the culture and day- today
activities of their organizations. In addition, federal agencies need
leaders capable of effectively managing and developing their organizations'
human capital by providing the training, tools, structures, incentives, and
accountability to achieve results. However, agencies do not yet have the
organizational cultures that foster and maintain a focus on results. Federal
managers' views on top leadership commitment to achieving results, the use
of performance measurement in managing their programs, and the presence of
effective human capital management- all essential ingredients in managing
for results- suggest that much more remains to be done. Thus, it is
especially important for agency leaders to have the skills and experience
required to successfully lead and inspire the federal workforce to embrace
and apply the principles of results- oriented management.

In earlier testimony before this subcommittee, we have observed that top
leadership must play a critical role in creating and sustaining
highperforming organizations. Without the clear and demonstrated commitment
of agency top leadership- both political and career- organizational cultures
will not be transformed, and new visions and ways of doing business will not
take root. 8

In 2000, managers reported that their top leadership still did not show a
consistently strong commitment to achieving results. About 57 percent of
managers overall reported such commitment to a great or very great extent in
1997, while about 53 percent reported such commitment to a great or very
great extent in 2000- not a statistically significant change. As shown in
figure 1, however, we continued to see a significant difference between the
perceptions of SES and non- SES managers on this issue. That is, the
percentage of SES managers reporting that top leadership demonstrated strong
commitment to a great or very great extent in 2000 was 20 percent higher
than for non- SES managers- the same significantly higher difference we saw
in 1997.

8 Management Reform: Using the Results Act and Quality Management to Improve
Federal Performance (GAO/ T- GGD- 99- 151, July 29, 1999); and Management
Reform: Elements of Successful Improvement Initiatives (GAO/ T- GGD- 00- 26,
Oct. 15, 1999). Federal Managers'

Views Show the Need to Ensure That Agency Leaders Demonstrate Results-
Oriented Management Practices

Top Leadership Does Not Consistently Show Commitment to Achieving Results

Page 7 GAO- 01- 127 Ensuring Management Skills

Source: GAO survey data.

We observed in our 1997 report that we would expect to see managers'
positive perceptions on items such as the extent to which top leadership is
committed to achieving results become more prevalent and the gap between SES
and non- SES managers begin to narrow as GPRA and related reforms are
implemented. However, these changes do not appear to be happening.

Results- oriented decisionmaking is one of four key areas that our suggested
confirmation questions address. Demonstrating the willingness and ability to
make decisions and manage programs based on results and the ability to
inspire others to embrace such a model are important indicators of
leadership commitment to results- oriented management. However, in both our
1997 and 2000 surveys, only about 16 percent of managers reported that
changes by management above their levels to the programs for which they were
responsible were based on results or outcome- oriented performance
information to a great or very great extent.

Figure 1: Percentage of Federal Managers Who Reported That Agency Top
Leadership Demonstrated to a Great or Very Great Extent a Strong Commitment
to Achieving Results

Page 8 GAO- 01- 127 Ensuring Management Skills

Our suggested questions for the confirmation of political appointees include
several questions related to candidates' results- oriented leadership skills
and experience, and emphasize the importance of agency leaders' commitment
to achieving goals and their ability to inspire their staffs to accomplish
those goals if they are to successfully lead their organizations into the
future. For example, we include questions regarding both tangible instances
where nominees' personal leadership skills were essential in getting
employees to accomplish challenging goals and the need for senior leaders to
be drivers for continuous improvement by motivating agency employees to
achieve excellence.

A fundamental element in an organization's efforts to manage for results is
its ability to set meaningful goals for performance and to measure
performance against those goals. In 2000, 84 percent of federal managers
overall said there were performance measures for the programs they were
involved with. This is a statistically significant increase over the 76
percent of managers who responded that way in 1997. Moreover, when we asked
managers who said they had performance measures which of five types of
measures 9 they had to a great or very great extent, they reported increases
in all five types of measures between 1997 and 2000, including statistically
significant increases in three of them- output measures, efficiency
measures, and outcome measures. (See fig. 2.)

9 Types of measures were defined in the questionnaire as follows:
Performance measures that tell us how many things we produce or services we
provide (output measures); performance measures that tell us if we are
operating efficiently (efficiency measures); performance measures that tell
us whether or not we are satisfying our customers (customer service
measures); performance measures that tell us about the quality of the
products or services we provide (quality measures); performance measures
that would demonstrate to someone outside of our agency whether or not we
are achieving our intended results (outcome measures). More Managers
Reported

Having Performance Measures, but Fewer Reported Using Them Extensively

Page 9 GAO- 01- 127 Ensuring Management Skills

Source: GAO survey data.

These responses indicate some progress by suggesting that measurement of
federal program performance is occurring to a greater extent currently than
in 1997. However, for none of these types of performance measures does the
percentage of managers responding that they have them to a great or very
great extent in 2000 exceed 50 percent. Additionally, for two of these types
of measures- quality and customer service measures- there have not been
significant changes between 1997 and 2000. In our suggested questions for
eliciting nominees' views on leadership and management issues, we included
questions related to the importance of agencies' customer focus and the need
to understand customer expectations and to establish performance goals and
measures to meet those expectations.

Figure 2: Percentage of Federal Managers Who Reported Having Specific Types
of Performance Measures to a Great or Very Great Extent

Page 10 GAO- 01- 127 Ensuring Management Skills

Because the benefit of collecting performance information is only fully
realized when this information is actually used by managers, we asked them
about the extent to which they used the information obtained from measuring
performance for various program management activities. Managers reported
that use of performance information to a great or very great extent in 2000
varied, depending on the activity, ranging from 38 percent for developing
and managing contracts to 56 percent for setting program priorities. For
five of the eight activities that we asked about in both our 1997 and 2000
surveys, the reported use to a great or very great extent was significantly
lower in 2000 than it was in 1997. For example, in 2000, 51 percent of
managers reported that they used information obtained from performance
measurement to a great or very great extent when adopting new program
approaches or changing work processes, as compared to 66 percent of managers
in 1997. For none of the eight activities was the reported extent of use in
2000 higher than it was in 1997. (See table 1.)

Management activity 1997 survey a 2000 survey a Difference

Setting program priorities 66% 56% -10% b Allocating resources 62 53 -9 b
Adopting new program approaches or changing work processes

66 51 -15 b Coordinating program efforts with other internal or external
organizations

57 43 -14 b Refining program performance measures

52 44 -8 Setting new or revising existing performance goals 58 51 -7 Setting
individual job expectations for my staff

61 51 -10 b Rewarding staff I manage or supervise

53 53 None Developing and managing contracts

N/ A c 38 N/ A c a Percentages based on those respondents answering on the
extent scale. b Statistically significant difference. c Not available;
question not asked in 1997.

Source: GAO survey data.

Table 1: Percentage of Federal Managers Who Reported Using Information
Obtained From Performance Measurement to a Great or Very Great Extent for
Various Management Activities

Page 11 GAO- 01- 127 Ensuring Management Skills

The significant reduction in the reported use of performance information in
making most of these critical management decisions is disturbing. It
suggests that efforts to increase the focus on results and the use of
performance information are not penetrating the federal bureaucracy. In our
proposed questions for the confirmation of political appointees, we include
questions on candidates' experiences related to both measuring performance
and supporting employees in the use of performance information to
systematically improve program performance, service quality, and customer
satisfaction. Such support by agency leaders is essential if agency managers
are to use performance information in managing their programs.

Managing people strategically and maintaining a highly skilled and energized
workforce that is empowered to focus on results are critically important.
Such human capital management practices are essential to the success of the
federal government in the 21st century and to maximizing the value of its
greatest asset- its people.

High- performing organizations seek to shift the focus of management and
accountability from activities and processes to contributions to achieving
results. In our proposed questions for political appointees, we discuss the
need to create an organizational culture that involves and empowers
employees to improve operational and program performance while ensuring
accountability and fairness for all employees. In both our 1997 and 2000
surveys, we asked managers about the amount of decisionmaking authority they
had and the degree to which they were held accountable for results.

There were significant differences in managers' responses on decisionmaking
authority among GS- 13 through SES managers in this year's survey. In 2000,
36 percent of federal managers overall reported that they had the
decisionmaking authority they needed to help the agency accomplish its
strategic goals to a great or very great extent, as compared to 31 percent
in 1997- not a statistically significant change. However, there were
substantial differences in 2000 between SES and lower- level managers.
Compared to the 56 percent of SES managers who reported having such
authority to a great or very great extent, 31 percent of GS- 13s and 32
percent of GS- 14s reported having that level of authority in 2000.
Moreover, while more than half of SES managers reported having such
authority to a great or very great extent, less than half of managers at
levels below SES expressed the same perception. (See fig. 3.) Managers
Continue to

Confront Human Capital Management Challenges

Federal Managers Report That They Are Held Accountable for Program Results
but Do Not Have the Decisionmaking Authority They Need to Accomplish Agency
Goals

Page 12 GAO- 01- 127 Ensuring Management Skills

Statistically significant differences between GS- 13/ 14s compared to GS-
15s or SES and between GS15s and SES.

Source: GAO survey data.

When asked about the extent to which managers or supervisors at their levels
were held accountable for the results of the programs for which they were
responsible, about 63 percent of managers reported having such
accountability to a great or very great extent in 2000 compared to about 55
percent who reported this perception in 1997. This increase approaches
statistical significance.

Unlike in other areas, where SES managers had significantly different views
from non- SES managers, there was little difference in the area of
accountability. Even when broken down by federal pay grade, none of the
differences in managers' perceptions about accountability were statistically
significant in 2000. For example, 65 percent of GS- 13s reported that
managers at their level were held accountable for the results of their
programs to a great or very great extent, while 66 percent of SES managers
reported accountability at that level. (See fig. 4.)

Figure 3: Percentage of Federal Managers in 2000 Reporting to a Great or
Very Great Extent That Managers/ Supervisors at Their Levels Had the
Decisionmaking Authority They Needed to Help the Agency Acccomplish Its
Strategic Goals

Page 13 GAO- 01- 127 Ensuring Management Skills

Source: GAO survey data.

These results show that while only 31 and 32 percent of managers at the GS-
13 and GS- 14 levels, respectively, perceived that managers at their levels
had the decisionmaking authority they needed to a great or very great
extent, more than 60 percent of managers at these grade levels perceived
that they were held accountable to a great or very great extent. In our
report containing the results of our 1997 survey, we noted that agencies
needed to concentrate their efforts on areas where managers were not
perceiving or experiencing progress, such as that concerning devolving
decisionmaking authority to managers throughout their organizations. Our
survey results do not indicate a positive change among federal agencies in
this regard, and the balance between authority and accountability that
fosters decisionmaking to achieve results seems still to be lacking. Simply
stated, managers are hard- pressed to achieve results when they do not have
sufficient authority to act.

In our report on eliciting political nominees' views on leadership and
management issues, we stressed the need for agencies to expend resources on
effective training and professional development to equip federal employees
to work effectively. However, no more than 45 percent of managers in 2000
answered “yes” when we asked them whether, during the past 3
years, their agencies had provided, arranged, or paid for training that
would help them accomplish any of seven critical results- oriented

Figure 4: Percentage of Federal Managers in 2000 Reporting That Managers/
Supervisors at Their Levels Were Held Accountable for Results to a Great or
Very Great Extent

Fewer Than Half of Managers Reported Training on Key Tasks

Page 14 GAO- 01- 127 Ensuring Management Skills

management- related tasks. Managers reported more training in 2000 than in
1997 for all six tasks that we asked about in both years. However, these
increases were statistically significant for only two of the tasks- setting
program performance goals and implementing the requirements of GPRA. (See
fig. 5.)

Source: GAO survey data.

The 2000 survey results suggest that there is a positive relationship
between agencies providing for training on setting program performance goals
and the use of performance information when setting or revising performance
goals. For those managers who responded “yes” to training on
setting performance goals, 57 percent also reported that they used
information obtained from performance measurement when setting new or
revising existing performance goals to a great or very great extent. In
contrast, for those managers who responded “no” to training on
setting

Figure 5: Percentage of Federal Managers Who Reported That During the Past 3
Years Their Agencies Provided, Arranged, or Paid for Training That Would
Help Them Accomplish Specific Tasks

Page 15 GAO- 01- 127 Ensuring Management Skills

performance goals, only 46 percent reported that they used information
obtained from performance measurement for setting new or revising existing
performance goals to a great or very great extent. The difference between
these percentages is statistically significant.

Another fundamental aspect of the human capital management challenge
agencies face is providing the incentives to their employees to encourage
results- oriented management. Pay increases and other monetary and
nonmonetary incentives can be used as a method for federal agencies to
reward employees and to motivate them to focus on results. Our questions for
political nominees solicit their views on motivating employees to achieve
excellence and ask whether the nominees think that pay decisions should be
more closely tied to the achievement of agencies' strategic and annual
performance goals.

Overall, few managers reported in either 1997 or 2000 that employees in
their agencies received positive recognition to a great or very great extent
for helping agencies accomplish their strategic goals. In 1997, 26 percent
of federal managers reported such an extent of positive recognition as
compared to 31 percent in 2000, which was not a statistically significant
change. When we compared the responses of SES and non- SES managers, the
statistically significant difference of 23 percent between these two groups
that existed in 1997 remained unchanged in 2000. However, more than half of
SES managers reported such an extent of positive recognition in 2000. (See
fig. 6.) Managers Perceive a Lack of

Positive Recognition for Helping Agencies Achieve Results

Page 16 GAO- 01- 127 Ensuring Management Skills

Source: GAO survey data.

In important ways, the data from our 1997 and 2000 surveys of federal
managers paint a discouraging picture. In key areas, such as top leadership
commitment to results, we have not seen significant increases since the last
time we surveyed federal managers. In other areas, such as managers'
responses on how they use performance information, there appears to be some
backsliding from the results we reported on in 1997. In our earlier work,
the importance of the statutory framework that Congress has put in place
over the last decade to foster high- performing federal organizations has
often been highlighted. Enacting that framework, however, was only the first
necessary step. Real improvements in performance will come only when federal
managers and other decisionmakers use the framework and results- oriented
performance data to make decisions. Federal managers are clearly saying that
much work remains before this is consistently the case across the federal
government.

In the coming months, we will continue to analyze the survey data both on a
governmentwide and agency- specific basis to seek insights into what must be
done to make the cultural transformations that Congress was

Figure 6: Percentage of Federal Managers Who Reported to a Great or Very
Great Extent That Employees in Their Agencies Received Positive Recognition
for Helping Their Agencies Accomplish Their Strategic Goals

Conclusions

Page 17 GAO- 01- 127 Ensuring Management Skills

looking for when it enacted GPRA. However, one vital next step is already
clear: top leadership within agencies must have a clear and demonstrated
commitment to results that is not merely stated verbally but is consistently
reinforced by specific actions. This subcommittee's initiative to work with
Senate committees to inquire about management and leadership capabilities as
part of the confirmation process will send an unmistakable message to the
political leadership within agencies that Congress is serious about results
and the cultural transformations that are needed to bring about fundamental
improvements in performance.

As agreed with your office, unless you announce the contents of this report
earlier, we plan no further distribution until 30 days after its issue date.
At that time, we will send copies of the report to Senator Richard J.
Durbin, Ranking Minority Member, Subcommittee on Oversight of Government
Management, Restructuring, and the District of Columbia, and to Senator Fred
Thompson, Chairman, and Senator Joseph Lieberman, Ranking Minority Member,
Senate Committee on Governmental Affairs. We will also send copies to Jacob
J. Lew, Director, Office of Management and Budget, and Janice LaChance,
Director, Office of Personnel Management. In addition, we will make copies
available to others upon request.

The major contributors to this report are listed in appendix III. If you
have any questions concerning this report, please contact me or Joyce Corry
on (202) 512- 8676.

Sincerely yours, J. Christopher Mihm Director, Strategic Issues

Page 18 GAO- 01- 127 Ensuring Management Skills

Contents 1 Letter 20 Appendix I Agencies Covered by the Chief Financial
Officers Act

21 Appendix II Scope and Methodology

24 Appendix III GAO Contacts and Staff Acknowledgments

26 Related GAO Products Table 1: Percentage of Federal Managers Who Reported

Using Information Obtained From Performance Measurement to a Great or Very
Great Extent for Various Management Activities

10 Tables Figure 1: Percentage of Federal Managers Who Reported

That Agency Top Leadership Demonstrated to a Great or Very Great Extent a
Strong Commitment to Achieving Results

7 Figure 2: Percentage of Federal Managers Who Reported

Having Specific Types of Performance Measures to a Great or Very Great
Extent

9 Figures

Figure 3: Percentage of Federal Managers in 2000 Reporting to a Great or
Very Great Extent That Managers/ Supervisors at Their Levels Had the
Decisionmaking Authority They Needed to Help the Agency Acccomplish Its
Strategic Goals

12

Contents Page 19 GAO- 01- 127 Ensuring Management Skills

Figure 4: Percentage of Federal Managers in 2000 Reporting That Managers/
Supervisors at Their Levels Were Held Accountable for Results to a Great or
Very Great Extent

13 Figure 5: Percentage of Federal Managers Who Reported

That During the Past 3 Years Their Agencies Provided, Arranged, or Paid for
Training That Would Help Them Accomplish Specific Tasks

14 Figure 6: Percentage of Federal Managers Who Reported

to a Great or Very Great Extent That Employees in Their Agencies Received
Positive Recognition for Helping Their Agencies Accomplish Their Strategic
Goals

16

Abbreviations

CFO chief financial officer CPDF Civilian Personnel Data File GM general
management GPRA Government Performance and Results Act of 1993 GS General
Schedule SES Senior Executive Service

Appendix I Agencies Covered by the Chief Financial Officers Act

Page 20 GAO- 01- 127 Ensuring Management Skills

Department of Agriculture Department of Commerce Department of Defense
Department of Education Department of Energy Department of Health and Human
Services Department of Housing and Urban Development Department of the
Interior Department of Justice Department of Labor Department of State
Department of Transportation Department of the Treasury Department of
Veterans Affairs Environmental Protection Agency Federal Emergency
Management Agency General Services Administration National Aeronautics and
Space Administration National Science Foundation Nuclear Regulatory
Commission Office of Personnel Management Small Business Administration
Social Security Administration U. S. Agency for International Development

Appendix II Scope and Methodology

Page 21 GAO- 01- 127 Ensuring Management Skills

A questionnaire on performance and management issues was sent to a
stratified random sample of 3, 816 full- time mid- level and upper level
civilian managers and supervisors working in the 24 executive branch
agencies covered by the Chief Financial Officers (CFO) Act of 1990. These
agencies represent about 97 percent of the executive branch full- time
workforce, excluding the Postal Service. In reporting the questionnaire
data, when we use the term governmentwide, we are referring to these 24 CFO
Act agencies, and when we use the term federal managers, we are referring to
both managers and supervisors. The sample was drawn from the Office of
Personnel Management's Central Personnel Data File (CPDF) as of March 30,
1999, using file designators indicating performance of managerial and
supervisory functions.

The questionnaire was designed to obtain the observations and perceptions of
respondents on such results- oriented management topics as the presence,
use, and usefulness of performance measures; hindrances to measuring and
using performance information; agency climate; information technology;
program evaluation; and various aspects of the Government Performance and
Results Act of 1993 (GPRA). Most of the items on the questionnaire were
closed- ended, meaning that, depending on the particular item, respondents
could choose one or more response categories or rate the strength of their
perception on a 5- point extent scale.

About half the items on the questionnaire were contained in an earlier
survey conducted between November 1996 and January 1997 as part of the work
we did in response to a GPRA requirement that we report on implementation of
the act. 1 This earlier survey, although done with a smaller sample size of
1,300 managers, covered the same agencies as the 2000 survey. The current
survey was sent out between January and August 2000. Individuals who did not
respond to the initial questionnaire were sent up to two follow- up
questionnaires. In some cases, we contacted individuals by telephone and
faxed the questionnaire to them to expedite completion of the survey.

The current survey was designed to update and further elaborate on the
results of the earlier survey. Similar to the earlier survey, the sample was
stratified by whether the manager or supervisor was Senior Executive Service
(SES) or non- SES. The management levels covered General Schedule (GS),
General Management (GM), or equivalent schedules at levels comparable to GS/
GM- 13 through career SES or equivalent levels of

1 The Government Performance and Results Act: 1997 Governmentwide
Implementation Will Be Uneven (GAO/ GGD- 97- 109, June 2, 1997).

Appendix II Scope and Methodology

Page 22 GAO- 01- 127 Ensuring Management Skills

executive service. Stratification was also done by the 24 CFO Act agencies,
with an additional breakout of four selected agencies from their
departments- Forest Service, Health Care Financing Administration, Federal
Aviation Administration, and Internal Revenue Service. These four agencies
were selected on the basis of previous GAO work identifying them as facing
significant managerial challenges.

The sample was also stratified to include special pay plans at some agencies
to improve our coverage of managers and supervisors working at those
agencies. For example, Senior Foreign Service executives from the State
Department and the Agency for International Development were included in the
sample. We included these special pay plan strata to ensure at least a 90-
percent coverage of all managers and supervisors at or comparable to the GS/
GM- 13 through career SES level at the 28 departments and agencies we
surveyed. Finally, we added additional strata in order to include a group of
respondents who answered the earlier survey and who still worked in the same
agency at the same management level at the time of the 2000 survey.

During the course of the survey, we deleted 212 persons from our sample who
had either retired, separated, died, or otherwise left the agency or had
some other reason that excluded them from the population of interest. We
received useable questionnaires from 2,510 sample respondents, or about 70
percent of the remaining eligible sample. The response rate across the 28
agencies ranged from about 54 percent to 76 percent.

We took several steps to check the quality of our survey data. We reviewed
and edited the completed questionnaires, made internal consistency checks on
selected items, and checked the accuracy of data entry on a sample of
surveys. We also followed up on a sample of nonrespondents to assess whether
their views differed from the views of those who returned the survey. We
randomly selected a subsample of 136 persons across all strata from that
group of individuals who had not returned a completed questionnaire a month
or more after the last of three attempts were made to elicit their
participation in our survey. We received 67 useable surveys from this group.
In addition, there were 41 individuals who, when contacted by telephone,
refused to participate in the survey but were willing to answer three key
questions from the survey. We included their answers to the three questions
in our analysis of nonrespondents on those three questions. We analyzed the
responses of these groups on selected items compared to the responses
received from all other respondents. Our analyses of selected items did not
show a sufficient or consistent degree of

Appendix II Scope and Methodology

Page 23 GAO- 01- 127 Ensuring Management Skills

difference between survey nonrespondents and respondents, and thus we
included the responses of our subsample with all other responses.

The overall survey results are generalizable to the CFO Act agencies. All
results are subject to some uncertainty or sampling error as well as
nonsampling error. In general, percentage estimates in this report for the
entire sample have confidence intervals ranging from about ï¿½2 to ï¿½7
percentage points at the 95 percent confidence interval. In other words, if
all CFO Act agency managers and supervisors in our population had been
surveyed, the chances are 95 out of 100 that the result obtained would not
differ from our sample estimate in the more extreme cases by more than ï¿½7
percent.

Because a complex sample design was used and different types of statistical
analyses are being done, the magnitude of sampling error will vary across
the particular groups or items being compared due to differences in the
underlying sample sizes and associated variances. Consequently, in some
instances, a difference of a certain magnitude may be statistically
significant. In other instances, depending on the nature of the comparison
being made, a difference of equal or even greater magnitude may not achieve
statistical significance. We note throughout the report when differences
between groups or items are significant at the .05 probability level.
Differences that are significant at the .06 probability level are also noted
and are characterized as approaching statistical significance.

Appendix III GAO Contacts and Staff Acknowledgments

Page 24 GAO- 01- 127 Ensuring Management Skills

J. Christopher Mihm or Joyce Corry, (202) 512- 8676 In addition to the
persons named above, Peter J. Del Toro, Thomas Beall, Catherine Hurley, Sid
Schwartz, and Stu Kaufman made key contributions to this report. GAO
Contacts

Acknowledgments

Page 25 GAO- 01- 127 Ensuring Management Skills

Related GAO Products

Page 26 GAO- 01- 127 Ensuring Management Skills

Human Capital: A Self- Assessment Checklist for Agency Leaders, Version 1
(GAO/ OCG- 00- 14G, September 2000).

Confirmation of Political Appointees: Eliciting Nominees' Views on
Leadership and Management Issues (GAO/ GGD- 00- 174, Aug. 11, 2000).

Management Reform: Elements of Successful Improvement Initiatives (GAO/ T-
GGD- 00- 26, Oct. 15, 1999).

Management Reform: Using the Results Act and Quality Management to Improve
Federal Performance (GAO/ T- GGD- 99- 151, July 29, 1999).

The Government Performance and Results Act: 1997 Governmentwide
Implementation Will Be Uneven (GAO/ GGD- 97- 109, June 2, 1997).

Executive Guide: Effectively Implementing the Government Performance and
Results Act (GAO/ GGD- 96- 118, June 1996).

Page 27 GAO- 01- 127 Ensuring Management Skills

Ordering Copies of GAO Reports The first copy of each GAO report and
testimony is free. Additional copies are $2 each. Orders should be sent to
the following address, accompanied by a check or money order made out to the
Superintendent of Documents, when necessary. VISA and MasterCard credit
cards are accepted, also. Orders for 100 or more copies to be mailed to a
single address are discounted 25 percent.

Order by mail: U. S. General Accounting Office P. O. Box 37050 Washington,
DC 20013

or visit: Room 1100 700 4 th St. NW (corner of 4 th and G Sts. NW) U. S.
General Accounting Office Washington, DC

Orders may also be placed by calling (202) 512- 6000 or by using fax number
(202) 512- 6061, or TDD (202) 512- 2537.

Each day, GAO issues a list of newly available reports and testimony. To
receive facsimile copies of the daily list or any list from the past 30
days, please call (202) 512- 6000 using a touchtone phone. A recorded menu
will provide information on how to obtain these lists.

Viewing GAO Reports on the Internet For information on how to access GAO
reports on the INTERNET, send e- mail message with “info” in the
body to:

info@ www. gao. gov or visit GAO's World Wide Web Home Page at: http:// www.
gao. gov Reporting Fraud, Waste, and Abuse in Federal Programs To contact
GAO FraudNET use: Web site: http:// www. gao. gov/ fraudnet/ fraudnet. htm
E- Mail: fraudnet@ gao. gov Telephone: 1- 800- 424- 5454 (automated
answering system)

United States General Accounting Office Washington, D. C. 20548- 0001

Official Business Penalty for Private Use $300

Address Correction Requested Bulk Rate

Postage & Fees Paid GAO Permit No. G100

(410402)
*** End of document. ***