Information Technology Training: Practices of Leading		 
Private-Sector Companies (31-JAN-03, GAO-03-390).		 
                                                                 
The rapid pace of technological change, with its potential to	 
transform the way the government delivers services, makes	 
information technology (IT) human capital a critical issue for	 
federal agencies.GAO has identified strategic human capital	 
management as a high risk area for the federal government, and	 
the demand for skilled IT workers is expected to increase over	 
the long term. Given that competition for workers affects the	 
federal government as it does any other employer, effective	 
training of staff is essential to developing and retaining a	 
qualified workforce. Some private-sector companies are recognized
for their effective and innovative training programs form the IT 
workforce, which could provide models and examples for federal	 
agencies. To help federal agencies better design and implement	 
such training programs, GAO was asked to examine private-sector  
practices for training both IT and non-IT professionals (e.g.,	 
business managers and other staff needing training in IT) that	 
could be used as a basis for addressing federal efforts.	 
-------------------------Indexing Terms------------------------- 
REPORTNUM:   GAO-03-390 					        
    ACCNO:   A06014						        
  TITLE:     Information Technology Training: Practices of Leading    
Private-Sector Companies					 
     DATE:   01/31/2003 
  SUBJECT:   Employment or training programs			 
	     Human resources utilization			 
	     Information technology				 
	     Personnel management				 

******************************************************************
** This file contains an ASCII representation of the text of a  **
** GAO Product.                                                 **
**                                                              **
** No attempt has been made to display graphic images, although **
** figure captions are reproduced.  Tables are included, but    **
** may not resemble those in the printed version.               **
**                                                              **
** Please see the PDF (Portable Document Format) file, when     **
** available, for a complete electronic file of the printed     **
** document's contents.                                         **
**                                                              **
******************************************************************
GAO-03-390

                                       A

Letter

January 31, 2003 The Honorable Tom Davis Chairman Committee on Government
Reform House of Representatives

The Honorable Jim Turner House of Representatives

The rapid pace of technological change, with its potential to transform
the way the government delivers services, makes information technology
(IT) human capital a critical issue for federal agencies. We have
identified strategic human capital management as a high risk area. In the
long term, demand for highly skilled IT workers is expected to increase.
According to a 2002 study of private- sector employers by the Information
Technology Association of America (ITAA), the demand for these highly
skilled IT workers exceeds supply. 1 Given that this reported shortage
affects the federal government as it does any other employer, effective
training of staff is essential to developing and retaining a qualified
workforce. Our objective was to examine private- sector workforce training
practices

for both IT and non- IT professionals (e. g., business managers and other
staff needing training in IT). To achieve this objective, we reviewed
existing research, held discussions with academic and professional
authorities, and

interviewed executives and managers at leading companies about their IT
training management practices and activities. We also collaborated with
the National Academy of Science to host a panel discussion with academic

authorities. The experts on this panel discussed overall training issues,
provided input on training management processes, and identified other
effective training practices. We used this information, as well as the
extensive research and trade literature available on IT training
practices, to develop a view of what leading private- sector organizations
are doing in the IT training area.

On October 18, 2002, we provided briefing slides on the results of our
study to you in your capacities as Chairman and Ranking Minority Member of
the Subcommittee on Technology and Procurement Policy. These results,
along

1 Information Technology Association of America, Bouncing Back: Jobs,
Skills, and the Continuing Demand for IT Workers (May 2002).

with additional information and new and expanded case studies, are
included as an appendix to this letter. The purpose of this letter is to
officially transmit the information in published form to you as Chairman
and Member of the Committee on Government Reform.

In brief, we found 22 existing and emerging practices that are used by
leading companies to implement effective IT training. The majority of the
companies performed 10 or more of the identified practices, but none was
performing all. The practices and case studies provided in the appendix
suggest approaches to IT training that government agencies could consider.

In addition, we noted several critical issues (e. g., funding constraints
and demonstrating return on investment) that should be considered in
implementing these practices. GAO*s review of private- sector practices
indicates that training is not simply a support function, but a strategic
element in achieving corporate objectives. Further, although companies are
adopting new ideas about training, many initiatives are in their early
stages, and private- sector officials expressed interest in learning about
innovative practices emerging from the public sector.

Many organizations contributed to our study. American Telephone &
Telegraph (AT& T), Cable & Wireless, Cisco, Delta Technology, Fannie Mae,
FleetBoston, International Business Machines (IBM), International Truck

and Engine, Raytheon, Science Applications International Corporation
(SAIC), and United Services Automobile Association (USAA) all met with us
to discuss their training programs. The federal Chief Information Officers
(CIO) Council, the Information Technology Association of America (ITAA),
the Private Sector Council, and the American Society for Training &
Development (ASTD), as well as IT consulting firms Gartner and Giga, also
provided assistance and information.

Unless you publicly announce the contents of this report earlier, we plan
no further distribution until 30 days from the report date. At that time,
we will send copies of this report to interested congressional committees.
In addition, copies will be made available to others upon request. Copies
of this report are also available at no charge on GAO*s Web site at www.
gao. gov.

If you or your offices should have any questions concerning this report,
please contact me at (202) 512- 6253 or Megen Davis, Assistant Director,
at (202) 512- 6398. We can also be reached by E- mail at willemssenj@ gao.
gov and davism@ gao. gov, respectively. Key contributors to this report
were

Barbara Collier, Vijay D*Souza, John Ortiz, Tomas Ramirez, Jr., and Glenn
Spiegel.

Joel Willemssen Managing Director, Information Technology

Appendi Appendi xes x I

Information Technology Training Information Technology Training: Practices
of Leading Private- Sector Companies

Committee on Government Reform House of Representatives

1

Contents Introduction and Objective Scope and Methodology Background
Private- Sector Practices

 Aligning IT training with business goals  Identifying and assessing IT
training needs  Allocating IT training resources  Designing and
delivering IT training  Evaluating/ demonstrating the value of IT
training Critical Issues Final Note

2

Introduction and Objective GAO has identified strategic human capital
management as a high risk area for the federal government. According to a
study of private- sector employers by the Information Technology
Association of America (ITAA), the demand for skilled IT workers exceeds
supply, and the shortage is expected to continue. 1 Given that the
shortage affects the federal government as it does any other employer,
effective training of existing staff is essential to developing and
retaining a qualified workforce.

Some private- sector companies are recognized for their effective and
innovative training programs, which could provide models and examples for
federal agencies. To help federal agencies better design and implement IT
workforce training programs, you asked us to examine private- sector
training practices both for IT professionals and for other staff needing
IT training (e. g., business managers) that could be used as a basis for
addressing federal efforts.

1 Information Technology Association of America Bouncing Back: Jobs,
Skills and the Continuing Demand for IT Workers (May 2002).

3

Scope and Methodology We selected companies considered leaders in IT
training by professional organizations, publications, and academic
experts. We based our selection on awards, significant accolades, and
expert recommendations. Specifically:

 We searched published sources and the Internet for companies recognized
as leaders by trade associations and industry publications. Examples
include

Computerworld and Training magazine rankings.  We solicited
recommendations from industry and academic experts.  To identify and
establish contacts with candidate companies, we consulted

with  the federal Chief Information Officers (CIO) Council; 
professional associations (the Information Technology Association of

America, the Private Sector Council, and the American Society for Training
& Development); and

 IT consulting firms (Gartner and Giga). 4

Scope and Methodology On the basis of our discussions and analyses, we
conducted site visits at 11 companies identified as leaders in IT
training:

 American Telephone & Telegraph  International Business Machines

(AT& T)* Telecom (IBM)* Technology

 Cable & Wireless* Telecom  International Truck and Engine*

 Cisco* Technology Manufacturing

 Delta Technology* Airline  Raytheon* Defense & Electronics

 Fannie Mae* Finance  Science Applications International

Corporation (SAIC)* Research &  FleetBoston* Finance

Engineering  United Services Automobile

Association (USAA)* Finance & Insurance

5

Scope and Methodology At the companies visited, we reviewed training
program documentation and interviewed executives, business unit managers,
training managers, and training recipients about their programs.

Specifically, we provided a standard set of questions to each company to
obtain information on the organizational placement, structure, and
management of training. We also obtained additional documentation on
specific training initiatives.

We analyzed the data to identify existing and emerging training practices.
The majority of the companies performed 10 or more of the identified
practices, although none was performing all. However, organizations and
experts agreed that all the practices could result in more effective
training management. The companies also reviewed and commented on a draft
list of the practices. These practices may suggest improvements in public
sector training programs.

We also analyzed the data to develop specific case studies that
exemplified one or more of the practices. At the conclusion of our
research, all companies reviewed and verified the accuracy of our
descriptions of their training initiatives. We did not, however, verify
the accuracy of claims made by the companies.

6

Scope and Methodology To organize our work, we grouped the practices and
case studies into five categories (training management processes) that we
identified based on input from industry experts, published research, and
our previous work. The categories are (1) aligning IT training with
business goals, (2) identifying and assessing IT training needs, (3)
allocating IT training resources, (4) designing and delivering IT
training, and (5) evaluating/ demonstrating the value of IT training.

To validate our methodology, we consulted internal and external experts on
workforce development and training issues.

Also, as part of the consultative process, we collaborated with the
National Academies of Science to host a 1- day panel discussion with
academic authorities. The panel experts discussed overall training issues,
validated our management processes, and identified effective training
practices. The panel*s insights provided a broader perspective than
companies alone could provide.

7

Scope and Methodology Further, we facilitated on- line discussions of
preliminary results with academic, private- sector, and cognizant GAO
staff to solicit comments and feedback on key issues.

We conducted our review from November 2001 to November 2002, in accordance
with generally accepted government auditing standards.

8

Background  Training is a vital part of the human capital equation. An
organization*s

approaches to human capital development and training should be aligned to
support its mission, vision, goals and objectives, and strategies.

 According to a study by the National Association of Public
Administration, 2

there is a lack of investment in continuous learning within the federal
government. The study further stated that this lack of investment is
especially problematic in the dynamic and rapidly changing world of IT.

 As companies are forced to develop and implement effective IT training
programs with fewer resources, they are exploring innovative approaches to
training using both existing and emerging practices.

 The following practices, grouped into management processes, may help
federal agencies improve their IT training programs.

2 National Academy of Public Administration, The Transforming Power of
Information Technology

(August 2001). 9

Private- Sector Practices

Align IT training with

Enlist executive- level champions (sponsorship) to ensure that training
strategies are incorporated into

business goals

corporate decisionmaking and aligned with business goals

Involve critical stakeholders, such as top management, business unit
managers, subject matter experts, human capital staff, and end users, in
planning IT training

Address future skill needs and new technologies as part of the planning
process

Identify and assess IT

Identify and document competencies/ skills required for each job
description

training needs

Maintain a current inventory of skills

Address overall career development issues as well as skill- specific
training issues

Perform a gap analysis to determine where training is needed

Use self- directed tools, such as individual development plans, to give
employees responsibility in assessing their development needs

Use a single portal to give staff and managers access to training and
career development information

Allocate IT training

Ensure that an investment process is in place to select and manage
training projects

resources

Consider the benefits and costs associated with various training design
and delivery methods* e. g., Internet- based as opposed to classroom
training

Identify people who have high potential and provide them specialized
training opportunities

Ensure that resources are allocated for management training* e. g.,
leadership and project management

Design and deliver IT

Provide IT trainees with the flexibility to choose among different IT
training delivery methods

training

Ensure that on- the- job training is planned and monitored as part of the
training process

Consider combining different teaching methods (for example, Web- based and
instructor- led) within the same course

Provide just- in- time training

Consider outsourcing training solutions* e. g., university partnerships
and external IT training and content providers

Build courses using reusable components

Evaluate/ demonstrate

Collect information on how job performance is affected by training

the value of IT

Validate IT content learning by testing and certification of specific
skills* e. g., Java or C ++

training Assess evaluation results in terms of business impact 10

Aligning IT Training with Business Goals

By linking IT training programs to their overall business strategy/ goals,
companies can promote staff development that can best achieve corporate
objectives.

While the majority of the companies we visited are performing one or more
of the key practices associated with aligning IT training to the overall
goals of the company, only two have developed comprehensive planning
processes to achieve this alignment.

11

Aligning IT Training with Goals

Practices

 Enlist executive- level champions (sponsorship) to ensure that training
strategies are incorporated into corporate decisionmaking and aligned with
business goals

 Involve critical stakeholders, such as top management, business unit
managers, subject matter experts, human capital staff, and end users, in
planning IT training

 Address future skill needs and new technologies as part of the planning
process

12

Aligning IT Training with Goals Case Study 1

A process was established to link learning strategy to overall business
direction.

Practice Illustrated

 Involve critical stakeholders

Background

Between 2001 and 2002, this large telecom company reorganized its training
program. The company believed it could no longer meet *new world
challenges using the old world model.* It did not have a process linking a
well- planned and defined training program to corporate business
strategies.

Challenge

To create a process that links training to an overall business strategy to
achieve business objectives.

13

Aligning IT Training with Goals Case Study 1

Solution

According to a senior training official, the company established the
*Governance Process Flow* model, which is intended to involve all levels
of the company in developing a training strategy linked to the business
strategy. To ensure that all stakeholders are included, corporate
strategic information is communicated throughout the company. The
stakeholders include the following:

 Learning Board (senior executives and one training organization
representative)  Learning Council (executives and two training
organization representatives)  Learning Services (training organization)
 Business Lines (executives, managers, and directors)

Reported Results

Training is better linked to the overall business strategy. Major business
units and the training unit are now working together more effectively
because they understand how each unit is making a contribution to
achieving business goals.

14

Aligning IT Training with Goals Case Study 1

Governance Process Flow Model Strategy is Learning

 Links strategy to business direction

communicated Board

 Approves learning strategy, funds budget

from top to

 Evaluates metrics, validates plan

bottom

 Final authority on competing priorities

Learning

 Facilitates common threads

Council

 Evaluates metrics, validates plan  Oversees appeals process  Validates
budget

Learning

 Defines learning strategy

Services

 Analyzes business needs/ demand  Develops plan and metrics 
Establishes budget

Business

 Identify business needs/ demand

Lines

 Support business impact evaluation Metrics are Issues needing

reported up to resolution are

all levels referred

upward

Source: Company- provided graphic, used with permission.

15

Aligning IT Training with Goals Case Study 2

Strategic Learning Plan process helps align learning programs with
business priorities.

Practices Illustrated

 Enlist executive- level champions  Involve critical stakeholders 
Address future skill needs and new

technologies as part of the planning process

Background

As a multinational corporation with a complex organizational structure,
this technology company needed to create more cohesion between learning
strategies and the strategic planning process.

Challenge

To develop a corporate learning approach that aligns learning priorities
with business priorities.

16

Aligning IT Training with Goals Case Study 2

Solution

The company established a process that develops a strategic learning plan
that aligns learning programs with business priorities. The process begins
with determining business priorities, and then identifying skill needs,
potential learning interventions to address gaps, and predicted business
results. Part of this process, the strategic learning framework, includes
validating plans against strategic business priorities. Top management is
involved at key points to ensure alignment with strategic direction.

Reported Result

The process enables the company to develop and implement training that is
linked to its strategic business objectives and delivers measurable
results.

17

Aligning IT Training with Goals Case Study 2

Strategic Learning Plan Process

Strategic Learning Strategic Learning

Business Results Framework

Plan Business Gaps and

Validate business Learning Interventions A

priorities issues

priorities Learning Interventions B

Build market

Development

Learning Interventions C

share needs

Understand business

Expected business

Drive

Barriers to

issues and objectives

results innovative

achieving

Learning objectives technology

goals

Identify learning

Business metrics development

interventions

Measurement plan

Investment requirement

Prioritize learning interventions

Source: Company- provided graphic, used with permission.

18

Aligning IT Training with Goals Case Study 2

Detailed Steps in Strategic Learning Framework Step 1

Step 2 Step 3

Step 4 Validate Business

Understand Business Identify Learning

Prioritize Learning Priorities

Issues and Objectives Interventions

Interventions

 Review business plans  Identify specific

 Brainstorm learning  Review list of proposed

 Identify critical priorities business issues and

intervention alternatives interventions

and cause and effect links objectives

 Select high potential  Apply prioritized

Activities

 Identify key measures  Select critical issues to

interventions process

 Validate with unit/ be addressed by

 Validate with business organization leadership

learning unit leadership

 Perform gap analysis  Strategy map highlighting

 Critical business issues  Learning interventions  Finalized strategic

Deliverables

business priorities  Gaps potentially to be

learning plan  Selected business

addressed by learning  Measurement plan

measures  Corporate Learning

 Corporate Learning  Corporate Learning

 Corporate Learning

Roles

 Executive Sponsor  Learning Contact

 Learning Contact  Executive Sponsor

 Learning Contact  Subject Matter Experts

 Subject Matter Experts  Subject Matter Experts

 Subject Matter Experts

Critical stakeholders involved at all stages

Involvement by executive sponsor in steps 1 and 4 Plans are validated and
revalidated

helps ensure alignment with strategic goals against strategic objectives

Source: Company- provided graphic, used with permission.

19

Identifying and Assessing IT Training Needs

A company*s skill needs may change because of new initiatives, new
technology, market forces, workforce attrition, or mergers and
acquisitions, creating a need for training. Companies should assess
employees* competencies and identify gaps between skills that employees
need and those they have. Gathering information from various levels of the
company allows managers to better assess training needs. In addition,
industry experts and practitioners are beginning to recognize that IT
training should focus on broader career development needs as well as
skillspecific training needs.

We found that, rather than simply fulfilling training requests, nearly all
(10) of the companies are performing practices to more effectively
identify and assess training needs. Companies are also beginning to
address long- term career development issues.

20

Identifying and Assessing Needs

Practices

 Identify and document the competencies/ skills required for each job
description

 Maintain a current inventory of skills  Address overall career
development issues as well as skill- specific training

issues  Perform a gap analysis to determine where training is needed 
Use self- directed tools, such as individual development plans, to give

employees responsibility in assessing their development needs  Use a
single portal 3 to give staff and managers access to training and career

development information

3 A site featuring a suite of commonly used services, serving as a
starting point and frequent gateway to the Web (Web portal).

21

Identifying and Assessing Needs Case Study 3

Senior management input, technology councils, and individual development
plans are used to identify and assess workforce needs.

Practices Illustrated

 Identify and document skills  Use self- directed tools

Background

This multinational technology leader with a technically diverse workforce
places a high value on learning. The company has engineering and
information technology professionals in manufacturing and service
positions worldwide with a very diverse set of knowledge requirements.

Challenge

To identify the training that satisfies the needs of this technically
diverse and geographically dispersed workforce and prioritize the courses
in greatest demand.

22

Identifying and Assessing Needs Case Study 3

Solution

The company*s training organization uses the following resources to
effectively identify and assess training gaps and needs (also see slide on
next page):

 Senior management identifies corporatewide topics such as quality
management that require training support.

 Technology councils representing technical disciplines throughout the
company identify and assess training curricula for different IT and
engineering jobs.

 Individual development plans (IDPs) are self- directed tools completed
by each employee and supervisor to identify needed training. The IDP
information is compiled by the training organization through the use of a
learning management system to identify, prioritize, and schedule training
companywide.

Reported Results

The company*s ability to identify and prioritize training needs is
enhanced because it is based on input from management, subject- matter
experts, and individuals.

Skills needed for each job position are more comprehensively defined based
on input from technical experts through the organization.

Individual development plans allow employees to better identify and manage
their own training needs and establish commitment from both employees and
managers to fulfill those needs.

23

Identifying and Assessing Needs Case Study 3

Sources Used to Identify and Assess IT Training Needs Source: GAO. Note:
Analysis of company- provided data.

24

Identifying and Assessing Needs Case Study 4

IT learning tracks help employees assess their development needs.
Practices Illustrated

 Identify and document skills  Use self- directed tools  Use a single
portal

Background

This technology company is using the Internet to transform how it conducts
business. The company has thousands of technical specialists located all
over the world, who need to stay technically and professionally current on
whole categories of technology and new Internet- based products and
services.

Challenge

To provide readily accessible career and functional training information
to a diverse and global technical workforce that can use this information
to identify and assess their technical training and development needs.

25

Identifying and Assessing Needs Case Study 4

Solution

To establish IT learning tracks, the company took the following steps: 
Developed a Web- based single point of entry system that allows employees
located

worldwide to access technical training and career development information.
 Established an advisory council that defined the required skills for
each technical

position. The council is made up of subject matter experts and
representatives from the technical management and training groups.

 Created curricula on line for specific technical positions. The slide on
the next page is an example of a screen from the company*s training
portal. It shows the curriculum for IT Engineer* Java. Each curriculum is
divided into core courses, elective courses, and informal learning
sources.

 Designed comprehensive guidance to help employees manage their career
paths and enhance their professional development.

Reported Results

Workers* ready access to development information allows them to control
their own development and career paths.

Employee loyalty is enhanced by the availability of opportunities for
employees to direct their own learning pace and development.

26

Identifying and Assessing Needs Case Study 4

Position Core

Informal company

learning courses

sources Courses related to job function

Includes certifi- cations

Source: Company- provided graphic, used with permission.

27

Identifying and Assessing Needs Case Study 5

IT Professional Program guides career development. Practice Illustrated

 Address overall career development issues as well as skillspecific
training issues

Background

The IT department of a manufacturing company found that it needed well-
rounded IT staff with the skills that allowed them to be effective in the
business environment, not just in technical areas.

Challenge

To develop balanced, well- rounded IT professionals rather than narrowly
focused technicians.

28

Identifying and Assessing Needs Case Study 5

Solution

The company developed the *IT Professional Program,* which creates a
career ladder to a technical position at the senior executive level. The
program is used to guide and evaluate the career development of IT
professionals. Competencies are identified in six categories, only one of
which is technical:

 Leadership (e. g., facilitation capabilities, persuasiveness, developing
others)  Innovation (e. g., strategic thinking, applying new skills) 
Effectiveness (e. g., teamwork, customer consciousness, prudent risk
taking)  Communications (e. g., written and personal communications,
negotiation)  Business processes (e. g., corporate- specific processes,
IT division processes)  Technical and business domain (e. g., job-
specific technical skills)

Employees advance based on their proficiency in the competencies. Because
the objective is to develop balanced, well- rounded staff proficient in
nontechnical and technical skills, all six evaluation categories are
weighted equally. Salary determination is based on the lowest- scoring
category.

29

Identifying and Assessing Needs Case Study 5

Solution ( cont d)

The company provides formal training through a corporate university, on-
line courses, and courses at local colleges, but it also uses on- the- job
training since company staff believe that formal training provides only a
small part of what an employee needs to know to perform effectively.

Reported Result

The IT Professional Program has helped develop well- rounded IT staff who
understand the business and can work well with other staff and business
units. Other benefits are improved retention and morale, because career
ladder steps are well defined, employees understand what they need to do
to get promoted, and there is a full career ladder up to a senior level.

30

Allocating IT Training Resources

As in any other business area, training must compete for corporate
resources. It is important that companies prioritize and manage resources
to ensure that training projects are effectively identified and
implemented.

The leading companies we visited use various approaches to deciding how to
allocate training resources. Four of these companies are establishing
structured investment processes to select and manage training projects.
Four others target large amounts of training resources at a fairly small
group of promising applicants.

31

Allocating IT Training Resources

Practices

 Ensure that an investment process is in place to select and manage
training projects

 Consider the benefits and costs associated with various training design
and delivery methods* e. g., Internet- based as opposed to classroom
training

 Identify people who have high potential and provide them specialized
training opportunities

 Ensure that resources are allocated for management training* e. g.,
leadership and project management

32

Allocating IT Training Resources Case Study 6

An integrated investment management process is used to select and manage
training projects.

Practices Illustrated

 Ensure that an investment process is in place  Consider the benefits
and costs associated with various training design and delivery methods

Background

During the mid- 1990s, because funding for training at this large
technology company was curtailed, the company reassessed its training
processes. Funding and investment decisions for training were done by
various business groups on an ad hoc basis. Little thought was given to
how new training initiatives impacted the company*s strategic goals.
Further, few processes were in place to ensure that the selection and
implementation of the most strategic training projects were being managed
properly.

Challenge

To institute an integrated training investment management process that
involves both the business unit and corporate levels.

Solution

Executive stakeholders from the major business groups reallocated 10% of
their training budgets to a companywide strategic training portfolio.

Key organizational decisionmakers became involved in the investment
management process. 33

Allocating IT Training Resources Case Study 6

Solution ( cont d)

The company instituted an investment management process (see next slide)
for all major training development projects that

 clearly defines responsibilities and authority for each stakeholder; 
includes a ranking process using weighted criteria to compare and rank
projects as part

of investment reviews (these criteria include anticipated demand for the
investment from internal sources; potential for increased revenue; risk of
unfavorable consequences if investment is not made; availability of
resources to support the project; and business value or strategic fit);
and

 requires that all training projects are monitored from initial design
through implementation.

Reported Results

Investment decisionmaking at the corporate level (through an Executive
Review Board and Integrated Portfolio Management Teams) is based on how
training investments support corporate business goals and results.

Project development teams can better manage risks and resources associated
with each training initiative.

34

Allocating IT Training Resources Case Study 6

Source: GAO. Note: Analysis of company- provided data.

35

Allocating IT Training Resources Case Study 7

A training program was established to target carefully screened
applicants.

Practice Illustrated

 Provide specialized training for people with high potential

Background

This major finance company is highly dependent on technology in conducting
its business, which includes selling software products to financial
institutions and developing unique systems. It has historically found it
difficult to hire enough qualified technical employees, especially during
the tight job market of the 1990s, and it needed a method of addressing
skills shortages.

Challenge

To institute a program that identifies promising nontechnical candidates
and rapidly trains them to become proficient in technical areas.

36

Allocating IT Training Resources Case Study 7

Solution

To fill staff shortages or anticipated staff shortages in technology
positions, the company established the Business Technologist Program. This
intensive 3- year training program is divided into individual programs
that focus on particular target audiences or job positions. For example,
one program focuses on carefully selected nontechnical new hires with
prior business experience.

A large component of the program cost is the salaries. Participants are
paid during 18 weeks of training before they are placed in productive
jobs. Training facility costs* such as workstations and networks* can also
be high. The company selects applicants very carefully before investing in
their training and development, and protects its investment by careful
monitoring.

During the classroom training, recruits are monitored by frequent testing
and helped if they seem to be falling behind. After being placed in jobs,
they are assigned mentors, and performance is evaluated at half- year
intervals instead of annually as with other staff.

37

Allocating IT Training Resources Case Study 7

Solution ( cont d)

The company keeps costs down by paying recruits approximately 50 percent
of the market rate for a seasoned IT professional during their training
period. This is usually a pay reduction for people entering the program.
Salaries can be increased after each semi- annual performance review and
rise to market levels during the 3 years of the program.

Reported Results

The company stated that the program is effective because of the number and
quality of its graduates* more than 500 people over the course of the
program. It has been able to quickly train staff to fill technical
positions, and retention of program participants has been better than
average. An additional benefit has been increased diversity in the
workforce.

38

Allocating IT Training Resources Case Study 8

Company targets resources to develop a cadre of skilled project managers.
Practices Illustrated

 Identify people who have high potential and provide them with
specialized training  Ensure that resources are allocated for management
training* e. g., leadership

and project management

Background

This large financial services company had a number of *painful costly
project experiences caused by a lack of project management knowledge and
experience.* Project managers from IT and from the various lines of
business tended to focus more on their respective needs rather than the
overall needs of the organization. The company lacked an internal network
of skilled, qualified project managers.

Challenge

To develop a cadre/ network of information technology managers (and
business managers) with the proper project management skills, internal
professional contacts, and management background to deliver projects
within cost and on schedule.

39

Allocating IT Training Resources Case Study 8

Solution

The company*s IT training group designed a comprehensive project
management development program (PMDP) for IT managers and others in
leadership roles. The program includes two separate 4- month training
programs: PMDP Level 1 for new project managers and PMDP Level 2 for
experienced project managers. The company took the following steps to
support the success of training:

 obtained a senior executive sponsor;  assembled an internal consulting
team possessing a broad view of the

company*s projects, and having management and training experience; 
identified the causes for project failures within the company and used the

information to help develop the program;  created the program around six
broad categories of required skills (project

planning, project control and management, managing the team, managing the
environment, risk management, and the company*s project life cycle);

40

Allocating IT Training Resources Case Study 8

Solution ( cont d)

 identified top performers and ensured an optimal mix of participants
that broadly represented the company (business and IT managers);

 created a program that allowed trainees to network and be mentored; and
 developed evaluation tools to assess the PMDP.

Reported Result

The program reduced the number of project difficulties caused by
inadequate project management knowledge and skills.

It expanded the internal network of skilled, qualified IT and business
project managers who provide a ready source of expertise.

It also facilitated the exchange of approaches and practices that help
ensure successful project management.

41

Designing and Delivering IT Training

Once training needs are identified and resources allocated, it is
important that the training design and delivery process ensures that
learning occurs during the training and also ensures that the employee
applies the training on the job.

The majority of the companies we studied said that they are still using
traditional instructor- led training. However, they are taking advantage
of more flexible design and delivery methods made possible by technology
to deliver training to the user*s desktop and to make training more
convenient.

42

Designing and Delivering Training

Practices

 Provide IT trainees with the flexibility to choose among different IT
training delivery methods

 Ensure that on- the- job training is planned and monitored as part of
the training process

 Consider combining different teaching methods (e. g., Web- based and
instructor- led) within the same course

 Provide just- in- time training  Consider outsourcing training
solutions* e. g., university partnerships and

external IT training and content providers  Build courses using reusable
components

43

Designing and Delivering Training Case Study 9

An e- learning solution offers flexibility in learning to a worldwide
workforce. Practices Illustrated

 Provide flexibility to choose among different delivery methods 
Consider combining different teaching methods  Provide just- in- time
training  Build courses using reusable components

Background

This technology company believes that e- learning is a revolutionary way
to empower a workforce with the skills and knowledge it needs. The company
implemented e- learning solutions internally so it can gain advantages and
also act as an e- learning model for its customers.

Challenge

To create an environment where learning resources and information are
clearly integrated into everyday job functions and readily accessible to
employees worldwide.

44

Designing and Delivering Training Case Study 9

Solution

To implement e- learning solutions, the company took the following steps:
 Established e- learning portals on the company*s network that are
integrated with its ebusiness strategy. (These portals are illustrated on
the following page.)

 Implemented a central e- learning system (see graphic on next slide),
accessible from the portals, that provides employees with Web- based
learning solutions, including video- ondemand (VOD)* which captures expert
instructors on video* computer- based training, an electronic library,
virtual labs, and net meetings. These delivery methods may be combined in
a single training course.

 Invested in infrastructure that supports the company*s content delivery
(VOD requires high bandwidth).

 Invested in a state- of- the- art video studio for its VOD learning
solutions.  Established a reusable object strategy that enables the
training workforce to create and

deliver modular, reusable learning *chunks* for different learning
experiences.

Reported Results

The company increased employee productivity and communications and
decreased costs. 45

Designing and Delivering Training Case Study 9

E- Learning/ Internet Learning Environment

Source: Company- provided graphic, used with permission.

46

Designing and Delivering Training Case Study 10

Reusable learning component strategy offers flexible approach to improve
course design/ delivery productivity and cost savings.

Practices Illustrated

 Provide just- in- time training  Build courses using reusable
components

Background

Career certification courses at this technology company were previously
delivered almost exclusively in an instructor- led format. In 2000, to
fulfill its goal of becoming a leader in e- learning, the company devised
a strategy to develop reusable components for its course offerings. These
components would allow courses to be packaged and delivered in various
ways, including traditional classroom training, live virtual training, or
self- paced e- learning.

Challenge

To move the company*s training courses from large inflexible courses to
searchable database- driven objects* reusable chunks of information* that
could be reused and modified independent of their delivery method.

47

Designing and Delivering Training Case Study 10

Solution

A strategy was developed that included the following tasks:  Define
standards and guidelines for designing and

developing reusable learning components. Curriculum

 Establish a hierarchical curriculum structure, as at Unit

right.  Create reusable learning objects (* lessons*) by

Module combining individual reusable information objects.

(Reusable Lesson

learning  Create reusable information objects (* topics* or

object) *pages*) by combining content items, practice items,

Topic (Reusable

and assessment items. information

object) The implementation strategy began with a limited number

Source: GAO. Note: Analysis of company- provided data.

of certification courses to give developers time to embrace the new
approach. The company now has 20,000 reusable learning objects that can be
reused for course development.

48

Designing and Delivering Training Case Study 10

Reported Results

Course development time was significantly reduced* i. e., selected courses
were developed in half the time/ budget because learning objects were
reused.

The company achieved a positive return on its investment. The approach
enabled all field staff to access on- demand, personalized training, in
the media of their choice.

Users have the ability to offer an assessment of the objects people need
to achieve the desired performance.

49

Evaluating/ Demonstrating the Value of IT Training

Companies can determine whether training has achieved its intended
business impact by integrating training measurements into their strategy
and operations. Increasingly, stakeholders who fund training are
interested in how training investments contribute to the company*s
business results.

Companies we visited recognize the importance of effectively evaluating
the impact of training on business goals. All the companies are going
beyond simply obtaining participant reactions. For example, some use tests
or certifications to validate that the content was understood; others are
deploying evaluation processes that measure business results.

50

Evaluating/ Demonstrating Value

Practices

 Collect information on how job performance is affected by training 
Validate IT content learning by testing and certification of specific
skills* e. g.,

Java or C ++  Assess evaluation results in terms of business impact

51

Evaluating/ Demonstrating Value Case Study 11

An automated evaluation process measures and reports training results.
Practices Illustrated

 Collect information on how job performance is affected  Validate
content learning by testing and certification

Background

This large financial services firm needed to improve the communication
between its IT training organization, called Learning Systems, and its IT
organization.

Challenge

To demonstrate the value of the training process to IT organization
managers and facilitate their involvement in the training process.

Solutions

An automated evaluation process has been implemented to demonstrate the
value of training through participant evaluations, automated surveys of
how well training is applied on the job, and to some extent, Web- based
testing and certifications.

52

Evaluating/ Demonstrating Value Case Study 11

Solutions ( cont d)

Specifically, the automated evaluation process is used for the following:
 Obtaining participant reaction:

 The Learning Systems computer system automatically generates an
evaluation form that participants must complete before leaving class. 
The data are compiled and analyzed by a proprietary database analysis tool

and used to determine changes in instructors, course design, and other
training factors.

 Assessing content learning:  Some courses use Web- based testing to
demonstrate proficiency.  Many related career certifications also require
testing to demonstrate

proficiency.  Determining how training is applied on the job:

 Each month, the training organization selects courses for further
assessment.  Participants receive an electronic survey about 1 month
after course

completion, in which they indicate how they applied the training on the
job. The participant*s manager receives a similar survey about the
participant.  The information is then used to refine training activities.

53

Evaluating/ Demonstrating Value Case Study 11

Solutions ( cont d)

The Learning Systems director meets with IT organization business managers
each month to discuss how learning strategies can best help the IT
organization achieve its business objectives. These discussions include
budget, hours of training delivered, reactions to training, and types of
delivery methods.

Reported Results

Participant evaluations, testing, and post- training feedback help assess
the value of training.

Collecting evaluation data after the employee has returned to the work
environment helps determine if training was applied on the job.

The monthly discussions help improve communication between the IT
organization and Learning Systems.

54

Evaluating/ Demonstrating Value Case Study 12

Certifications are used to measure learned skills. Practice Illustrated

 Validate content learning by testing and certification

Background

This telecom company created a new group to be responsible for the
corporate Web site and for sites used by customers to communicate with the
company. The vice president in charge worked with the chief technologist
to develop a list of skills, matching needed skills to job categories.
Once the group was up and running, the skills list was refined by
consultations with managers and staff.

Challenge

To measure progress in developing the skills needed to provide new and
improved Web services to clients.

55

Evaluating/ Demonstrating Value Case Study 12

Solution

The company contracted for Web- based certification testing on the
required skills, such as Java programming. Employees were required to pass
all the certifications needed for their jobs, but were allowed to fail and
retake the certification tests as many times as needed without penalties.
Incentives and rewards, such as public recognition or lunch with the
senior vice president, were offered to motivate staff. Training relevant
to the certifications was provided by about 400 on- line courses, as well
as through books recommended by the certification vendor and by informal,
on- the- job training.

The senior vice president in charge of the group tracked the group*s
progress on certifications against performance metrics, including time to
complete projects and number of pages of program code rejected by quality
control. Improvements in the code reject rate were seen after employees
passed certification tests.

56

Evaluating/ Demonstrating Value Case Study 12

Reported Results

By linking training and certification, the company could directly measure
employee content learning and also validate training courses.

By monitoring individuals* performance using the metric of pages of code
rejected, the company was able to demonstrate improved job performance
correlated with employees* becoming certified.

The certification process and how it was administered led to increased
employee motivation to take training and improved morale, since employees
valued the certifications.

57

Issues Critical to Effective IT Training The case studies illustrate that
practices identified and performed by leading private sector companies can
result in more effective training management. However, based on company
and expert input, we noted several crosscutting issues that should be
considered in implementing effective IT training practices. These issues
include

 Responding to the rapid pace of technological and social change . Rapid
changes in technology (e. g., the growth of e- learning) affect how
training is delivered as well as what competencies are needed (e. g., the
growing need for information security skills). Changes in the IT
workforce, such as increased diversity, may require a different mix of
skills.

 Demonstrating return on investment in IT training . Although there is
pressure to show return on investment (ROI) from training, it is difficult
and costly to demonstrate ROI. Only two companies reported that they were
actually calculating ROI on IT training.

 Managing funding constraints during an economic downturn . The economic
climate and its impact on training expenditures may affect the overall
level of training delivered by organizations or the mix of delivery
methods* such as outsourcing or e- learning.

58

Issues Critical to Effective IT Training  Obtaining visibility for
training at the highest levels of the organization.

Several of our companies have existing or emerging processes that include
executive management involvement. This can enhance an organization*s
ability to incorporate training in its planning, and can improve a
training organization*s ability to undertake new initiatives.

59

A Final Note GAO*s review of private sector practices indicates that
training is not simply a support function, but a strategic element in
achieving corporate objectives. Traditional training activities are being
reinvented to focus on implementing a comprehensive learning strategy and
demonstrating business impact. Companies are adopting new ideas about
training, but many initiatives are in their early stages, and private
sector officials expressed interest in learning about innovative practices
emerging from the public sector.

60

(310444)

GAO*s Mission The General Accounting Office, the investigative arm of
Congress, exists to support Congress in meeting its constitutional
responsibilities and to help improve

the performance and accountability of the federal government for the
American people. GAO examines the use of public funds; evaluates federal
programs and policies; and provides analyses, recommendations, and other
assistance to help Congress make informed oversight, policy, and funding
decisions. GAO*s commitment to good government is reflected in its core
values of accountability, integrity, and reliability.

Obtaining Copies of The fastest and easiest way to obtain copies of GAO
documents at no cost is

through the Internet. GAO*s Web site (www. gao. gov) contains abstracts
and fulltext GAO Reports and

files of current reports and testimony and an expanding archive of older
Testimony

products. The Web site features a search engine to help you locate
documents using key words and phrases. You can print these documents in
their entirety, including charts and other graphics.

Each day, GAO issues a list of newly released reports, testimony, and
correspondence. GAO posts this list, known as *Today*s Reports,* on its
Web site daily. The list contains links to the full- text document files.
To have GAO e- mail this

list to you every afternoon, go to www. gao. gov and select *Subscribe to
GAO Mailing Lists* under *Order GAO Products* heading.

Order by Mail or Phone The first copy of each printed report is free.
Additional copies are $2 each. A check or money order should be made out
to the Superintendent of Documents. GAO

also accepts VISA and Mastercard. Orders for 100 or more copies mailed to
a single address are discounted 25 percent. Orders should be sent to:

U. S. General Accounting Office 441 G Street NW, Room LM Washington, D. C.
20548

To order by Phone: Voice: (202) 512- 6000 TDD: (202) 512- 2537 Fax: (202)
512- 6061

To Report Fraud, Contact:

Waste, and Abuse in Web site: www. gao. gov/ fraudnet/ fraudnet. htm

E- mail: fraudnet@ gao. gov Federal Programs

Automated answering system: (800) 424- 5454 or (202) 512- 7470 Public
Affairs Jeff Nelligan, Managing Director, NelliganJ@ gao. gov (202) 512-
4800

U. S. General Accounting Office, 441 G Street NW, Room 7149 Washington, D.
C. 20548

Report to Congressional Requesters

January 2003 INFORMATION TECHNOLOGY TRAINING

Practices of Leading Private- Sector Companies

GAO- 03- 390

Letter 1 Appendix

Appendix I: Information Technology Training 4

This is a work of the U. S. Government and is not subject to copyright
protection in the United States. It may be reproduced and distributed in
its entirety without further permission from GAO. It may contain
copyrighted graphics, images or other materials. Permission from the
copyright holder may be necessary should you wish to reproduce copyrighted
materials separately from GAO*s product.

a

GAO United States General Accounting Office

GAO identified 22 existing and emerging training practices used by leading
companies to implement effective IT training. We organized these practices
and accompanying case studies under five training management processes
that we defined based on input from industry experts, published research,
and previous GAO work (see table below). Although none of the companies
was performing all the practices, the majority performed 10 or more.

Organizations and experts agree that these practices could result in more
effective training management, but in applying the identified practices,
we noted several critical issues (e. g., funding constraints and
demonstrating

return on investment) that should be considered. The practices may also
suggest approaches to IT training for government agencies to consider.

GAO*s discussions with leading private sector companies indicate that
training is not simply a support function, but a strategic element in
achieving corporate objectives. Further, although companies are adopting
new ideas about training, many initiatives are in their early stages, and
private sector officials expressed interest in learning about innovative
practices emerging from the public sector.

IT Training Management Processes and Sample Practices (see app. I for full
list) Management processes Sample practices Enlist executive- level
champions Align IT training with

business goals Involve critical stakeholders Document competencies/ skills
required for each job description Identify and assess IT training needs
Perform a gap analysis to determine needed training

Use an investment process to select and manage training projects Allocate
IT training resources Provide resources for management training, e. g.,
leadership and

project management Give trainees choices among different training delivery
methods Design and deliver IT training Build courses using reusable
components

Collect information on how job performance is affected by training
Evaluate/ demonstrate the value of IT training Assess evaluation results
in terms of business impact

Source: GAO. Note: Analysis of company- provided information. INFORMATION
TECHNOLOGY TRAINING

Practices of Leading Private- Sector Companies www. gao. gov/ cgi- bin/
getrpt? GAO- 03- 390. To view the full report, including the scope

and methodology, click on the link above. For more information, contact
Joel Willemssen at (202) 512- 6253 or willemssenj@ gao. gov. Highlights of
GAO- 03- 390, a report to the

Chairman, Tom Davis, Committee on Government Reform and Representative Jim
Turner, House of Representatives

January 2003

The rapid pace of technological change, with its potential to transform
the way the government

delivers services, makes information technology (IT) human capital a
critical issue for federal agencies.

GAO has identified strategic human capital management as a high risk area
for the federal government, and the demand for skilled IT workers is
expected to increase

over the long term. Given that competition for workers affects the federal
government as it does any

other employer, effective training of staff is essential to developing and
retaining a qualified workforce. Some private- sector companies are
recognized for their effective and innovative training programs for the IT
workforce, which could provide models and examples for federal agencies.
To help federal agencies better design and

implement such training programs, GAO was asked to examine privatesector
practices for training both IT and non- IT professionals (e. g., business
managers and other staff

needing training in IT) that could be used as a basis for addressing
federal efforts.

Page i GAO- 03- 390 Information Technology Training

Contents

Page 1 GAO- 03- 390 Information Technology Training United States General
Accounting Office

Washington, D. C. 20548 Page 1 GAO- 03- 390 Information Technology
Training

A

Page 2 GAO- 03- 390 Information Technology Training

Page 3 GAO- 03- 390 Information Technology Training

Page 4 GAO- 03- 390 Information Technology Training

Appendix I

Appendix I Information Technology Training

Page 5 GAO- 03- 390 Information Technology Training

Appendix I Information Technology Training

Page 6 GAO- 03- 390 Information Technology Training

Appendix I Information Technology Training

Page 7 GAO- 03- 390 Information Technology Training

Appendix I Information Technology Training

Page 8 GAO- 03- 390 Information Technology Training

Appendix I Information Technology Training

Page 9 GAO- 03- 390 Information Technology Training

Appendix I Information Technology Training

Page 10 GAO- 03- 390 Information Technology Training

Appendix I Information Technology Training

Page 11 GAO- 03- 390 Information Technology Training

Appendix I Information Technology Training

Page 12 GAO- 03- 390 Information Technology Training

Appendix I Information Technology Training

Page 13 GAO- 03- 390 Information Technology Training

Appendix I Information Technology Training

Page 14 GAO- 03- 390 Information Technology Training

Appendix I Information Technology Training

Page 15 GAO- 03- 390 Information Technology Training

Appendix I Information Technology Training

Page 16 GAO- 03- 390 Information Technology Training

Appendix I Information Technology Training

Page 17 GAO- 03- 390 Information Technology Training

Appendix I Information Technology Training

Page 18 GAO- 03- 390 Information Technology Training

Appendix I Information Technology Training

Page 19 GAO- 03- 390 Information Technology Training

Appendix I Information Technology Training

Page 20 GAO- 03- 390 Information Technology Training

Appendix I Information Technology Training

Page 21 GAO- 03- 390 Information Technology Training

Appendix I Information Technology Training

Page 22 GAO- 03- 390 Information Technology Training

Appendix I Information Technology Training

Page 23 GAO- 03- 390 Information Technology Training

Appendix I Information Technology Training

Page 24 GAO- 03- 390 Information Technology Training

Appendix I Information Technology Training

Page 25 GAO- 03- 390 Information Technology Training

Appendix I Information Technology Training

Page 26 GAO- 03- 390 Information Technology Training

Appendix I Information Technology Training

Page 27 GAO- 03- 390 Information Technology Training

Appendix I Information Technology Training

Page 28 GAO- 03- 390 Information Technology Training

Appendix I Information Technology Training

Page 29 GAO- 03- 390 Information Technology Training

Appendix I Information Technology Training

Page 30 GAO- 03- 390 Information Technology Training

Appendix I Information Technology Training

Page 31 GAO- 03- 390 Information Technology Training

Appendix I Information Technology Training

Page 32 GAO- 03- 390 Information Technology Training

Appendix I Information Technology Training

Page 33 GAO- 03- 390 Information Technology Training

Appendix I Information Technology Training

Page 34 GAO- 03- 390 Information Technology Training

Appendix I Information Technology Training

Page 35 GAO- 03- 390 Information Technology Training

Appendix I Information Technology Training

Page 36 GAO- 03- 390 Information Technology Training

Appendix I Information Technology Training

Page 37 GAO- 03- 390 Information Technology Training

Appendix I Information Technology Training

Page 38 GAO- 03- 390 Information Technology Training

Appendix I Information Technology Training

Page 39 GAO- 03- 390 Information Technology Training

Appendix I Information Technology Training

Page 40 GAO- 03- 390 Information Technology Training

Appendix I Information Technology Training

Page 41 GAO- 03- 390 Information Technology Training

Appendix I Information Technology Training

Page 42 GAO- 03- 390 Information Technology Training

Appendix I Information Technology Training

Page 43 GAO- 03- 390 Information Technology Training

Appendix I Information Technology Training

Page 44 GAO- 03- 390 Information Technology Training

Appendix I Information Technology Training

Page 45 GAO- 03- 390 Information Technology Training

Appendix I Information Technology Training

Page 46 GAO- 03- 390 Information Technology Training

Appendix I Information Technology Training

Page 47 GAO- 03- 390 Information Technology Training

Appendix I Information Technology Training

Page 48 GAO- 03- 390 Information Technology Training

Appendix I Information Technology Training

Page 49 GAO- 03- 390 Information Technology Training

Appendix I Information Technology Training

Page 50 GAO- 03- 390 Information Technology Training

Appendix I Information Technology Training

Page 51 GAO- 03- 390 Information Technology Training

Appendix I Information Technology Training

Page 52 GAO- 03- 390 Information Technology Training

Appendix I Information Technology Training

Page 53 GAO- 03- 390 Information Technology Training

Appendix I Information Technology Training

Page 54 GAO- 03- 390 Information Technology Training

Appendix I Information Technology Training

Page 55 GAO- 03- 390 Information Technology Training

Appendix I Information Technology Training

Page 56 GAO- 03- 390 Information Technology Training

Appendix I Information Technology Training

Page 57 GAO- 03- 390 Information Technology Training

Appendix I Information Technology Training

Page 58 GAO- 03- 390 Information Technology Training

Appendix I Information Technology Training

Page 59 GAO- 03- 390 Information Technology Training

Appendix I Information Technology Training

Page 60 GAO- 03- 390 Information Technology Training

Appendix I Information Technology Training

Page 61 GAO- 03- 390 Information Technology Training

Appendix I Information Technology Training

Page 62 GAO- 03- 390 Information Technology Training

Appendix I Information Technology Training

Page 63 GAO- 03- 390 Information Technology Training

United States General Accounting Office Washington, D. C. 20548- 0001
Official Business Penalty for Private Use $300 Address Service Requested

Presorted Standard Postage & Fees Paid

GAO Permit No. GI00
*** End of document. ***