Information Technology Training: Practices of Leading
Private-Sector Companies (31-JAN-03, GAO-03-390).
The rapid pace of technological change, with its potential to
transform the way the government delivers services, makes
information technology (IT) human capital a critical issue for
federal agencies.GAO has identified strategic human capital
management as a high risk area for the federal government, and
the demand for skilled IT workers is expected to increase over
the long term. Given that competition for workers affects the
federal government as it does any other employer, effective
training of staff is essential to developing and retaining a
qualified workforce. Some private-sector companies are recognized
for their effective and innovative training programs form the IT
workforce, which could provide models and examples for federal
agencies. To help federal agencies better design and implement
such training programs, GAO was asked to examine private-sector
practices for training both IT and non-IT professionals (e.g.,
business managers and other staff needing training in IT) that
could be used as a basis for addressing federal efforts.
-------------------------Indexing Terms-------------------------
REPORTNUM: GAO-03-390
ACCNO: A06014
TITLE: Information Technology Training: Practices of Leading
Private-Sector Companies
DATE: 01/31/2003
SUBJECT: Employment or training programs
Human resources utilization
Information technology
Personnel management
******************************************************************
** This file contains an ASCII representation of the text of a **
** GAO Product. **
** **
** No attempt has been made to display graphic images, although **
** figure captions are reproduced. Tables are included, but **
** may not resemble those in the printed version. **
** **
** Please see the PDF (Portable Document Format) file, when **
** available, for a complete electronic file of the printed **
** document's contents. **
** **
******************************************************************
GAO-03-390
A
Letter
January 31, 2003 The Honorable Tom Davis Chairman Committee on Government
Reform House of Representatives
The Honorable Jim Turner House of Representatives
The rapid pace of technological change, with its potential to transform
the way the government delivers services, makes information technology
(IT) human capital a critical issue for federal agencies. We have
identified strategic human capital management as a high risk area. In the
long term, demand for highly skilled IT workers is expected to increase.
According to a 2002 study of private- sector employers by the Information
Technology Association of America (ITAA), the demand for these highly
skilled IT workers exceeds supply. 1 Given that this reported shortage
affects the federal government as it does any other employer, effective
training of staff is essential to developing and retaining a qualified
workforce. Our objective was to examine private- sector workforce training
practices
for both IT and non- IT professionals (e. g., business managers and other
staff needing training in IT). To achieve this objective, we reviewed
existing research, held discussions with academic and professional
authorities, and
interviewed executives and managers at leading companies about their IT
training management practices and activities. We also collaborated with
the National Academy of Science to host a panel discussion with academic
authorities. The experts on this panel discussed overall training issues,
provided input on training management processes, and identified other
effective training practices. We used this information, as well as the
extensive research and trade literature available on IT training
practices, to develop a view of what leading private- sector organizations
are doing in the IT training area.
On October 18, 2002, we provided briefing slides on the results of our
study to you in your capacities as Chairman and Ranking Minority Member of
the Subcommittee on Technology and Procurement Policy. These results,
along
1 Information Technology Association of America, Bouncing Back: Jobs,
Skills, and the Continuing Demand for IT Workers (May 2002).
with additional information and new and expanded case studies, are
included as an appendix to this letter. The purpose of this letter is to
officially transmit the information in published form to you as Chairman
and Member of the Committee on Government Reform.
In brief, we found 22 existing and emerging practices that are used by
leading companies to implement effective IT training. The majority of the
companies performed 10 or more of the identified practices, but none was
performing all. The practices and case studies provided in the appendix
suggest approaches to IT training that government agencies could consider.
In addition, we noted several critical issues (e. g., funding constraints
and demonstrating return on investment) that should be considered in
implementing these practices. GAO*s review of private- sector practices
indicates that training is not simply a support function, but a strategic
element in achieving corporate objectives. Further, although companies are
adopting new ideas about training, many initiatives are in their early
stages, and private- sector officials expressed interest in learning about
innovative practices emerging from the public sector.
Many organizations contributed to our study. American Telephone &
Telegraph (AT& T), Cable & Wireless, Cisco, Delta Technology, Fannie Mae,
FleetBoston, International Business Machines (IBM), International Truck
and Engine, Raytheon, Science Applications International Corporation
(SAIC), and United Services Automobile Association (USAA) all met with us
to discuss their training programs. The federal Chief Information Officers
(CIO) Council, the Information Technology Association of America (ITAA),
the Private Sector Council, and the American Society for Training &
Development (ASTD), as well as IT consulting firms Gartner and Giga, also
provided assistance and information.
Unless you publicly announce the contents of this report earlier, we plan
no further distribution until 30 days from the report date. At that time,
we will send copies of this report to interested congressional committees.
In addition, copies will be made available to others upon request. Copies
of this report are also available at no charge on GAO*s Web site at www.
gao. gov.
If you or your offices should have any questions concerning this report,
please contact me at (202) 512- 6253 or Megen Davis, Assistant Director,
at (202) 512- 6398. We can also be reached by E- mail at willemssenj@ gao.
gov and davism@ gao. gov, respectively. Key contributors to this report
were
Barbara Collier, Vijay D*Souza, John Ortiz, Tomas Ramirez, Jr., and Glenn
Spiegel.
Joel Willemssen Managing Director, Information Technology
Appendi Appendi xes x I
Information Technology Training Information Technology Training: Practices
of Leading Private- Sector Companies
Committee on Government Reform House of Representatives
1
Contents Introduction and Objective Scope and Methodology Background
Private- Sector Practices
Aligning IT training with business goals Identifying and assessing IT
training needs Allocating IT training resources Designing and
delivering IT training Evaluating/ demonstrating the value of IT
training Critical Issues Final Note
2
Introduction and Objective GAO has identified strategic human capital
management as a high risk area for the federal government. According to a
study of private- sector employers by the Information Technology
Association of America (ITAA), the demand for skilled IT workers exceeds
supply, and the shortage is expected to continue. 1 Given that the
shortage affects the federal government as it does any other employer,
effective training of existing staff is essential to developing and
retaining a qualified workforce.
Some private- sector companies are recognized for their effective and
innovative training programs, which could provide models and examples for
federal agencies. To help federal agencies better design and implement IT
workforce training programs, you asked us to examine private- sector
training practices both for IT professionals and for other staff needing
IT training (e. g., business managers) that could be used as a basis for
addressing federal efforts.
1 Information Technology Association of America Bouncing Back: Jobs,
Skills and the Continuing Demand for IT Workers (May 2002).
3
Scope and Methodology We selected companies considered leaders in IT
training by professional organizations, publications, and academic
experts. We based our selection on awards, significant accolades, and
expert recommendations. Specifically:
We searched published sources and the Internet for companies recognized
as leaders by trade associations and industry publications. Examples
include
Computerworld and Training magazine rankings. We solicited
recommendations from industry and academic experts. To identify and
establish contacts with candidate companies, we consulted
with the federal Chief Information Officers (CIO) Council;
professional associations (the Information Technology Association of
America, the Private Sector Council, and the American Society for Training
& Development); and
IT consulting firms (Gartner and Giga). 4
Scope and Methodology On the basis of our discussions and analyses, we
conducted site visits at 11 companies identified as leaders in IT
training:
American Telephone & Telegraph International Business Machines
(AT& T)* Telecom (IBM)* Technology
Cable & Wireless* Telecom International Truck and Engine*
Cisco* Technology Manufacturing
Delta Technology* Airline Raytheon* Defense & Electronics
Fannie Mae* Finance Science Applications International
Corporation (SAIC)* Research & FleetBoston* Finance
Engineering United Services Automobile
Association (USAA)* Finance & Insurance
5
Scope and Methodology At the companies visited, we reviewed training
program documentation and interviewed executives, business unit managers,
training managers, and training recipients about their programs.
Specifically, we provided a standard set of questions to each company to
obtain information on the organizational placement, structure, and
management of training. We also obtained additional documentation on
specific training initiatives.
We analyzed the data to identify existing and emerging training practices.
The majority of the companies performed 10 or more of the identified
practices, although none was performing all. However, organizations and
experts agreed that all the practices could result in more effective
training management. The companies also reviewed and commented on a draft
list of the practices. These practices may suggest improvements in public
sector training programs.
We also analyzed the data to develop specific case studies that
exemplified one or more of the practices. At the conclusion of our
research, all companies reviewed and verified the accuracy of our
descriptions of their training initiatives. We did not, however, verify
the accuracy of claims made by the companies.
6
Scope and Methodology To organize our work, we grouped the practices and
case studies into five categories (training management processes) that we
identified based on input from industry experts, published research, and
our previous work. The categories are (1) aligning IT training with
business goals, (2) identifying and assessing IT training needs, (3)
allocating IT training resources, (4) designing and delivering IT
training, and (5) evaluating/ demonstrating the value of IT training.
To validate our methodology, we consulted internal and external experts on
workforce development and training issues.
Also, as part of the consultative process, we collaborated with the
National Academies of Science to host a 1- day panel discussion with
academic authorities. The panel experts discussed overall training issues,
validated our management processes, and identified effective training
practices. The panel*s insights provided a broader perspective than
companies alone could provide.
7
Scope and Methodology Further, we facilitated on- line discussions of
preliminary results with academic, private- sector, and cognizant GAO
staff to solicit comments and feedback on key issues.
We conducted our review from November 2001 to November 2002, in accordance
with generally accepted government auditing standards.
8
Background Training is a vital part of the human capital equation. An
organization*s
approaches to human capital development and training should be aligned to
support its mission, vision, goals and objectives, and strategies.
According to a study by the National Association of Public
Administration, 2
there is a lack of investment in continuous learning within the federal
government. The study further stated that this lack of investment is
especially problematic in the dynamic and rapidly changing world of IT.
As companies are forced to develop and implement effective IT training
programs with fewer resources, they are exploring innovative approaches to
training using both existing and emerging practices.
The following practices, grouped into management processes, may help
federal agencies improve their IT training programs.
2 National Academy of Public Administration, The Transforming Power of
Information Technology
(August 2001). 9
Private- Sector Practices
Align IT training with
Enlist executive- level champions (sponsorship) to ensure that training
strategies are incorporated into
business goals
corporate decisionmaking and aligned with business goals
Involve critical stakeholders, such as top management, business unit
managers, subject matter experts, human capital staff, and end users, in
planning IT training
Address future skill needs and new technologies as part of the planning
process
Identify and assess IT
Identify and document competencies/ skills required for each job
description
training needs
Maintain a current inventory of skills
Address overall career development issues as well as skill- specific
training issues
Perform a gap analysis to determine where training is needed
Use self- directed tools, such as individual development plans, to give
employees responsibility in assessing their development needs
Use a single portal to give staff and managers access to training and
career development information
Allocate IT training
Ensure that an investment process is in place to select and manage
training projects
resources
Consider the benefits and costs associated with various training design
and delivery methods* e. g., Internet- based as opposed to classroom
training
Identify people who have high potential and provide them specialized
training opportunities
Ensure that resources are allocated for management training* e. g.,
leadership and project management
Design and deliver IT
Provide IT trainees with the flexibility to choose among different IT
training delivery methods
training
Ensure that on- the- job training is planned and monitored as part of the
training process
Consider combining different teaching methods (for example, Web- based and
instructor- led) within the same course
Provide just- in- time training
Consider outsourcing training solutions* e. g., university partnerships
and external IT training and content providers
Build courses using reusable components
Evaluate/ demonstrate
Collect information on how job performance is affected by training
the value of IT
Validate IT content learning by testing and certification of specific
skills* e. g., Java or C ++
training Assess evaluation results in terms of business impact 10
Aligning IT Training with Business Goals
By linking IT training programs to their overall business strategy/ goals,
companies can promote staff development that can best achieve corporate
objectives.
While the majority of the companies we visited are performing one or more
of the key practices associated with aligning IT training to the overall
goals of the company, only two have developed comprehensive planning
processes to achieve this alignment.
11
Aligning IT Training with Goals
Practices
Enlist executive- level champions (sponsorship) to ensure that training
strategies are incorporated into corporate decisionmaking and aligned with
business goals
Involve critical stakeholders, such as top management, business unit
managers, subject matter experts, human capital staff, and end users, in
planning IT training
Address future skill needs and new technologies as part of the planning
process
12
Aligning IT Training with Goals Case Study 1
A process was established to link learning strategy to overall business
direction.
Practice Illustrated
Involve critical stakeholders
Background
Between 2001 and 2002, this large telecom company reorganized its training
program. The company believed it could no longer meet *new world
challenges using the old world model.* It did not have a process linking a
well- planned and defined training program to corporate business
strategies.
Challenge
To create a process that links training to an overall business strategy to
achieve business objectives.
13
Aligning IT Training with Goals Case Study 1
Solution
According to a senior training official, the company established the
*Governance Process Flow* model, which is intended to involve all levels
of the company in developing a training strategy linked to the business
strategy. To ensure that all stakeholders are included, corporate
strategic information is communicated throughout the company. The
stakeholders include the following:
Learning Board (senior executives and one training organization
representative) Learning Council (executives and two training
organization representatives) Learning Services (training organization)
Business Lines (executives, managers, and directors)
Reported Results
Training is better linked to the overall business strategy. Major business
units and the training unit are now working together more effectively
because they understand how each unit is making a contribution to
achieving business goals.
14
Aligning IT Training with Goals Case Study 1
Governance Process Flow Model Strategy is Learning
Links strategy to business direction
communicated Board
Approves learning strategy, funds budget
from top to
Evaluates metrics, validates plan
bottom
Final authority on competing priorities
Learning
Facilitates common threads
Council
Evaluates metrics, validates plan Oversees appeals process Validates
budget
Learning
Defines learning strategy
Services
Analyzes business needs/ demand Develops plan and metrics
Establishes budget
Business
Identify business needs/ demand
Lines
Support business impact evaluation Metrics are Issues needing
reported up to resolution are
all levels referred
upward
Source: Company- provided graphic, used with permission.
15
Aligning IT Training with Goals Case Study 2
Strategic Learning Plan process helps align learning programs with
business priorities.
Practices Illustrated
Enlist executive- level champions Involve critical stakeholders
Address future skill needs and new
technologies as part of the planning process
Background
As a multinational corporation with a complex organizational structure,
this technology company needed to create more cohesion between learning
strategies and the strategic planning process.
Challenge
To develop a corporate learning approach that aligns learning priorities
with business priorities.
16
Aligning IT Training with Goals Case Study 2
Solution
The company established a process that develops a strategic learning plan
that aligns learning programs with business priorities. The process begins
with determining business priorities, and then identifying skill needs,
potential learning interventions to address gaps, and predicted business
results. Part of this process, the strategic learning framework, includes
validating plans against strategic business priorities. Top management is
involved at key points to ensure alignment with strategic direction.
Reported Result
The process enables the company to develop and implement training that is
linked to its strategic business objectives and delivers measurable
results.
17
Aligning IT Training with Goals Case Study 2
Strategic Learning Plan Process
Strategic Learning Strategic Learning
Business Results Framework
Plan Business Gaps and
Validate business Learning Interventions A
priorities issues
priorities Learning Interventions B
Build market
Development
Learning Interventions C
share needs
Understand business
Expected business
Drive
Barriers to
issues and objectives
results innovative
achieving
Learning objectives technology
goals
Identify learning
Business metrics development
interventions
Measurement plan
Investment requirement
Prioritize learning interventions
Source: Company- provided graphic, used with permission.
18
Aligning IT Training with Goals Case Study 2
Detailed Steps in Strategic Learning Framework Step 1
Step 2 Step 3
Step 4 Validate Business
Understand Business Identify Learning
Prioritize Learning Priorities
Issues and Objectives Interventions
Interventions
Review business plans Identify specific
Brainstorm learning Review list of proposed
Identify critical priorities business issues and
intervention alternatives interventions
and cause and effect links objectives
Select high potential Apply prioritized
Activities
Identify key measures Select critical issues to
interventions process
Validate with unit/ be addressed by
Validate with business organization leadership
learning unit leadership
Perform gap analysis Strategy map highlighting
Critical business issues Learning interventions Finalized strategic
Deliverables
business priorities Gaps potentially to be
learning plan Selected business
addressed by learning Measurement plan
measures Corporate Learning
Corporate Learning Corporate Learning
Corporate Learning
Roles
Executive Sponsor Learning Contact
Learning Contact Executive Sponsor
Learning Contact Subject Matter Experts
Subject Matter Experts Subject Matter Experts
Subject Matter Experts
Critical stakeholders involved at all stages
Involvement by executive sponsor in steps 1 and 4 Plans are validated and
revalidated
helps ensure alignment with strategic goals against strategic objectives
Source: Company- provided graphic, used with permission.
19
Identifying and Assessing IT Training Needs
A company*s skill needs may change because of new initiatives, new
technology, market forces, workforce attrition, or mergers and
acquisitions, creating a need for training. Companies should assess
employees* competencies and identify gaps between skills that employees
need and those they have. Gathering information from various levels of the
company allows managers to better assess training needs. In addition,
industry experts and practitioners are beginning to recognize that IT
training should focus on broader career development needs as well as
skillspecific training needs.
We found that, rather than simply fulfilling training requests, nearly all
(10) of the companies are performing practices to more effectively
identify and assess training needs. Companies are also beginning to
address long- term career development issues.
20
Identifying and Assessing Needs
Practices
Identify and document the competencies/ skills required for each job
description
Maintain a current inventory of skills Address overall career
development issues as well as skill- specific training
issues Perform a gap analysis to determine where training is needed
Use self- directed tools, such as individual development plans, to give
employees responsibility in assessing their development needs Use a
single portal 3 to give staff and managers access to training and career
development information
3 A site featuring a suite of commonly used services, serving as a
starting point and frequent gateway to the Web (Web portal).
21
Identifying and Assessing Needs Case Study 3
Senior management input, technology councils, and individual development
plans are used to identify and assess workforce needs.
Practices Illustrated
Identify and document skills Use self- directed tools
Background
This multinational technology leader with a technically diverse workforce
places a high value on learning. The company has engineering and
information technology professionals in manufacturing and service
positions worldwide with a very diverse set of knowledge requirements.
Challenge
To identify the training that satisfies the needs of this technically
diverse and geographically dispersed workforce and prioritize the courses
in greatest demand.
22
Identifying and Assessing Needs Case Study 3
Solution
The company*s training organization uses the following resources to
effectively identify and assess training gaps and needs (also see slide on
next page):
Senior management identifies corporatewide topics such as quality
management that require training support.
Technology councils representing technical disciplines throughout the
company identify and assess training curricula for different IT and
engineering jobs.
Individual development plans (IDPs) are self- directed tools completed
by each employee and supervisor to identify needed training. The IDP
information is compiled by the training organization through the use of a
learning management system to identify, prioritize, and schedule training
companywide.
Reported Results
The company*s ability to identify and prioritize training needs is
enhanced because it is based on input from management, subject- matter
experts, and individuals.
Skills needed for each job position are more comprehensively defined based
on input from technical experts through the organization.
Individual development plans allow employees to better identify and manage
their own training needs and establish commitment from both employees and
managers to fulfill those needs.
23
Identifying and Assessing Needs Case Study 3
Sources Used to Identify and Assess IT Training Needs Source: GAO. Note:
Analysis of company- provided data.
24
Identifying and Assessing Needs Case Study 4
IT learning tracks help employees assess their development needs.
Practices Illustrated
Identify and document skills Use self- directed tools Use a single
portal
Background
This technology company is using the Internet to transform how it conducts
business. The company has thousands of technical specialists located all
over the world, who need to stay technically and professionally current on
whole categories of technology and new Internet- based products and
services.
Challenge
To provide readily accessible career and functional training information
to a diverse and global technical workforce that can use this information
to identify and assess their technical training and development needs.
25
Identifying and Assessing Needs Case Study 4
Solution
To establish IT learning tracks, the company took the following steps:
Developed a Web- based single point of entry system that allows employees
located
worldwide to access technical training and career development information.
Established an advisory council that defined the required skills for
each technical
position. The council is made up of subject matter experts and
representatives from the technical management and training groups.
Created curricula on line for specific technical positions. The slide on
the next page is an example of a screen from the company*s training
portal. It shows the curriculum for IT Engineer* Java. Each curriculum is
divided into core courses, elective courses, and informal learning
sources.
Designed comprehensive guidance to help employees manage their career
paths and enhance their professional development.
Reported Results
Workers* ready access to development information allows them to control
their own development and career paths.
Employee loyalty is enhanced by the availability of opportunities for
employees to direct their own learning pace and development.
26
Identifying and Assessing Needs Case Study 4
Position Core
Informal company
learning courses
sources Courses related to job function
Includes certifi- cations
Source: Company- provided graphic, used with permission.
27
Identifying and Assessing Needs Case Study 5
IT Professional Program guides career development. Practice Illustrated
Address overall career development issues as well as skillspecific
training issues
Background
The IT department of a manufacturing company found that it needed well-
rounded IT staff with the skills that allowed them to be effective in the
business environment, not just in technical areas.
Challenge
To develop balanced, well- rounded IT professionals rather than narrowly
focused technicians.
28
Identifying and Assessing Needs Case Study 5
Solution
The company developed the *IT Professional Program,* which creates a
career ladder to a technical position at the senior executive level. The
program is used to guide and evaluate the career development of IT
professionals. Competencies are identified in six categories, only one of
which is technical:
Leadership (e. g., facilitation capabilities, persuasiveness, developing
others) Innovation (e. g., strategic thinking, applying new skills)
Effectiveness (e. g., teamwork, customer consciousness, prudent risk
taking) Communications (e. g., written and personal communications,
negotiation) Business processes (e. g., corporate- specific processes,
IT division processes) Technical and business domain (e. g., job-
specific technical skills)
Employees advance based on their proficiency in the competencies. Because
the objective is to develop balanced, well- rounded staff proficient in
nontechnical and technical skills, all six evaluation categories are
weighted equally. Salary determination is based on the lowest- scoring
category.
29
Identifying and Assessing Needs Case Study 5
Solution ( cont d)
The company provides formal training through a corporate university, on-
line courses, and courses at local colleges, but it also uses on- the- job
training since company staff believe that formal training provides only a
small part of what an employee needs to know to perform effectively.
Reported Result
The IT Professional Program has helped develop well- rounded IT staff who
understand the business and can work well with other staff and business
units. Other benefits are improved retention and morale, because career
ladder steps are well defined, employees understand what they need to do
to get promoted, and there is a full career ladder up to a senior level.
30
Allocating IT Training Resources
As in any other business area, training must compete for corporate
resources. It is important that companies prioritize and manage resources
to ensure that training projects are effectively identified and
implemented.
The leading companies we visited use various approaches to deciding how to
allocate training resources. Four of these companies are establishing
structured investment processes to select and manage training projects.
Four others target large amounts of training resources at a fairly small
group of promising applicants.
31
Allocating IT Training Resources
Practices
Ensure that an investment process is in place to select and manage
training projects
Consider the benefits and costs associated with various training design
and delivery methods* e. g., Internet- based as opposed to classroom
training
Identify people who have high potential and provide them specialized
training opportunities
Ensure that resources are allocated for management training* e. g.,
leadership and project management
32
Allocating IT Training Resources Case Study 6
An integrated investment management process is used to select and manage
training projects.
Practices Illustrated
Ensure that an investment process is in place Consider the benefits
and costs associated with various training design and delivery methods
Background
During the mid- 1990s, because funding for training at this large
technology company was curtailed, the company reassessed its training
processes. Funding and investment decisions for training were done by
various business groups on an ad hoc basis. Little thought was given to
how new training initiatives impacted the company*s strategic goals.
Further, few processes were in place to ensure that the selection and
implementation of the most strategic training projects were being managed
properly.
Challenge
To institute an integrated training investment management process that
involves both the business unit and corporate levels.
Solution
Executive stakeholders from the major business groups reallocated 10% of
their training budgets to a companywide strategic training portfolio.
Key organizational decisionmakers became involved in the investment
management process. 33
Allocating IT Training Resources Case Study 6
Solution ( cont d)
The company instituted an investment management process (see next slide)
for all major training development projects that
clearly defines responsibilities and authority for each stakeholder;
includes a ranking process using weighted criteria to compare and rank
projects as part
of investment reviews (these criteria include anticipated demand for the
investment from internal sources; potential for increased revenue; risk of
unfavorable consequences if investment is not made; availability of
resources to support the project; and business value or strategic fit);
and
requires that all training projects are monitored from initial design
through implementation.
Reported Results
Investment decisionmaking at the corporate level (through an Executive
Review Board and Integrated Portfolio Management Teams) is based on how
training investments support corporate business goals and results.
Project development teams can better manage risks and resources associated
with each training initiative.
34
Allocating IT Training Resources Case Study 6
Source: GAO. Note: Analysis of company- provided data.
35
Allocating IT Training Resources Case Study 7
A training program was established to target carefully screened
applicants.
Practice Illustrated
Provide specialized training for people with high potential
Background
This major finance company is highly dependent on technology in conducting
its business, which includes selling software products to financial
institutions and developing unique systems. It has historically found it
difficult to hire enough qualified technical employees, especially during
the tight job market of the 1990s, and it needed a method of addressing
skills shortages.
Challenge
To institute a program that identifies promising nontechnical candidates
and rapidly trains them to become proficient in technical areas.
36
Allocating IT Training Resources Case Study 7
Solution
To fill staff shortages or anticipated staff shortages in technology
positions, the company established the Business Technologist Program. This
intensive 3- year training program is divided into individual programs
that focus on particular target audiences or job positions. For example,
one program focuses on carefully selected nontechnical new hires with
prior business experience.
A large component of the program cost is the salaries. Participants are
paid during 18 weeks of training before they are placed in productive
jobs. Training facility costs* such as workstations and networks* can also
be high. The company selects applicants very carefully before investing in
their training and development, and protects its investment by careful
monitoring.
During the classroom training, recruits are monitored by frequent testing
and helped if they seem to be falling behind. After being placed in jobs,
they are assigned mentors, and performance is evaluated at half- year
intervals instead of annually as with other staff.
37
Allocating IT Training Resources Case Study 7
Solution ( cont d)
The company keeps costs down by paying recruits approximately 50 percent
of the market rate for a seasoned IT professional during their training
period. This is usually a pay reduction for people entering the program.
Salaries can be increased after each semi- annual performance review and
rise to market levels during the 3 years of the program.
Reported Results
The company stated that the program is effective because of the number and
quality of its graduates* more than 500 people over the course of the
program. It has been able to quickly train staff to fill technical
positions, and retention of program participants has been better than
average. An additional benefit has been increased diversity in the
workforce.
38
Allocating IT Training Resources Case Study 8
Company targets resources to develop a cadre of skilled project managers.
Practices Illustrated
Identify people who have high potential and provide them with
specialized training Ensure that resources are allocated for management
training* e. g., leadership
and project management
Background
This large financial services company had a number of *painful costly
project experiences caused by a lack of project management knowledge and
experience.* Project managers from IT and from the various lines of
business tended to focus more on their respective needs rather than the
overall needs of the organization. The company lacked an internal network
of skilled, qualified project managers.
Challenge
To develop a cadre/ network of information technology managers (and
business managers) with the proper project management skills, internal
professional contacts, and management background to deliver projects
within cost and on schedule.
39
Allocating IT Training Resources Case Study 8
Solution
The company*s IT training group designed a comprehensive project
management development program (PMDP) for IT managers and others in
leadership roles. The program includes two separate 4- month training
programs: PMDP Level 1 for new project managers and PMDP Level 2 for
experienced project managers. The company took the following steps to
support the success of training:
obtained a senior executive sponsor; assembled an internal consulting
team possessing a broad view of the
company*s projects, and having management and training experience;
identified the causes for project failures within the company and used the
information to help develop the program; created the program around six
broad categories of required skills (project
planning, project control and management, managing the team, managing the
environment, risk management, and the company*s project life cycle);
40
Allocating IT Training Resources Case Study 8
Solution ( cont d)
identified top performers and ensured an optimal mix of participants
that broadly represented the company (business and IT managers);
created a program that allowed trainees to network and be mentored; and
developed evaluation tools to assess the PMDP.
Reported Result
The program reduced the number of project difficulties caused by
inadequate project management knowledge and skills.
It expanded the internal network of skilled, qualified IT and business
project managers who provide a ready source of expertise.
It also facilitated the exchange of approaches and practices that help
ensure successful project management.
41
Designing and Delivering IT Training
Once training needs are identified and resources allocated, it is
important that the training design and delivery process ensures that
learning occurs during the training and also ensures that the employee
applies the training on the job.
The majority of the companies we studied said that they are still using
traditional instructor- led training. However, they are taking advantage
of more flexible design and delivery methods made possible by technology
to deliver training to the user*s desktop and to make training more
convenient.
42
Designing and Delivering Training
Practices
Provide IT trainees with the flexibility to choose among different IT
training delivery methods
Ensure that on- the- job training is planned and monitored as part of
the training process
Consider combining different teaching methods (e. g., Web- based and
instructor- led) within the same course
Provide just- in- time training Consider outsourcing training
solutions* e. g., university partnerships and
external IT training and content providers Build courses using reusable
components
43
Designing and Delivering Training Case Study 9
An e- learning solution offers flexibility in learning to a worldwide
workforce. Practices Illustrated
Provide flexibility to choose among different delivery methods
Consider combining different teaching methods Provide just- in- time
training Build courses using reusable components
Background
This technology company believes that e- learning is a revolutionary way
to empower a workforce with the skills and knowledge it needs. The company
implemented e- learning solutions internally so it can gain advantages and
also act as an e- learning model for its customers.
Challenge
To create an environment where learning resources and information are
clearly integrated into everyday job functions and readily accessible to
employees worldwide.
44
Designing and Delivering Training Case Study 9
Solution
To implement e- learning solutions, the company took the following steps:
Established e- learning portals on the company*s network that are
integrated with its ebusiness strategy. (These portals are illustrated on
the following page.)
Implemented a central e- learning system (see graphic on next slide),
accessible from the portals, that provides employees with Web- based
learning solutions, including video- ondemand (VOD)* which captures expert
instructors on video* computer- based training, an electronic library,
virtual labs, and net meetings. These delivery methods may be combined in
a single training course.
Invested in infrastructure that supports the company*s content delivery
(VOD requires high bandwidth).
Invested in a state- of- the- art video studio for its VOD learning
solutions. Established a reusable object strategy that enables the
training workforce to create and
deliver modular, reusable learning *chunks* for different learning
experiences.
Reported Results
The company increased employee productivity and communications and
decreased costs. 45
Designing and Delivering Training Case Study 9
E- Learning/ Internet Learning Environment
Source: Company- provided graphic, used with permission.
46
Designing and Delivering Training Case Study 10
Reusable learning component strategy offers flexible approach to improve
course design/ delivery productivity and cost savings.
Practices Illustrated
Provide just- in- time training Build courses using reusable
components
Background
Career certification courses at this technology company were previously
delivered almost exclusively in an instructor- led format. In 2000, to
fulfill its goal of becoming a leader in e- learning, the company devised
a strategy to develop reusable components for its course offerings. These
components would allow courses to be packaged and delivered in various
ways, including traditional classroom training, live virtual training, or
self- paced e- learning.
Challenge
To move the company*s training courses from large inflexible courses to
searchable database- driven objects* reusable chunks of information* that
could be reused and modified independent of their delivery method.
47
Designing and Delivering Training Case Study 10
Solution
A strategy was developed that included the following tasks: Define
standards and guidelines for designing and
developing reusable learning components. Curriculum
Establish a hierarchical curriculum structure, as at Unit
right. Create reusable learning objects (* lessons*) by
Module combining individual reusable information objects.
(Reusable Lesson
learning Create reusable information objects (* topics* or
object) *pages*) by combining content items, practice items,
Topic (Reusable
and assessment items. information
object) The implementation strategy began with a limited number
Source: GAO. Note: Analysis of company- provided data.
of certification courses to give developers time to embrace the new
approach. The company now has 20,000 reusable learning objects that can be
reused for course development.
48
Designing and Delivering Training Case Study 10
Reported Results
Course development time was significantly reduced* i. e., selected courses
were developed in half the time/ budget because learning objects were
reused.
The company achieved a positive return on its investment. The approach
enabled all field staff to access on- demand, personalized training, in
the media of their choice.
Users have the ability to offer an assessment of the objects people need
to achieve the desired performance.
49
Evaluating/ Demonstrating the Value of IT Training
Companies can determine whether training has achieved its intended
business impact by integrating training measurements into their strategy
and operations. Increasingly, stakeholders who fund training are
interested in how training investments contribute to the company*s
business results.
Companies we visited recognize the importance of effectively evaluating
the impact of training on business goals. All the companies are going
beyond simply obtaining participant reactions. For example, some use tests
or certifications to validate that the content was understood; others are
deploying evaluation processes that measure business results.
50
Evaluating/ Demonstrating Value
Practices
Collect information on how job performance is affected by training
Validate IT content learning by testing and certification of specific
skills* e. g.,
Java or C ++ Assess evaluation results in terms of business impact
51
Evaluating/ Demonstrating Value Case Study 11
An automated evaluation process measures and reports training results.
Practices Illustrated
Collect information on how job performance is affected Validate
content learning by testing and certification
Background
This large financial services firm needed to improve the communication
between its IT training organization, called Learning Systems, and its IT
organization.
Challenge
To demonstrate the value of the training process to IT organization
managers and facilitate their involvement in the training process.
Solutions
An automated evaluation process has been implemented to demonstrate the
value of training through participant evaluations, automated surveys of
how well training is applied on the job, and to some extent, Web- based
testing and certifications.
52
Evaluating/ Demonstrating Value Case Study 11
Solutions ( cont d)
Specifically, the automated evaluation process is used for the following:
Obtaining participant reaction:
The Learning Systems computer system automatically generates an
evaluation form that participants must complete before leaving class.
The data are compiled and analyzed by a proprietary database analysis tool
and used to determine changes in instructors, course design, and other
training factors.
Assessing content learning: Some courses use Web- based testing to
demonstrate proficiency. Many related career certifications also require
testing to demonstrate
proficiency. Determining how training is applied on the job:
Each month, the training organization selects courses for further
assessment. Participants receive an electronic survey about 1 month
after course
completion, in which they indicate how they applied the training on the
job. The participant*s manager receives a similar survey about the
participant. The information is then used to refine training activities.
53
Evaluating/ Demonstrating Value Case Study 11
Solutions ( cont d)
The Learning Systems director meets with IT organization business managers
each month to discuss how learning strategies can best help the IT
organization achieve its business objectives. These discussions include
budget, hours of training delivered, reactions to training, and types of
delivery methods.
Reported Results
Participant evaluations, testing, and post- training feedback help assess
the value of training.
Collecting evaluation data after the employee has returned to the work
environment helps determine if training was applied on the job.
The monthly discussions help improve communication between the IT
organization and Learning Systems.
54
Evaluating/ Demonstrating Value Case Study 12
Certifications are used to measure learned skills. Practice Illustrated
Validate content learning by testing and certification
Background
This telecom company created a new group to be responsible for the
corporate Web site and for sites used by customers to communicate with the
company. The vice president in charge worked with the chief technologist
to develop a list of skills, matching needed skills to job categories.
Once the group was up and running, the skills list was refined by
consultations with managers and staff.
Challenge
To measure progress in developing the skills needed to provide new and
improved Web services to clients.
55
Evaluating/ Demonstrating Value Case Study 12
Solution
The company contracted for Web- based certification testing on the
required skills, such as Java programming. Employees were required to pass
all the certifications needed for their jobs, but were allowed to fail and
retake the certification tests as many times as needed without penalties.
Incentives and rewards, such as public recognition or lunch with the
senior vice president, were offered to motivate staff. Training relevant
to the certifications was provided by about 400 on- line courses, as well
as through books recommended by the certification vendor and by informal,
on- the- job training.
The senior vice president in charge of the group tracked the group*s
progress on certifications against performance metrics, including time to
complete projects and number of pages of program code rejected by quality
control. Improvements in the code reject rate were seen after employees
passed certification tests.
56
Evaluating/ Demonstrating Value Case Study 12
Reported Results
By linking training and certification, the company could directly measure
employee content learning and also validate training courses.
By monitoring individuals* performance using the metric of pages of code
rejected, the company was able to demonstrate improved job performance
correlated with employees* becoming certified.
The certification process and how it was administered led to increased
employee motivation to take training and improved morale, since employees
valued the certifications.
57
Issues Critical to Effective IT Training The case studies illustrate that
practices identified and performed by leading private sector companies can
result in more effective training management. However, based on company
and expert input, we noted several crosscutting issues that should be
considered in implementing effective IT training practices. These issues
include
Responding to the rapid pace of technological and social change . Rapid
changes in technology (e. g., the growth of e- learning) affect how
training is delivered as well as what competencies are needed (e. g., the
growing need for information security skills). Changes in the IT
workforce, such as increased diversity, may require a different mix of
skills.
Demonstrating return on investment in IT training . Although there is
pressure to show return on investment (ROI) from training, it is difficult
and costly to demonstrate ROI. Only two companies reported that they were
actually calculating ROI on IT training.
Managing funding constraints during an economic downturn . The economic
climate and its impact on training expenditures may affect the overall
level of training delivered by organizations or the mix of delivery
methods* such as outsourcing or e- learning.
58
Issues Critical to Effective IT Training Obtaining visibility for
training at the highest levels of the organization.
Several of our companies have existing or emerging processes that include
executive management involvement. This can enhance an organization*s
ability to incorporate training in its planning, and can improve a
training organization*s ability to undertake new initiatives.
59
A Final Note GAO*s review of private sector practices indicates that
training is not simply a support function, but a strategic element in
achieving corporate objectives. Traditional training activities are being
reinvented to focus on implementing a comprehensive learning strategy and
demonstrating business impact. Companies are adopting new ideas about
training, but many initiatives are in their early stages, and private
sector officials expressed interest in learning about innovative practices
emerging from the public sector.
60
(310444)
GAO*s Mission The General Accounting Office, the investigative arm of
Congress, exists to support Congress in meeting its constitutional
responsibilities and to help improve
the performance and accountability of the federal government for the
American people. GAO examines the use of public funds; evaluates federal
programs and policies; and provides analyses, recommendations, and other
assistance to help Congress make informed oversight, policy, and funding
decisions. GAO*s commitment to good government is reflected in its core
values of accountability, integrity, and reliability.
Obtaining Copies of The fastest and easiest way to obtain copies of GAO
documents at no cost is
through the Internet. GAO*s Web site (www. gao. gov) contains abstracts
and fulltext GAO Reports and
files of current reports and testimony and an expanding archive of older
Testimony
products. The Web site features a search engine to help you locate
documents using key words and phrases. You can print these documents in
their entirety, including charts and other graphics.
Each day, GAO issues a list of newly released reports, testimony, and
correspondence. GAO posts this list, known as *Today*s Reports,* on its
Web site daily. The list contains links to the full- text document files.
To have GAO e- mail this
list to you every afternoon, go to www. gao. gov and select *Subscribe to
GAO Mailing Lists* under *Order GAO Products* heading.
Order by Mail or Phone The first copy of each printed report is free.
Additional copies are $2 each. A check or money order should be made out
to the Superintendent of Documents. GAO
also accepts VISA and Mastercard. Orders for 100 or more copies mailed to
a single address are discounted 25 percent. Orders should be sent to:
U. S. General Accounting Office 441 G Street NW, Room LM Washington, D. C.
20548
To order by Phone: Voice: (202) 512- 6000 TDD: (202) 512- 2537 Fax: (202)
512- 6061
To Report Fraud, Contact:
Waste, and Abuse in Web site: www. gao. gov/ fraudnet/ fraudnet. htm
E- mail: fraudnet@ gao. gov Federal Programs
Automated answering system: (800) 424- 5454 or (202) 512- 7470 Public
Affairs Jeff Nelligan, Managing Director, NelliganJ@ gao. gov (202) 512-
4800
U. S. General Accounting Office, 441 G Street NW, Room 7149 Washington, D.
C. 20548
Report to Congressional Requesters
January 2003 INFORMATION TECHNOLOGY TRAINING
Practices of Leading Private- Sector Companies
GAO- 03- 390
Letter 1 Appendix
Appendix I: Information Technology Training 4
This is a work of the U. S. Government and is not subject to copyright
protection in the United States. It may be reproduced and distributed in
its entirety without further permission from GAO. It may contain
copyrighted graphics, images or other materials. Permission from the
copyright holder may be necessary should you wish to reproduce copyrighted
materials separately from GAO*s product.
a
GAO United States General Accounting Office
GAO identified 22 existing and emerging training practices used by leading
companies to implement effective IT training. We organized these practices
and accompanying case studies under five training management processes
that we defined based on input from industry experts, published research,
and previous GAO work (see table below). Although none of the companies
was performing all the practices, the majority performed 10 or more.
Organizations and experts agree that these practices could result in more
effective training management, but in applying the identified practices,
we noted several critical issues (e. g., funding constraints and
demonstrating
return on investment) that should be considered. The practices may also
suggest approaches to IT training for government agencies to consider.
GAO*s discussions with leading private sector companies indicate that
training is not simply a support function, but a strategic element in
achieving corporate objectives. Further, although companies are adopting
new ideas about training, many initiatives are in their early stages, and
private sector officials expressed interest in learning about innovative
practices emerging from the public sector.
IT Training Management Processes and Sample Practices (see app. I for full
list) Management processes Sample practices Enlist executive- level
champions Align IT training with
business goals Involve critical stakeholders Document competencies/ skills
required for each job description Identify and assess IT training needs
Perform a gap analysis to determine needed training
Use an investment process to select and manage training projects Allocate
IT training resources Provide resources for management training, e. g.,
leadership and
project management Give trainees choices among different training delivery
methods Design and deliver IT training Build courses using reusable
components
Collect information on how job performance is affected by training
Evaluate/ demonstrate the value of IT training Assess evaluation results
in terms of business impact
Source: GAO. Note: Analysis of company- provided information. INFORMATION
TECHNOLOGY TRAINING
Practices of Leading Private- Sector Companies www. gao. gov/ cgi- bin/
getrpt? GAO- 03- 390. To view the full report, including the scope
and methodology, click on the link above. For more information, contact
Joel Willemssen at (202) 512- 6253 or willemssenj@ gao. gov. Highlights of
GAO- 03- 390, a report to the
Chairman, Tom Davis, Committee on Government Reform and Representative Jim
Turner, House of Representatives
January 2003
The rapid pace of technological change, with its potential to transform
the way the government
delivers services, makes information technology (IT) human capital a
critical issue for federal agencies.
GAO has identified strategic human capital management as a high risk area
for the federal government, and the demand for skilled IT workers is
expected to increase
over the long term. Given that competition for workers affects the federal
government as it does any
other employer, effective training of staff is essential to developing and
retaining a qualified workforce. Some private- sector companies are
recognized for their effective and innovative training programs for the IT
workforce, which could provide models and examples for federal agencies.
To help federal agencies better design and
implement such training programs, GAO was asked to examine privatesector
practices for training both IT and non- IT professionals (e. g., business
managers and other staff
needing training in IT) that could be used as a basis for addressing
federal efforts.
Page i GAO- 03- 390 Information Technology Training
Contents
Page 1 GAO- 03- 390 Information Technology Training United States General
Accounting Office
Washington, D. C. 20548 Page 1 GAO- 03- 390 Information Technology
Training
A
Page 2 GAO- 03- 390 Information Technology Training
Page 3 GAO- 03- 390 Information Technology Training
Page 4 GAO- 03- 390 Information Technology Training
Appendix I
Appendix I Information Technology Training
Page 5 GAO- 03- 390 Information Technology Training
Appendix I Information Technology Training
Page 6 GAO- 03- 390 Information Technology Training
Appendix I Information Technology Training
Page 7 GAO- 03- 390 Information Technology Training
Appendix I Information Technology Training
Page 8 GAO- 03- 390 Information Technology Training
Appendix I Information Technology Training
Page 9 GAO- 03- 390 Information Technology Training
Appendix I Information Technology Training
Page 10 GAO- 03- 390 Information Technology Training
Appendix I Information Technology Training
Page 11 GAO- 03- 390 Information Technology Training
Appendix I Information Technology Training
Page 12 GAO- 03- 390 Information Technology Training
Appendix I Information Technology Training
Page 13 GAO- 03- 390 Information Technology Training
Appendix I Information Technology Training
Page 14 GAO- 03- 390 Information Technology Training
Appendix I Information Technology Training
Page 15 GAO- 03- 390 Information Technology Training
Appendix I Information Technology Training
Page 16 GAO- 03- 390 Information Technology Training
Appendix I Information Technology Training
Page 17 GAO- 03- 390 Information Technology Training
Appendix I Information Technology Training
Page 18 GAO- 03- 390 Information Technology Training
Appendix I Information Technology Training
Page 19 GAO- 03- 390 Information Technology Training
Appendix I Information Technology Training
Page 20 GAO- 03- 390 Information Technology Training
Appendix I Information Technology Training
Page 21 GAO- 03- 390 Information Technology Training
Appendix I Information Technology Training
Page 22 GAO- 03- 390 Information Technology Training
Appendix I Information Technology Training
Page 23 GAO- 03- 390 Information Technology Training
Appendix I Information Technology Training
Page 24 GAO- 03- 390 Information Technology Training
Appendix I Information Technology Training
Page 25 GAO- 03- 390 Information Technology Training
Appendix I Information Technology Training
Page 26 GAO- 03- 390 Information Technology Training
Appendix I Information Technology Training
Page 27 GAO- 03- 390 Information Technology Training
Appendix I Information Technology Training
Page 28 GAO- 03- 390 Information Technology Training
Appendix I Information Technology Training
Page 29 GAO- 03- 390 Information Technology Training
Appendix I Information Technology Training
Page 30 GAO- 03- 390 Information Technology Training
Appendix I Information Technology Training
Page 31 GAO- 03- 390 Information Technology Training
Appendix I Information Technology Training
Page 32 GAO- 03- 390 Information Technology Training
Appendix I Information Technology Training
Page 33 GAO- 03- 390 Information Technology Training
Appendix I Information Technology Training
Page 34 GAO- 03- 390 Information Technology Training
Appendix I Information Technology Training
Page 35 GAO- 03- 390 Information Technology Training
Appendix I Information Technology Training
Page 36 GAO- 03- 390 Information Technology Training
Appendix I Information Technology Training
Page 37 GAO- 03- 390 Information Technology Training
Appendix I Information Technology Training
Page 38 GAO- 03- 390 Information Technology Training
Appendix I Information Technology Training
Page 39 GAO- 03- 390 Information Technology Training
Appendix I Information Technology Training
Page 40 GAO- 03- 390 Information Technology Training
Appendix I Information Technology Training
Page 41 GAO- 03- 390 Information Technology Training
Appendix I Information Technology Training
Page 42 GAO- 03- 390 Information Technology Training
Appendix I Information Technology Training
Page 43 GAO- 03- 390 Information Technology Training
Appendix I Information Technology Training
Page 44 GAO- 03- 390 Information Technology Training
Appendix I Information Technology Training
Page 45 GAO- 03- 390 Information Technology Training
Appendix I Information Technology Training
Page 46 GAO- 03- 390 Information Technology Training
Appendix I Information Technology Training
Page 47 GAO- 03- 390 Information Technology Training
Appendix I Information Technology Training
Page 48 GAO- 03- 390 Information Technology Training
Appendix I Information Technology Training
Page 49 GAO- 03- 390 Information Technology Training
Appendix I Information Technology Training
Page 50 GAO- 03- 390 Information Technology Training
Appendix I Information Technology Training
Page 51 GAO- 03- 390 Information Technology Training
Appendix I Information Technology Training
Page 52 GAO- 03- 390 Information Technology Training
Appendix I Information Technology Training
Page 53 GAO- 03- 390 Information Technology Training
Appendix I Information Technology Training
Page 54 GAO- 03- 390 Information Technology Training
Appendix I Information Technology Training
Page 55 GAO- 03- 390 Information Technology Training
Appendix I Information Technology Training
Page 56 GAO- 03- 390 Information Technology Training
Appendix I Information Technology Training
Page 57 GAO- 03- 390 Information Technology Training
Appendix I Information Technology Training
Page 58 GAO- 03- 390 Information Technology Training
Appendix I Information Technology Training
Page 59 GAO- 03- 390 Information Technology Training
Appendix I Information Technology Training
Page 60 GAO- 03- 390 Information Technology Training
Appendix I Information Technology Training
Page 61 GAO- 03- 390 Information Technology Training
Appendix I Information Technology Training
Page 62 GAO- 03- 390 Information Technology Training
Appendix I Information Technology Training
Page 63 GAO- 03- 390 Information Technology Training
United States General Accounting Office Washington, D. C. 20548- 0001
Official Business Penalty for Private Use $300 Address Service Requested
Presorted Standard Postage & Fees Paid
GAO Permit No. GI00
*** End of document. ***