    Information Technology: Training Can Be Enhanced by Greater Use
    of Leading Practices (24-JUN-04, GAO-04-791).

    Effective training of information technology (IT) staff, as
    called for in the E-Government (E-Gov) Act of 2002, is essential
    to developing and retaining a qualified workforce. In an earlier
    report (GAO-03-390), we identified 22 leading practices, grouped
    into 5 key training management processes, used by private-sector
    companies to implement effective IT training. These practices
    suggest approaches that government agencies could consider. To
    assess IT training in the federal government, including its use
    of leading practices, we were asked to determine, among other
    things, to what extent federal agencies use our leading
    practices, the major obstacles in providing effective IT training
    and how agencies address them, and the progress the Office of
    Personnel Management (OPM) is making in issuing policies and
    performing evaluations to encourage agencies to provide effective
    IT training.
    -------------------------Indexing Terms-------------------------
    REPORTNUM:   GAO-04-791
        ACCNO:   A10637
      TITLE:     Information Technology: Training Can Be Enhanced by
    Greater Use of Leading Practices
         DATE:   06/24/2004
      SUBJECT:   Education or training
    	     Electronic government
    	     Federal employees
    	     Information technology
    	     Personnel management
    	     Productivity in government
    	     Training utilization
    	     Best practices
    	     Comparative benchmarking products

    ******************************************************************
    ** This file contains an ASCII representation of the text of a  **
    ** GAO Product.                                                 **
    **                                                              **
    ** No attempt has been made to display graphic images, although **
    ** figure captions are reproduced.  Tables are included, but    **
    ** may not resemble those in the printed version.               **
    **                                                              **
    ** Please see the PDF (Portable Document Format) file, when     **
    ** available, for a complete electronic file of the printed     **
    ** document's contents.                                         **
    **                                                              **
    ******************************************************************
    GAO-04-791

    United States General Accounting Office

                         GAO Report to Congressional Requesters

    June 2004

                                      INFORMATION
                                       TECHNOLOGY

             Training Can Be Enhanced by Greater Use of Leading Practices

                                           a

    GAO-04-791

    Highlights of GAO-04-791, a report to congressional requesters

    Effective training of information technology (IT) staff, as called for in
    the E-Government (E-Gov) Act of 2002, is essential to developing and
    retaining a qualified workforce. In an earlier report (GAO-03-390), we
    identified 22 leading practices, grouped into 5 key training management
    processes, used by private-sector companies to implement effective IT
    training. These practices suggest approaches that government agencies
    could consider.

    To assess IT training in the federal government, including its use of
    leading practices, we were asked to determine, among other things, to what
    extent federal agencies use our leading practices, the major obstacles in
    providing effective IT training and how agencies address them, and the
    progress the Office of Personnel Management (OPM) is making in issuing
    policies and performing evaluations to encourage agencies to provide
    effective IT training.

    GAO is making recommendations to OPM, including that it issue IT training
    policies and evaluate implementation of the training provisions of the
    E-Gov Act.

    In response to a draft of this report, OPM disagreed with the conclusions
    on its role and efforts with respect to federal IT training. Most other
    agencies generally agreed with the report or had no comment.

    June 2004

    INFORMATION TECHNOLOGY

    Training Can Be Enhanced by Greater Use of Leading Practices

    Although federal agencies differ widely in how much IT training they
    provide, their use of the 22 leading IT training practices that we
    identified was generally not extensive. Of these practices, only 5 were in
    use to a great or very great extent in a majority of agencies (see figure
    below). In particular, of the three practices we identified in the area of
    evaluating training, none was widely used. For example, for the practice
    of collecting information on how job performance is affected by training,
    only three agencies reported use to a great or very great extent.

    The most commonly cited obstacles to effective IT training were funding
    and the time training takes away from work. To address these obstacles,
    agencies are looking at ways to reduce training time and costs-for
    example, by greater use of e-learning.

    OPM has made limited progress in issuing policies or performing
    evaluations regarding IT training. The E-Gov Act sets requirements for
    agency IT training programs. To provide oversight, it requires OPM to
    issue policies to promote the development of performance standards for
    training, and to evaluate agency implementation of the act's IT training
    provisions. These policies are particularly important in view of the lack
    of extensive agency use of the evaluation practices we identified. OPM has
    begun drafting guidance, but it has not issued policies or evaluated
    agency implementation of the act. Until policies are issued and progress
    is measured, oversight of federal IT training will continue to fall short
    of what the act calls for.

    Percentage of Agencies Using Leading Practices Extensively

    www.gao.gov/cgi-bin/getrpt?GAO-04-791.

    To view the full product, including the scope and methodology, click on
    the link above. For more information, contact Dave Powner at (202)
    512-9286 or pownerd@gao.gov.

    Contents

                                        Letter 1

    Recommendations for Executive Action 3

    Agency Comments and Our Evaluation 3

    Appendixes

    Appendix I: Information Technology: Federal IT Training 7

    Appendix II: Comments from the Office of Personnel Management 70

    Appendix III: Comments from the Department of Veterans Affairs 72

    Appendix IV: 	Comments from the Department of Health and Human
    Services 74

    Abbreviations

    APPL Academy of Program and Project Leadership
    BLM NTC Bureau of Land Management National Training Center
    CIO Chief Information Officer
    DHS Department of Homeland Security
    E-Gov Act E-Government Act of 2002
    GoLearn Gov Online Learning Center
    GSA General Services Administration
    HHS Department of Health and Human Services
    IDP Individual Development Plan
    IRMC National Defense University Information Resources

    Management College IT Information Technology NASA National Aeronautics and Space Administration NIH National Institutes of Health OMB Office of Management and Budget OPM Office of Personnel Management PMDP Project Management Development Process STAR Strategic and Tactical Advocates for Results USDA U.S. Department of Agriculture VA Department of Veterans Affairs

    This is a work of the U.S. government and is not subject to copyright protection in the United States. It may be reproduced and distributed in its entirety without further permission from GAO. However, because this work may contain copyrighted images or other material, permission from the copyright holder may be necessary if you wish to reproduce this material separately.

    A

    United States General Accounting Office

    Washington, D.C. 20548

    June 24, 2004

    The Honorable Tom Davis Chairman Committee on Government Reform House of Representatives

    The Honorable Jim Turner House of Representatives

    The rapid pace of technological change, with its potential to transform the way the government delivers services, makes information technology (IT) human capital a critical issue for federal agencies.1 To ensure that the federal government meets its human capital needs, effective training of IT staff, as called for in the E-Government (E-Gov) Act of 2002,2 is essential to developing and retaining a qualified workforce. At your request, in an earlier report, we examined IT training practices of leading private-sector companies. We found 22 existing and emerging practices (see attachment 1 in app. I) that are used to implement effective IT training. The practices and associated case studies that we presented suggest approaches that government agencies could consider. In addition, we noted several critical issues (e.g., funding constraints and demonstrating return on investment) that should be considered in implementing these practices. More recently, we issued a final version of a guide for assessing strategic training efforts that provides a framework that federal agencies can use to ensure that their training investments are targeted strategically.3

    To assess IT training in the federal government including its use of leading practices, you asked us to determine (1) how federal agencies provide ITrelated training and to what extent they use the leading practices that we identified, (2) the major obstacles in providing effective IT training and how agencies address them, (3) the factors agencies should consider in preparing for future IT training, and (4) the progress the Office of

    1We have identified strategic human capital management as a high-risk area. See U.S. General Accounting Office, High
    Risk Series: Strategic Human Capital
    Management, GAO-03-120 (Washington, D.C.: January 2003).

    2Pub. L. 107-347 (Dec. 17, 2002).

    3U.S. General Accounting Office, Human Capital: A Guide for Assessing
    Strategic Training and Development Efforts in the Federal
    Government, GAO-04-546G (Washington, D.C.: March 2004).

    Personnel Management (OPM) is making in issuing policies and performing evaluations to encourage agencies to provide effective IT training. To accomplish these objectives, we conducted about 100 structured interviews with executives, training managers, and training recipients at 26 agencies and departments. We administered a survey to agencies and asked them to provide basic statistics and to self-assess the extent to which they were using the leading practices we had identified. We followed up with agencies to verify or correct data, but we did not independently verify survey responses. We researched existing reports, including previous reports by us and other agencies; reviewed related training legislation and guidance; and conducted interviews with experts on federal IT training. To evaluate OPM's progress in implementing provisions of the E-Government Act, we analyzed OPM's responses to written questions and held discussions with OPM training officials.

    Earlier this year, we provided briefing slides to your staff on the results of our study. These results, updated to include additional information requested, are included as an appendix to this letter. The purpose of this letter is to officially transmit the information to you.

    In brief, federal agencies provide IT training in a variety of ways. They use multiple delivery methods, acquire training from multiple sources, and differ widely in the amount of training they provide and the amount they spend. Agencies reported using leading IT training practices, but most were not used to a great or very great extent. Of the 22 practices we identified, only 5 were in use to a great or very great extent in a majority of agencies. Increased use of these practices can result in more effective training management. We also noted that several federal agencies and other entities have programs or initiatives to provide IT training or training resources to employees governmentwide.

    Agencies reported that the most common obstacles to effective training are funding and the time that training takes away from normal work hours. To address these obstacles, agencies are looking at ways to reduce the time and costs associated with training-for example, by increasing the use of elearning.

    Major factors in preparing for future IT training cited by agencies included rapidly changing technologies and the loss of skills and expertise as the aging workforce retires. Further, federal officials emphasized the need for training to be focused on areas such as contract management, project management, and information security.

    OPM has made limited progress in issuing policies or performing evaluations regarding IT training. The E-Gov Act requires agencies to have IT training programs that are developed and applied according to rigorous standards. To provide oversight, the act requires OPM, in coordination with the Office of Management and Budget (OMB), to issue policies to promote the development of performance standards for training, and to evaluate agency implementation of the act's IT training provisions. Such policies are particularly important in view of the lack of extensive agency use of the leading training evaluation practices that we described. In May, OPM told us that it plans to consider incorporating our leading practices in future guidance. OPM has begun drafting guidance and is in the final stages of issuing a report on governmentwide IT training, but it has not issued policies or evaluated agency implementation of the act. Until policies are established that promote the development of performance standards for training, and progress is measured against these standards, agency and OPM oversight of federal IT training programs will continue to fall short of the expectations established in the act with respect to standards.

      Recommendations for Executive Action

    Agency Comments and Our Evaluation

    We recommend that, as part of OPM's oversight responsibilities under the E-Gov Act, the Director of OPM

    o
       	issue governmentwide IT training policies, consistent with our strategic training guide, that promote development of performance standards for training and encourage agencies to use the leading practices that we identified; and

    o
       	set specific milestones for evaluating agency implementation of the provisions cited above.

    We sent a draft of this report to all 24 agencies4 that provided information and to OMB. We received written, oral, or e-mail comments from 9 agencies; 11 agencies responded that they did not have comments; 5 agencies did not respond.

    4Although we counted Department of Defense components separately in our data, Defense handled agency comments centrally as a single response.

    In a written response, OPM disagreed with our conclusions about its role and efforts with respect to federal IT training. OPM stated that it has not fallen short of the expectations established by the E-Gov Act, noting that the draft report focuses on only one of six statutory responsibilities under the act and that we fail to acknowledge that OPM is to work with the CIO Council and OMB in addressing federal IT personnel needs. Our report focuses on policies because we believe these are a foundation for exercising oversight of the other areas of responsibility mentioned. We cite OPM's plans to coordinate with OMB and the CIO Council on slide 37 (p. 43), and we note that OMB generally agreed with the findings and conclusions in our report. We adjusted our report to clarify that OPM oversight will fall short of the expectations of the E-Gov Act with respect to standards.

    With respect to issuing policies, OPM states that the report does not fully and fairly inform the reader as to the status of the policies. In its response, OPM refers to an unpublished report in final clearance that will recommend a governmentwide IT training framework and announces that it has developed milestones for issuing policies. We mention the unpublished report on slide 37, but it has not been provided to us, so we are unable to comment on it. We have modified our report to reflect OPM's announcement of milestones. Setting milestones is a positive step toward issuing policies.

    OPM also took exception to our report's statement that it has made little progress in evaluating agency implementation of the E-Gov Act provisions. The response states that OPM will incorporate a review of agency use of the Governmentwide IT Training Framework and standards in its regular review of agencies' mandated training carried out under its Human Capital Assessment and Accountability Framework. Since the IT Training Framework is to be set forth in the unpublished report referred to in the previous paragraph, it is clear that no specific evaluation of IT training has yet been done.

    OPM also objected to "the use of GAO's 22 standards as a mandatory benchmark." We believe this is a misreading of our recommendation. We cite 22 leading practices, not standards, and we recommend that OPM encourage their use. We believe, and experts agree, that greater use of these practices could result in better IT training management.

    OPM's comments are reproduced in appendix II.

    Eight other agencies submitted comments:

    o
       	In oral comments, representatives of the Office of Management and Budget Personnel Policy Branch and Office of E-Government and Information Technology generally agreed with the findings and conclusions in the report and also provided some technical comments, which were addressed as appropriate.

    o
       	In written comments, the Department of Veterans Affairs (VA) generally agreed with our conclusions and provided additional information about its training programs. VA's comments are reproduced in appendix III.

    o
       	The GAO Liaison for the Department of Transportation indicated via email that Transportation generally concurred with our report.

    o
       	The Acting Principal Deputy Inspector General of the Department of Health and Human Services (HHS) characterized our report as "generally on the mark" but had some concerns about how our leading practices-particularly those on identifying training needs and evaluating training-would apply to professional researchers at the National Institutes of Health (NIH). The response states that because our practices represent conventional wisdom in training theory, they might not apply to creative programs that meet unique training needs, such as those at NIH. We agree that medical researchers at NIH are not typical of the federal IT workforce. However, the leading practices are general in nature, are widely accepted, and allow for flexibility in application. HHS's comments are reproduced in appendix IV.

    o
       	The GAO Review Liaison of the Nuclear Regulatory Commission indicated via e-mail that the commission supports the effort to encourage greater use of leading practices across federal agencies.

    o
       	The Deputy Associate CIO for Business and Information Management of the Department of Energy, commenting via&#
    *** End of document. ***