Social Security Administration: Technical and Performance Challenges
Threaten Progress of Modernization (Letter Report, 06/19/98,
GAO/AIMD-98-136).
Pursuant to a congressional request, GAO reviewed the Social Security
Administration's (SSA) ongoing efforts to implement its intelligent
workstation/local area network (IWS/LAN) project, focusing on: (1)
determining the status of SSA's implementation of IWS/LAN; (2) assessing
whether SSA and state disability determination service (DDS) operations
have been disrupted by the installations of IWS/LAN equipment; and (3)
assessing SSA's practices for managing its investment in IWS/LAN.
GAO noted that: (1) SSA has moved aggressively in installing intelligent
workstations and LANs since initiating IWS/LAN acquisitions in December
1996; (2) as of mid-March 1998, it had completed the installation of
about 31,000 workstations and 850 LANs, generally meeting its
implementation schedule for phase I of the initiative; (3) the
contractor that is installing IWS/LAN has expressed concerns about the
future availability of the intelligent workstations that SSA is
acquiring; (4) problems encountered in developing software intended to
operate on IWS/LAN could affect SSA's planned schedule for proceeding
with phase II of this initiative; (5) staff in SSA offices generally
reported no significant disruptions in their ability to serve the public
during the installation and operation of their IWS/LAN equipment; (6)
some state DDSs reported that SSA's decision to manage and control DDS
networks remotely and the IWS/LAN contractor's inadequate responses to
DDS' service calls have led to disruptions in some of their operations;
(7) because IWS/LAN is expected to correct year 2000 deficiencies in
some states' hardware, delaying the installation of IWS/LAN could affect
states' progress in becoming year 2000 compliant; (8) consistent with
the Clinger-Cohen Act of 1996 and Office of Management and Budget
guidance, SSA has followed some of the essential practices required to
effectively manage its IWS/LAN investment; (9) SSA has not established
essential practices for measuring IWS/LAN's contribution to improving
the agency's mission performance; (10) although the agency has developed
baseline data and performance measures that could be used to assess the
project's impact on mission performance, it has not defined target
performance goals or instituted a process for using the measures to
assess the impact of IWS/LAN on mission performance; (11) SSA does not
plan to conduct a post-implementation review of IWS/LAN once it is fully
implemented; and (12) without targeted goals and a defined process for
measuring performance both during and after the implementation of
IWS/LAN, SSA cannot be assured of the extent to which this project is
improving service to the public or that it is actually yielding the
savings anticipated from this investment.
--------------------------- Indexing Terms -----------------------------
REPORTNUM: AIMD-98-136
TITLE: Social Security Administration: Technical and Performance
Challenges Threaten Progress of Modernization
DATE: 06/19/98
SUBJECT: Computer software verification and validation
Federal social security programs
Information resources management
Systems conversions
Claims processing
Local area networks
IDENTIFIER: SSA Year 2000 Program
Old Age Survivors and Disability Insurance Program
SSA Reengineered Disability Program
Social Security Program
******************************************************************
** This file contains an ASCII representation of the text of a **
** GAO report. Delineations within the text indicating chapter **
** titles, headings, and bullets are preserved. Major **
** divisions and subdivisions of the text, such as Chapters, **
** Sections, and Appendixes, are identified by double and **
** single lines. The numbers on the right end of these lines **
** indicate the position of each of the subsections in the **
** document outline. These numbers do NOT correspond with the **
** page numbers of the printed product. **
** **
** No attempt has been made to display graphic images, although **
** figure captions are reproduced. Tables are included, but **
** may not resemble those in the printed version. **
** **
** Please see the PDF (Portable Document Format) file, when **
** available, for a complete electronic file of the printed **
** document's contents. **
** **
** A printed copy of this report may be obtained from the GAO **
** Document Distribution Center. For further details, please **
** send an e-mail message to: **
** **
** **
** **
** with the message 'info' in the body. **
******************************************************************
Cover
================================================================ COVER
Report to the Chairman, Subcommittee on Social Security, Committee on
Ways and Means, House of Representatives
June 1998
SOCIAL SECURITY ADMINISTRATION -
TECHNICAL AND PERFORMANCE
CHALLENGES THREATEN PROGRESS OF
MODERNIZATION
GAO/AIMD-98-136
SSA Modernization
(511249)
Abbreviations
=============================================================== ABBREV
DDS - Disability Determination Service
DI - x
GAO - General Accounting Office
GSA - General Services Administration
IWS/LAN - Intelligent Workstation/Local Area Network
LAN - Local Area Network
NCDDD - National Council of Disability Determination Directors
OMB - Office of Management and Budget
OASI - x
OTA - Office of Technology Assessment
RDS - Reengineered Disability System
SSA - Social Security Administration
Letter
=============================================================== LETTER
B-279868
June 19, 1998
The Honorable Jim Bunning
Chairman, Subcommittee on Social Security
Committee on Ways and Means
House of Representatives
Dear Mr. Chairman:
This report responds to your request concerning the Social Security
Administration's (SSA) ongoing efforts to implement its Intelligent
Workstation/Local Area Network (IWS/LAN) project.\1 As you know, SSA
is in the process of redesigning its work processes and modernizing
its computer systems to better serve an increasing beneficiary
population and achieve improvements in productivity. IWS/LAN is
expected to play a major role in this modernization effort by
providing SSA with the basic automation infrastructure to achieve
increased processing capabilities that will be essential to its major
service delivery and process redesign initiatives. The first phase
of the planned project is a 7-year, approximately $1 billion effort
to acquire more than 56,000 intelligent workstations and 1,700 local
area networks.
Because of the cost and resources that SSA plans to invest in
acquiring IWS/LAN and the project's potential impact on public
service, you requested that we provide information on this
initiative. Specifically, you asked us to (1) determine the status
of SSA's implementation of IWS/LAN, (2) assess whether SSA and state
DDS operations have been disrupted by the installations of IWS/LAN
equipment, and (3) assess SSA's practices for managing its investment
in IWS/LAN.
During testimony before the House Ways and Means Subcommittees on
Human Resources and Social Security in March 1998, we discussed
generally the challenges that SSA faces in implementing IWS/LAN and
other information technology initiatives.\2 This report provides
additional and more specific information on the issues identified
during our review.
--------------------
\1 In June 1996, SSA awarded a national IWS/LAN contract to modernize
and standardize the distributed processing environment in its
headquarters and field components and in state Disability
Determination Services (DDS). This initiative is intended to provide
a distributed processing platform comprised of intelligent
workstations (i.e., personal computers) on employee desktops,
connected to each other and to SSA's mainframe computers by local and
wide area networks.
\2 Social Security Administration: Information Technology Challenges
Facing the Commissioner (GAO/T-AIMD-98-109, March 12, 1998).
RESULTS IN BRIEF
------------------------------------------------------------ Letter :1
SSA has moved aggressively in installing intelligent workstations and
local area networks (LAN) since initiating IWS/LAN acquisitions in
December 1996. As of mid-March 1998, it had completed the
installation of about 31,000 workstations and 850 LANs, generally
meeting its implementation schedule for phase I of the initiative.
However, the contractor that is installing IWS/LAN has expressed
concerns about the future availability of the intelligent
workstations that SSA is acquiring--noting that the 100-megahertz
workstations specified in the contract are increasingly difficult to
obtain. In addition, problems encountered in developing software
intended to operate on IWS/LAN could affect SSA's planned schedule
for proceeding with phase II of this initiative--an effort to provide
additional hardware and software to the computer infrastructure
created in phase I.
Staff in SSA offices generally reported no significant disruptions in
their ability to serve the public during the installation and
operation of their IWS/LAN equipment. However, some state DDSs
reported that SSA's decision to manage and control DDS networks
remotely and the IWS/LAN contractor's inadequate responses to DDS'
service calls have led to disruptions in some of their operations.
At least one state DDS deferred its installations of IWS/LAN because
of its concerns about the lack of network control in the IWS/LAN
environment. Because IWS/LAN is expected to correct Year 2000
deficiencies in some states' hardware, delaying the installation of
IWS/LAN could affect these states' progress in becoming Year 2000
compliant.
Consistent with the Clinger-Cohen Act of 1996 and Office of
Management and Budget (OMB) guidance, SSA has followed some of the
essential practices required to effectively manage its IWS/LAN
investment. This includes assessing costs, benefits, and risks to
justify the agency's investment in IWS/LAN and monitoring the
progress of the project against competing priorities, projected
costs, schedules, and resource availability. However, SSA has not
established essential practices for measuring IWS/LAN's contribution
to improving the agency's mission performance.
Although the agency has developed baseline data and performance
measures that could be used to assess the project's impact on mission
performance, it has not defined target performance goals or
instituted a process for using the measures to assess the impact of
IWS/LAN on mission performance. Further, although the Clinger-Cohen
Act and OMB requirements call for agencies to perform evaluations
after completing information technology projects, SSA does not plan
to conduct a post-implementation review of IWS/LAN once it is fully
implemented. Without targeted goals and a defined process for
measuring performance both during and after the implementation of
IWS/LAN, SSA cannot be assured of the extent to which this project is
improving service to the public or that it is actually yielding the
savings anticipated from this investment.
BACKGROUND
------------------------------------------------------------ Letter :2
Handling increasing service workloads is a critical challenge facing
SSA. The agency is processing a growing number of claims for Social
Security benefits.\3
SSA estimates that it will face continued growth in beneficiaries
over the next few decades as the population ages and life
expectancies increase. The number of OASI and DI beneficiaries is
estimated to increase substantially between calendar years 1997 and
2010--from approximately 44 million to over 54 million.
Recognizing constraints on its staff and resources, SSA has moved to
better serve its increasing beneficiary population and improve its
productivity by redesigning its work processes and modernizing the
computer systems used to support these processes. A key aspect of
the modernization effort is the agency's transition from its current
centralized mainframe-based computer processing environment to a
highly distributed client/server processing environment.\4
IWS/LAN is expected to play a critical role in the modernization by
providing the basic automation infrastructure for using client/server
technology to support the redesigned work processes and improve the
availability and timeliness of information to employees and
appropriate users. Under this initiative, SSA plans to replace
approximately 40,000 "dumb" terminals and other computer equipment
used in over 2,000 SSA and state DDS sites with an infrastructure
consisting of networks of intelligent workstations connected to each
other and to SSA's mainframe computers.\5
The national IWS/LAN initiative consists of two phases. During phase
I, SSA plans to acquire 56,500 workstations,\6 1,742 LANs, 2,567
notebook computers, systems furniture, and other peripheral devices.
Implementation of this platform is intended to provide employees in
the sites with office automation and programmatic functionality from
one terminal. It also aims to provide the basic, standardized
infrastructure to which additional applications and functionality can
later be added. The projected 7-year life-cycle cost of phase I is
$1.046 billion, covering the acquisition, installation, and
maintenance of the IWS/LAN equipment. Under a contract with Unisys
Corporation, SSA began installing equipment for this phase in
December 1996; it anticipates completing these installations in June
1999. Through fiscal year 1997, SSA had reported spending
approximately $565 million on acquiring workstations, LANs, and other
services.\7
Phase II is intended to build upon the IWS/LAN infrastructure
provided through the phase I effort. Specifically, during this
phase, SSA plans to acquire additional hardware and software, such as
database engines, scanners, bar code readers, and facsimile and
imaging servers, needed to support future process redesign
initiatives and client/server applications. SSA plans to award a
series of phase II contracts in fiscal year 1999 and to carry out
actual installations under these contracts during fiscal years 1999
through 2001.
Currently, SSA is developing the first major programmatic software
application to operate on IWS/LAN. This software--the Reengineered
Disability System (RDS)--is intended to support SSA's modernized
disability claims process in the new client/server environment.
Specifically, RDS is intended to automate and improve the Title II
and Title XVI\8 disability claims processes from the initial
claims-taking in the field office, to the gathering and evaluation of
medical evidence in state DDSs, to payment execution in the field
office or processing center and the handling of appeals in hearing
offices. In August 1997, SSA began pilot testing RDS for the
specific purposes of (1) assessing the performance, cost, and
benefits of this software and (2) determining supporting IWS/LAN
phase II equipment requirements.
Agencies, in undertaking systems modernization efforts, are required
by the Clinger-Cohen Act of 1996 to ensure that their information
technology investments are effectively managed and significantly
contribute to improvements in mission performance. The Government
Performance and Results Act of 1993 requires agencies to set goals,
measure performance, and report on their accomplishments. One of the
challenges that SSA faces in implementing IWS/LAN is ensuring that
the planned systems and other resources are focused on helping its
staff process all future workloads and deliver improved service to
the public. In a letter and a report to SSA in 1993 and 1994,
respectively, we expressed concerns about SSA's ability to measure
the progress of IWS/LAN because it had not established measurable
cost and performance goals for this initiative.\9
In addition, SSA faces the critical challenge of ensuring that all of
its information systems are Year 2000 compliant. By the end of this
century, SSA must review all of its computer software and make the
changes needed to ensure that its systems can correctly process
information relating to dates. These changes affect not only SSA's
new network but computer programs operating on both its mainframe and
personal computers. In October 1997, we reported that while SSA had
made significant progress in its Year 2000 efforts, it faced the risk
that not all of its mission-critical systems will be corrected by the
turn of the century.\10 At particular risk were the systems used by
state DDSs to help SSA process disability claims.
--------------------
\3 The Old Age and Survivors Insurance (OASI) and the Disability
Insurance (DI) programs, together commonly known as Social Security,
provide benefits to retired and disabled workers and their dependents
and survivors.
\4 In a client/server environment, servers and individual
workstations are all capable of performing tasks that previously only
the mainframe computer could accomplish. This can sometimes result
in improvements over mainframe performance.
\5 SSA's "dumb" terminals are connected to its mainframe computers
through its data network and are controlled by software executed on
the mainframes. Its personal computers, called intelligent
workstations, have their own data storage and processing
capabilities.
\6 At the conclusion of our review, SSA officials were considering
options for acquiring additional workstations under the national
initiative, which, if exercised, could result in the installation of
as many as 70,000 workstations.
\7 The other services include site preparation, support services and
training, and telecommunications and maintenance.
\8 Titles II and XVI of the Social Security Act authorize SSA's Old
Age, Survivors, and Disability Insurance and Supplemental Security
Income programs, respectively.
\9 Letter from the Director, Human Resources Information Systems,
Information Management and Technology Division, GAO, to the Acting
Commissioner of SSA, March 30, 1994; and Social Security
Administration: Risks Associated With Information Technology
Investment Continue (GAO/AIMD-94-143, September 30, 1994).
\10 Social Security Administration: Significant Progress Made in
Year 2000 Effort, But Key Risks Remain (GAO/AIMD-98-6, October 22,
1997).
OBJECTIVES, SCOPE, AND
METHODOLOGY
------------------------------------------------------------ Letter :3
Our objectives were to (1) determine the status of SSA's
implementation of IWS/LAN, (2) assess whether SSA and state DDS
operations have been disrupted by the installations of IWS/LAN
equipment, and (3) assess SSA's practices for managing its investment
in the IWS/LAN initiative.
To determine the status of SSA's implementation of IWS/LAN, we
analyzed key project documentation, including the IWS/LAN contract,
project plans, and implementation schedules. We observed
implementation activities at select SSA field offices in Alabama,
Florida, Georgia, Minnesota, South Carolina, Texas, and Virginia; at
program service centers in Birmingham, Alabama, and Philadelphia,
Pennsylvania; and at teleservice centers in Minneapolis, Minnesota,
and Fort Lauderdale, Florida. In addition, we reviewed IWS/LAN plans
and observed activities being undertaken by state DDS officials in
Alabama, Georgia, and Minnesota. We also interviewed representatives
of the IWS/LAN contractor--Unisys Corporation--to discuss the status
of the implementation activities.
To assess whether SSA and state DDS operations have been disrupted by
the installations of IWS/LAN equipment, we reviewed planning guidance
supporting the implementation process, such as the IWS/LAN Project
Plan, and analyzed reports summarizing implementation activities and
performance results identified during pilot efforts. We also
interviewed SSA site managers, contractor representatives, and
IWS/LAN users to identify installation and/or performance issues, and
observed operations in select SSA offices where IWS/LAN equipment
installations had been completed. In addition, we discussed IWS/LAN
problems and concerns with DDS officials in 10 states: Alabama,
Arkansas, Arizona, Delaware, Florida, Louisiana, New York, Virginia,
Washington, and Wisconsin, and with the president of the National
Council of Disability Determination Directors, which is a
representative body for all state DDSs.
To assess SSA's management of the IWS/LAN investment, we applied our
guide for evaluating and assessing how well federal agencies select
and manage their investments in information technology resources.\11
We evaluated SSA's responses to detailed questions about its
investment review process that were generated from the evaluation
guide and compared the responses to key agency documents generated to
satisfy SSA's process requirements. We also reviewed IWS/LAN cost,
benefit, and risk analyses to assess their compliance with OMB
guidance. We did not, however, validate the data contained in SSA's
documentation.
We performed our work from July 1997 through March 1998 in accordance
with generally accepted government auditing standards. We requested
comments on a draft of this report from the Commissioner of Social
Security or his designee. The Commissioner provided us with written
comments, which are discussed in the "Agency Comments and Our
Evaluation" section and are reprinted in appendix I.
--------------------
\11 Assessing Risks and Returns: A Guide for Evaluating Federal
Agencies' IT Investment Decision-making, Version 1 (GAO/AIMD-10.1.13,
February 1997).
SSA MET ITS IWS/LAN MILESTONES
THROUGH MARCH 1998, BUT FUTURE
MILESTONES MAY BE MISSED
------------------------------------------------------------ Letter :4
Using a strategy that includes installing workstations and LANs in up
to 20 sites per weekend, SSA, through mid-March 1998, had generally
met its phase I schedule for implementing IWS/LAN. However, the
contractor installing IWS/LAN has expressed concerns about the
availability of the workstations specified in the contract, raising
questions as to whether they can continue to be acquired. In
addition, the pilot effort that SSA began in August 1997 to assess
the performance, cost, and benefits of RDS and identify IWS/LAN phase
II requirements has experienced delays that could affect the schedule
for implementing phase II of this initiative.
IWS/LAN PHASE I IMPLEMENTED
ON SCHEDULE THROUGH MARCH
1998
---------------------------------------------------------- Letter :4.1
Under the phase I schedule, 56,500 intelligent workstations and 1,742
LANs are to be installed in approximately 2,000 SSA and state DDS
sites between December 1996 and June 1999. The schedule called for
approximately 30,500 workstations and about 850 LANs to be installed
by mid-March 1998. According to SSA records, the agency generally
met this schedule with the actual installation of 31,261 workstations
and 850 LANs by March 15, 1998. These installations occurred at 753
SSA sites and 20 DDS sites\12 (covering 12 states and the federal
DDS).\13 SSA reported in its fiscal year 1997 accountability report
that the number of front-line employees using IWS/LAN increased to
50.2 percent--exceeding by 2.2 percent the fiscal year 1997 Results
Act goal.\14
The standard intelligent workstation configuration includes a
100-megahertz Pentium personal computer with 32 megabytes of random
access memory, the Windows NT 4.0 operating system, a 1.2-gigabyte
hard (fixed) disk drive, 15-inch color display monitor, and 16-bit
network card with adaptation cable. Last year the contractor,
Unisys, submitted a proposal to upgrade the intelligent workstation
by substituting a higher speed processor at additional cost. Unisys
noted that it was having difficulty obtaining 100-megahertz
workstations. However, SSA did not accept Unisys' upgrade proposal.
Further, the Deputy Commissioner for Systems stated that SSA did not
believe it was necessary to upgrade to a faster processor because the
100-megahertz workstation meets its current needs.
For its modernization efforts to succeed, SSA must have the necessary
workstations to support its processing needs. This is particularly
important given the agency's expressed intent to operate future
client/server software applications on IWS/LAN to support redesigned
work processes. Adding database engines, facsimile, imaging, and
other features like those planned by SSA during phase II of the
IWS/LAN initiative could demand a workstation with more memory,
larger disk storage, and a processing speed higher than 100
megahertz. Personal computers available in today's market operate at
about three times this speed.
Preliminary testing of the RDS software has already shown the need
for SSA to upgrade the workstation's random access memory from 32
megabytes to 64 megabytes. However, systems officials told us that
their tests have not demonstrated a need for a faster workstation.
As discussed in the following section, SSA is encountering problems
and delays in completing its tests of the RDS software. In addition,
at the conclusion of our review, SSA had begun holding discussions
with Unisys regarding the availability of the 100-megahertz
workstations.
--------------------
\12 Some of these DDS sites had received only partial installations
of IWS/LAN.
\13 The federal DDS provides back-up services to state DDSs when the
state offices cannot process their workloads and serves as a model
office for testing new technologies and work processes.
\14 Social Security Administration Accountability Report for Fiscal
Year 1997.
PROBLEMS IN RDS PILOT COULD
DELAY IWS/LAN PHASE II
IMPLEMENTATION
---------------------------------------------------------- Letter :4.2
SSA has experienced problems and delays in the pilot effort that it
initiated in August 1997 to assess the performance, cost, and
benefits of RDS and identify IWS/LAN phase II requirements. Under
the pilot, an early release of the software is being tested in one
SSA field office and the federal DDS to acquire feedback from end
users regarding its performance. SSA planned to make improvements to
the software based on these pilot results and then expand its testing
of the software to all SSA and DDS components in the state of
Virginia. The results of the pilot testing in Virginia were to be
used in determining hardware and software requirements to support
IWS/LAN phase II acquisitions, beginning in fiscal year 1999.
SSA encountered problems with RDS during its initial pilot testing.
For example, systems officials stated that, using RDS, the reported
productivity of claims representatives in the SSA field office
dropped. Specifically, the officials stated that before the
installation of RDS, each field office claims representative
processed approximately five case interviews per day. After RDS was
installed, each claims representative could process only about three
cases per day.
At the conclusion of our review, systems officials stated that
because the RDS software has not performed as anticipated, SSA has
entered into a contract with Booz-Allen and Hamilton to independently
evaluate and recommend options for proceeding with the development of
RDS. In addition, SSA has delayed expanding the pilot by 9
months--from October 1997 to July 1998. This is expected to further
delay SSA's national roll-out and implementation of RDS.\15
Moreover, because RDS is essential to identifying IWS/LAN phase II
requirements, the Deputy Commissioner for Systems has stated that
delaying the pilot will likely result in slippages in SSA's schedule
for acquiring and implementing phase II equipment.
--------------------
\15 In September 1996, we reported that software development problems
had delayed the scheduled implementation of RDS by more than 2 years.
See Social Security Administration: Effective Leadership Needed to
Meet Daunting Challenges (GAO/HEHS-96-196, September 12, 1996).
SSA OFFICES REPORTED SMOOTH
TRANSITION TO IWS/LAN, BUT
NETWORK MANAGEMENT CONCERNS IN
STATE OFFICES COULD AFFECT
SERVICE TO THE PUBLIC
------------------------------------------------------------ Letter :5
Nationwide implementation of IWS/LAN is a complex logistical task for
SSA, requiring coordination of site preparation (such as electrical
wiring and cabling) in over 2,000 remote locations,
contractor-supplied and installed furniture and intelligent
workstation components, and training of over 70,000 employees in SSA
and DDS locations. Moreover, once installed, these systems must be
managed and maintained in a manner that ensures consistent and
quality service to the public.
During our review, staff in the 11 SSA offices that we visited
generally stated that they had not experienced any significant
disruptions in their ability to serve the public during the
installation and operation of IWS/LAN. They attributed the smooth
transition to SSA's implementation of a well-defined strategy for
conducting site preparations, equipment installations, and employee
training. Part of that strategy required equipment installation and
testing to be performed on weekends so that the IWS/LAN equipment
would be operational by the start of business on Monday. In
addition, staff were rotated through training and client service
positions and augmented with staff borrowed from other field offices
to maintain service to the public during the post-installation
training period. Further, because the new workstations provide
access to the same SSA mainframe software applications as did the old
terminals and LAN equipment, staff were able to process their
workloads in a similar manner as with the previous environment.
State DDSs generally were less satisfied with the installation and
operation of IWS/LAN in their offices. Administrators and systems
staff in the 10 DDS sites that we visited expressed concerns about
the loss of network management and control over IWS/LAN operations in
their offices and dissatisfaction with the service and technical
support received from the contractor following the installation of
IWS/LAN equipment.
In particular, SSA initially planned to centrally manage the
operation and maintenance of IWS/LAN equipment. However, DDS
officials in 7 of the 10 offices expressed concern that with SSA
managing their networks and operations, DDSs can no longer make
changes or fixes to their equipment locally and instead, must rely on
SSA for system changes or network maintenance. Eight of the 10 DDSs
reported that under this arrangement, the IWS/LAN contractor had been
untimely in responding to certain of their requests for service,
resulting in disruptions to their operations. For example, a DDS
official in one state told us that at the time of our discussion, she
had been waiting for approximately 2 weeks for the IWS/LAN contractor
to repair a hard disk drive in one of the office's workstations.
In January 1998, the National Council of Disability Determination
Directors (NCDDD), which represents the state DDSs, wrote to SSA to
express the collective concerns of the DDSs regarding SSA's plan to
manage and control their IWS/LAN networks. NCDDD recommended that
SSA pilot the IWS/LAN equipment in one or more DDS office to evaluate
options for allowing the states more flexibility in managing their
networks. It further proposed that IWS/LAN installations be delayed
for states whose operations would be adversely affected by the loss
of network control. At least one state DDS--Florida--refused to
continue with the roll-out of IWS/LAN in its offices until this issue
is resolved. Because IWS/LAN is expected to correct Year 2000
deficiencies in some states' hardware, however, NCDDD cautioned that
delaying the installation of IWS/LAN could affect the states'
progress in becoming Year 2000 compliant. At the conclusion of our
review, the Deputy Commissioner for Systems told us that SSA had
begun holding discussions with state officials in early March 1998 to
identify options for addressing the states' concerns about the
management of their networks under the IWS/LAN environment.
SSA WILL NOT MEASURE BENEFITS
DERIVED FROM IWS/LAN
------------------------------------------------------------ Letter :6
Federal legislation and OMB directives require agencies to manage
major information technology acquisitions as investments. In
implementing IWS/LAN, SSA has followed a number of practices that are
consistent with these requirements, such as involving executive staff
in the selection and management of the initiative and assessing the
cost, benefits, and risks of the project to justify its acquisition.
However, SSA's practices have fallen short of ensuring full and
effective management of the investment in IWS/LAN because it did not
include plans for measuring the project's actual contributions to
improved mission performance.
MANAGEMENT OVERSIGHT AND
ANALYSIS SUPPORTED IWS/LAN
IMPLEMENTATION
---------------------------------------------------------- Letter :6.1
According to the Clinger-Cohen Act and OMB guidance,\16 effective
technology investment decision-making requires that processes be
implemented and data collected to ensure that (1) project proposals
are funded on the basis of management evaluations of costs, risks,
and expected benefits to mission performance and (2) once funded,
projects are controlled by examining costs, the development schedule,
and actual versus expected results. These goals are accomplished by
considering viable alternatives, preparing valid cost-benefit
analyses, and having senior management consistently make data-driven
decisions on major projects.
SSA followed an established process for acquiring IWS/LAN that met a
number of these requirements. For example, senior management
reviewed and approved the project's acquisition and has regularly
monitored the progress of the initiative against competing
priorities, projected costs, schedules, and resource availability.
SSA also conducted a cost-benefit analysis to justify its
implementation of IWS/LAN. This analysis was based on comparisons of
the time required to perform certain work tasks before and after the
installation of IWS/LAN equipment in 10 SSA offices selected for a
pilot study during January through June 1992. For example, the pilot
tested the time savings attributed to SSA employees not having to
walk from their desks or wait in line to use a shared personal
computer. Based on the before and after time savings identified for
each work task, SSA projected annual savings from IWS/LAN of 2,160
workyears that could be used to process growing workloads and improve
service. In a review of the IWS/LAN initiative in 1994, the Office
of Technology Assessment (OTA)\17 found SSA's cost-benefit analysis
to be sufficient for justifying the acquisition of IWS/LAN.
--------------------
\16 OMB Circular A-11 requires that planned information technology
acquisitions be based on a cost-benefit analysis. Similarly, OMB
Circular A-94 requires that decisions to initiate government projects
be based on an analysis of expected life-cycle costs and benefits,
and that alternative means of achieving program objectives be
considered.
\17 The Social Security Administration's Decentralized Computer
Strategy: Issues and Options (OTA-TCT-592, April 1994).
SSA IS NOT USING KEY
PERFORMANCE MEASURES TO
ASSESS THE IMPACT OF IWS/LAN
ON MISSION PERFORMANCE
---------------------------------------------------------- Letter :6.2
Although SSA followed certain essential practices for acquiring
IWS/LAN, it has not yet implemented performance goals and measures to
assess the impact of this investment on productivity and mission
performance. Under the Clinger-Cohen Act, agencies are to establish
performance measures to gauge how well their information technology
supports program efforts and better link their information technology
plans and usage to program missions and goals. Successful
organizations rely heavily upon performance measures to
operationalize mission goals and objectives, quantify problems,
evaluate alternatives, allocate resources, track progress, and learn
from mistakes.\18
Performance measures also help organizations determine whether their
information systems projects are really making a difference, and
whether that difference is worth the cost. The Clinger-Cohen Act
also requires that large information technology projects be
implemented incrementally and that each phase should be cost
effective and provide mission-related benefits. It further requires
that performance measures be established for each phase to determine
whether expected benefits were actually achieved.
In our September 1994 report,\19 we noted that as part of an effort
with the General Services Administration (GSA) to develop a
"yardstick" to measure the benefits that IWS/LAN will provide the
public,\20 SSA had initiated actions aimed at identifying cost and
performance goals for IWS/LAN. SSA identified six categories of
performance measures that could be used to determine the impact of
IWS/LAN technology on service delivery goals and reengineering
efforts.\21 It had planned to establish target productivity gains for
each measure upon award of the IWS/LAN contract. GSA was to then use
these measures to assess IWS/LAN's success.
As of March 1998, however, SSA had established neither the target
goals to help link the performance measures to the agency's strategic
objectives nor a process for using the measures to assess IWS/LAN's
impact on agency productivity and mission performance. In addition,
although the Clinger-Cohen Act and OMB guidance\22 state that
agencies should perform retrospective evaluations after completing an
information technology project, SSA officials told us that they do
not plan to conduct a post-implementation review of the IWS/LAN
project once it is fully implemented. According to the Director of
the Information Technology Systems Review Staff,\23 SSA currently
does not plan to use any of the measures to assess the project's
impact on agency productivity and mission performance because (1) the
measures had been developed to fulfill a specific GSA procurement
requirement that no longer exists and (2) it believes the results of
the pilots conducted in 1992 sufficiently demonstrated the savings
that will be achieved with each IWS/LAN installation.
It is essential that SSA follow through with the implementation of a
performance measurement process for each phase of the IWS/LAN effort.
Measuring performance is necessary to show how this investment is
contributing to the agency's goal of improving productivity. Among
leading organizations that we have observed, managers use performance
information to continuously improve organizational processes,
identify performance gaps, and set improvement goals.\24
The performance problems that SSA has already encountered in piloting
software on IWS/LAN make it even more critical for SSA to implement
performance measures and conduct post-implementation reviews for each
phase of this initiative.
SSA believes that the results of its pilot effort undertaken in 1992
to justify the acquisition of IWS/LAN sufficiently demonstrate that
it will achieve its estimated workyear savings. However, the pilot
results are not an acceptable substitute for determining the actual
contribution of IWS/LAN to improved productivity. In particular,
although the pilots assessed task savings for specific functions
performed in each office, they did not demonstrate IWS/LAN's actual
contribution to improved services gained through improvements in the
accuracy of processing or improvements in processing times. In
addition, OTA noted in its 1994 review\25
that the relatively small number of pilots may not have adequately
tested all the potential problems that could arise when the equipment
is deployed at all of SSA's sites.
Further, information gained from post-implementation reviews is
critical for improving how the organization selects, manages, and
uses its investment resources. Without a post-implementation review
of each phase of the IWS/LAN project, SSA cannot validate projected
savings, identify needed changes in systems development practices,
and ascertain the overall effectiveness of each phase of this project
in serving the public. Post-implementation reviews also serve as the
basis for improving management practices and avoiding past mistakes.
--------------------
\18 Executive Guide: Improving Mission Performance Through Strategic
Information Management and Technology (GAO/AIMD-94-115, May 1994).
\19 GAO/AIMD-94-143, September 19, 1994.
\20 This effort resulted from the National Performance Review, which
required agencies to include performance measures on all information
technology purchases of $100 million or more.
\21 These measures were (1) productivity benefits of baseline
automation with IWS/LAN in the state DDSs (a computation of the DDS
productivity gain by comparing pre-IWS/LAN baseline data with
post-IWS/LAN implementation data), (2) SSA baseline automation
savings generated by the implementation of IWS/LAN in SSA and Office
of Hearing and Appeal field office components, (3) improvements in
payment and service delivery accuracy resulting from use of the 800
Number Expert System and dial-in remote access via IWS/LAN, (4)
number of IWS/LANs installed per month as compared to the IWS/LAN
implementation schedule, (5) existing terminal redeployment and
phase-out, and (6) contract cost and pricing.
\22 OMB Circular A-130, Section 8b(1).
\23 The Information Technology Systems Review Staff, within the
Office of Finance, Assessment, and Management, performs independent
reviews of proposed information technology projects for the Chief
Information Officer. Its oversight objectives include ensuring that
initiatives (1) are appropriately prioritized, priced, and timed, and
are supportable and affordable, and (2) progress within the approved
cost and schedule and result in systems that meet mission needs and
provide anticipated benefits.
\24 Executive Guide: Effectively Implementing the Government
Performance and Results Act (GAO/GGD-96-118, June 1996).
\25 OTA-TCT-592, April 1994.
CONCLUSIONS
------------------------------------------------------------ Letter :7
SSA is relying on IWS/LAN to play a vital role in efforts to
modernize its work processes and improve service delivery, and it has
made good progress in implementing workstations and LANs that are a
part of this effort. However, equipment availability and capability
issues, problems in developing software that is to operate on the
IWS/LAN workstations, and concerns among state DDSs that their
equipment will not be adequately managed and serviced by SSA,
threaten the continued progress and success of this initiative.
Moreover, absent target goals and a defined process for measuring
performance, SSA will not be able to determine whether its investment
in each phase of IWS/LAN is yielding expected improvements in service
to the public.
RECOMMENDATIONS
------------------------------------------------------------ Letter :8
To strengthen SSA's management of its IWS/LAN investment, we
recommend that the Commissioner of Social Security direct the Deputy
Commissioner for Systems to
-- immediately assess the adequacy of workstations specified in the
IWS/LAN contract, and based on this assessment, determine (1)
the number and capacity of workstations required to support the
IWS/LAN initiative and (2) its impact on the IWS/LAN
implementation schedule;
-- work closely with state DDSs to promptly identify and resolve
network management concerns and establish a strategy for
ensuring the compliance of those states relying on IWS/LAN
hardware for Year 2000 corrections;
-- establish a formal oversight process for measuring the actual
performance of each phase of IWS/LAN, including identifying the
impact that each IWS/LAN phase has on mission performance and
conducting post-implementation reviews of the IWS/LAN project
once it is fully implemented.
AGENCY COMMENTS AND OUR
EVALUATION
------------------------------------------------------------ Letter :9
In commenting on a draft of this report, SSA generally agreed with
the issues we identified and described actions that it is taking in
response to our recommendations to resolve them. These actions
include (1) determining remaining IWS/LAN workstation needs, (2)
addressing state DDS network management concerns and related Year
2000 compliance issues, and (3) implementing a performance
measurement strategy for the IWS/LAN initiative. These actions are
important to the continued progress and success of the IWS/LAN
initiative, and SSA must be diligent in ensuring that they are fully
implemented.
In responding to our first recommendation to assess the adequacy of
workstations specified in the IWS/LAN contract, SSA stated that it
had determined the number of workstations required to complete the
IWS/LAN implementation and was working on a procurement strategy and
schedule for this effort. SSA also stated that its current tests do
not show a need for workstations with a processing speed higher than
100 megahertz. The agency further noted that terms and conditions in
the IWS/LAN contract will enable it to acquire a higher powered
computer or other technology upgrades when the need arises.
As discussed earlier in our report, it is important that SSA have the
necessary workstations to support its processing needs in the
redesigned work environment. Therefore, as SSA continues its
aggressive pace in implementing IWS/LAN, it should take all necessary
steps to ensure that it has fully considered its functional
requirements over the life of these workstations. Doing so is
especially important since SSA has encountered problems and delays in
completing tests of the RDS software that are vital to determining
future IWS/LAN requirements.
Our second recommendation concerned SSA's working closely with state
DDSs to identify and resolve network management concerns and
establish a strategy for ensuring the compliance of those states
relying on IWS/LAN hardware for Year 2000 corrections. SSA
identified various actions, which if successfully implemented, could
help resolve DDS concerns regarding network management and the
maintenance of IWS/LAN equipment, and facilitate its efforts in
becoming Year 2000 compliant.
In responding to our final recommendation that it establish a formal
oversight process for measuring the actual performance of each phase
of IWS/LAN, SSA agreed that performance goals and measures should be
prescribed to determine how well information technology investments
support its programs and provide expected results. SSA stated that
it is determining whether expected benefits are being realized from
IWS/LAN installations through in-process and postimplementation
assessments. SSA further noted that its planning and budgeting
process ensures that it regularly assesses the impact of IWS/LAN on
agency productivity and mission performance.
However, during the course of our review, SSA could not provide
specific information to show how its planning and budgeting process
and data on workyear savings resulting from IWS/LAN installations
were being used to assess the project's actual contributions to
improved productivity and mission performance. In addition, two of
the three measures that SSA identified in its response--the number of
IWS/LANs installed per month and existing terminal redeployment and
phase-out--provide information that is more useful for assessing the
progress of SSA's IWS/LAN installations and existing terminal
redeployment efforts.
To ensure that its investments are sound, it is crucial that SSA
develop measures to assess mission-related benefits, and use them in
making project decisions. We will continue to monitor SSA's efforts
in assessing the benefits of IWS/LAN installations through its
in-process and postimplementation assessments and its planning and
budgeting process.
---------------------------------------------------------- Letter :9.1
We are sending copies of this report to the Commissioner of Social
Security; the Director of the Office of Management and Budget;
appropriate congressional committees; and other interested parties.
Copies will also be made available to others upon request.
Please contact me at (202) 512-6253 or by e-mail at
[email protected] if you have any questions concerning this
report. Major contributors to this report are listed in appendix II.
Sincerely yours,
Joel C. Willemssen
Director, Civil Agencies Information Systems
(See figure in printed edition.)Appendix I
COMMENTS FROM THE SOCIAL SECURITY
ADMINISTRATION
============================================================== Letter
(See figure in printed edition.)
(See figure in printed edition.)
(See figure in printed edition.)
(See figure in printed edition.)
MAJOR CONTRIBUTORS TO THIS REPORT
========================================================== Appendix II
ACCOUNTING AND INFORMATION
MANAGEMENT DIVISION, WASHINGTON,
D.C.
Valerie C. Melvin, Assistant Director
Michael A. Alexander, Senior Information Systems Analyst
Gwendolyn M. Adelekun, Senior Business Process Analyst
ATLANTA FIELD OFFICE
Pamlutricia Greenleaf, Senior Evaluator
Kenneth A. Johnson, Senior Information Systems Analyst
*** End of document. ***