Social Security: Telephone Access Enhanced at Field Offices Under
Demonstration Project (Letter Report, 02/23/96, GAO/HEHS-96-70).
Pursuant to a legislative requirement, GAO reviewed the Social Security
Administration's (SSA) telephone access demonstration project.
GAO found that: (1) under one of the two telephone demonstration project
configurations tested, telephone access improved significantly,
busy-signal rates dropped, and the number of callers reaching SSA
improved; (2) although improvements were statistically significant under
a broader access definition, more callers were placed on hold because
staffing had not increased; (3) configuration results varied among SSA
offices; (4) SSA field office staff believe that the installation of
voice mail equipment has improved office efficiency and public service;
(5) SSA expects to complete its internal evaluations of the project's
effect on local operations, employees, and public relations by the end
of February 1996; and (6) the equipment tested in the demonstration
project has the potential to further SSA public service goals, but SSA
must assess the costs and contributions of the equipment before
installing it systemwide.
--------------------------- Indexing Terms -----------------------------
REPORTNUM: HEHS-96-70
TITLE: Social Security: Telephone Access Enhanced at Field Offices
Under Demonstration Project
DATE: 02/23/96
SUBJECT: Federal social security programs
Telecommunications equipment
Telephone communications operations
Public relations
Productivity
Customer service
Information dissemination operations
Statistical methods
IDENTIFIER: SSA Automated Attendant and Voice Mail Demonstration Project
Retirement, Survivors, and Disability Insurance Program
Supplemental Security Income Program
******************************************************************
** This file contains an ASCII representation of the text of a **
** GAO report. Delineations within the text indicating chapter **
** titles, headings, and bullets are preserved. Major **
** divisions and subdivisions of the text, such as Chapters, **
** Sections, and Appendixes, are identified by double and **
** single lines. The numbers on the right end of these lines **
** indicate the position of each of the subsections in the **
** document outline. These numbers do NOT correspond with the **
** page numbers of the printed product. **
** **
** No attempt has been made to display graphic images, although **
** figure captions are reproduced. Tables are included, but **
** may not resemble those in the printed version. **
** **
** Please see the PDF (Portable Document Format) file, when **
** available, for a complete electronic file of the printed **
** document's contents. **
** **
** A printed copy of this report may be obtained from the GAO **
** Document Distribution Center. For further details, please **
** send an e-mail message to: **
** **
** **
** **
** with the message 'info' in the body. **
******************************************************************
Cover
================================================================ COVER
Report to Congressional Committees
February 1996
SOCIAL SECURITY - TELEPHONE ACCESS
ENHANCED AT FIELD OFFICES UNDER
DEMONSTRATION PROJECT
GAO/HEHS-96-70
Improving Phone Access at Field Offices
(105144)
Abbreviations
=============================================================== ABBREV
OPIR - Office of Program Integrity Reviews
OWA - Office of Workforce Analysis
SSA - Social Security Administration
SSI - Supplemental Security Income
Letter
=============================================================== LETTER
B-260584
February 23, 1996
The Honorable William Roth
Chairman
The Honorable Daniel Patrick Moynihan
Ranking Minority Member
Committee on Finance
United States Senate
The Honorable Bill Archer
Chairman
The Honorable Sam M. Gibbons
Ranking Minority Member
Committee on Ways and Means
U.S. House of Representatives
The Social Security Administration (SSA) administers the Retirement
and Survivors and the Disability Insurance Programs and the
Supplemental Security Income Program. Together, these programs
affect the lives of almost every person in this country. SSA has an
extensive field office structure to administer these programs,
consisting of more than 1,300 local offices. It also has a
nationwide toll-free telephone number and is testing enhanced local
office telephone service at selected offices.
SSA has a long history of trying to provide caring and courteous
service to its customers. Recently, it has established a goal of
providing world-class public service in administering Social Security
programs. Accessible telephone service that will address callers'
questions and concerns is a critical part of SSA's service delivery
goal. SSA recently initiated a demonstration project to test whether
new telephone equipment and technology would improve phone access to
its local offices.
Beginning in late February 1995, SSA began installing new telephone
equipment, called automated attendant and voice mail, at 30 of its
more than 800 nationwide field offices that list their phone numbers
in local telephone directories.\1 The equipment was installed in
different configurations. All staff in each office had voice mail
installed on their phones. Some of the offices also received
automated attendant with automated service options on their general
inquiry phone lines.\2 As part of the demonstration, SSA planned two
internal evaluations of the project to assess the equipment's effect
on local office efficiency and employees' views of the equipment and
to identify callers' views about this new service.
The Social Security Independence and Program Improvements Act of 1994
directed us to determine whether this new equipment would improve the
public's telephone access to local SSA offices and report our results
by January 31, 1996. Also on the basis of congressional interest, we
obtained information about SSA's efforts to evaluate its
demonstration project.
To gauge how access changed with the new equipment's installation, we
met with officials responsible for the project and conducted two
tests of telephone service at the participating offices. One test
established baseline performance data for phone service before
installation of the equipment. The second test measured service
after installation of the equipment.\3 We measured how access changed
for those wanting to speak directly to an SSA representative about a
general matter. We did not, however, measure the use of the
automated services. Appendix I details the design and execution of
our tests.
As part of our work, we also visited 12 of the 30 local offices
participating in the demonstration project. At these offices we met
with managers, supervisors, and staff using the new equipment to
discuss its effect on daily operations and any public reaction. We
also met with SSA personnel responsible for conducting internal
evaluations of the project. We reviewed documents about study
objectives and SSA strategies to achieve those objectives. We began
this assignment in January 1995 and completed our analysis in
December 1995. This assignment was performed in accordance with
generally accepted government auditing standards.
--------------------
\1 Automated attendant refers to equipment that answers telephone
calls; voice mail refers to equipment that plays prerecorded messages
and can record information left by callers.
\2 "Automated service options" refers to question-and-answer voice
mail features. Callers who choose this option hear prerecorded
questions that solicit basic information such as their name, address,
and Social Security number. Responses to these questions are
recorded so that SSA can later take appropriate action to fulfill
service requests.
\3 For test purposes, we selected the eight busiest telephone-call
days within a 1-month period. Therefore, our results are not
comparable with those from our prior report on local office
busy-signal rates, which reported an average busy-signal rate over a
1-month period.
RESULTS IN BRIEF
------------------------------------------------------------ Letter :1
Telephone access (calls reaching an SSA employee either with the
caller spending no time on hold or less than 2 minutes on hold) under
one of the configurations being tested by SSA showed an improvement
of 23 percentage points. In addition, busy-signal rates dropped by
more than 55 percentage points; however, because staffing did not
increase, many callers reaching SSA spent some time on hold before
speaking with an SSA representative. SSA field office staff also
perceived the installation of voice mail equipment at their desks as
having a very positive effect on office efficiency and public
service.
SSA has initiated two internal evaluations of the demonstration
project. Neither SSA's study of the new equipment's effect on local
operations nor its study of public reaction to the new equipment,
however, was completed as of early February 1996.
Our work suggests that the technology tested in the demonstration
project has the potential to further SSA's public service goals.
However, public reaction and the effect on operations will need to be
factored in as SSA assesses the costs and contributions of this
technology to meeting these goals.
BACKGROUND
------------------------------------------------------------ Letter :2
For many years, the Congress has expressed concerns about the
public's telephone access to SSA. Efforts to improve this access
have resulted in a dual system of telephone service (a nationwide 800
number and local office service at more than 800 of SSA's field
offices) and also led to the current demonstration project.
TELEPHONE SERVICE AT SSA
---------------------------------------------------------- Letter :2.1
In 1989, SSA established a nationwide toll-free 800 number to replace
its local office telephone service. With the implementation of this
service, SSA directed its local offices to remove their general
inquiry telephone numbers from local phone directories. In their
place, the offices listed the new toll-free 800 number.
In establishing this toll-free network, SSA intended to provide all
of its customers with equal and toll-free access to program services.
SSA envisioned that the public would call the 800 number with basic
questions about the program, when reporting changes in benefit
status, with problems or concerns specific to Social Security
records, or to make appointments with local field office staff. The
public could continue to contact local office staff when necessary by
requesting the unpublished telephone number for any office from SSA's
800 number staff.
The establishment of a national toll-free telephone network was
planned to facilitate an agencywide 20-percent staff reduction that
occurred between 1985 and 1990. By transferring a large workload
from its field offices to the 800 number, SSA hoped the downsized
offices would be better able to conduct nontelephone business.
SSA had start-up problems when the 800 number went on-line
nationwide. It had underestimated the volume of calls that would be
made to the 800 number and was not able to staff the service
adequately, especially when call volumes were heaviest. High
busy-signal rates made it difficult for the public to reach SSA,
generating complaints to SSA and to the Congress.
In response, SSA took several steps to expand its capacity to handle
the volume of 800 number calls. These included actions to increase
staff devoted to handling calls during the heaviest calling periods,
converting additional facilities to 800 number phone centers, and
increasing the number of telephone lines devoted to 800 number calls.
Even with these actions, busy-signal rates remained high because the
number of calls placed to the 800 number continued to grow rapidly.
For example, in 1990, callers placed 85 million calls to SSA, and the
overall busy-signal rate was 34 percent. In 1994, callers placed
almost 117 million calls to the 800 number, and the overall
busy-signal rate grew to about 45 percent.
During the start-up of the 800 number, these problems concerned the
Congress so much that, in 1990, it required SSA to restore telephone
access to local offices. As a result, SSA reinstated direct local
telephone service to about 830 of its more than 1,300 local offices
by publishing their telephone numbers in local directories in
addition to the 800 number. However, the Congress did not provide
any additional resources for SSA to either purchase telephone
equipment or increase staff to handle the reinstated workload.
Because it had fewer field office staff due to its downsizing in the
late 1980s, SSA chose to implement the local office telephone service
with a minimum number of telephone lines and staff. In June 1992,
the House Committee on Ways and Means asked us to evaluate the
public's ability to access local offices that offered local phone
service. In March 1993, we reported that the busy- signal rate at
local offices averaged 47.3 percent during the month tested.\4 In
October 1993, SSA advised the Congress about its plans to conduct a
demonstration project to enhance local office operations and perhaps
improve telephone access to its local offices.
--------------------
\4 Social Security: Telephone Busy Signal Rates at Local SSA Field
Offices (GAO/HRD 93-49, Mar. 4, 1993).
TELEPHONE DEMONSTRATION
PROJECT: DESIGN AND
INSTALLATION
---------------------------------------------------------- Letter :2.2
To improve the public's telephone access to its local offices, SSA is
conducting a demonstration project to test telephone equipment known
as automated attendant and voice mail. SSA's demonstration project
involves 30 of its field offices and three different configurations
of the automated attendant and voice mail equipment (referred to as
methods A, B, and C in this report). SSA wanted local offices from
each of its 10 regions involved in the project, and it allowed the
regions to select these offices on the basis of the type of telephone
equipment they were already using and their willingness to
participate in the project.\5
Each method being tested in the demonstration project represents a
different configuration of equipment. In method A offices, SSA added
automated attendant and question-and-answer mail boxes to its general
inquiry lines. In addition, it added voice mail to staff member
extensions.
A caller to method A offices hears a recorded greeting that
identifies the agency, office hours, and address. This basic
information answers caller questions in many cases. Callers seeking
other types of assistance have other options:
Callers may press the extension number of a particular employee
with whom they may be working on a claim or other matter.
If not already working with an SSA representative, callers may also
select an automated service menu for routine matters such as
reporting changes in address, making an appointment to file for
benefits, or requesting an original or duplicate Social Security
card. These services are provided without direct staff
intervention through the use of question-and-answer voice mail
messages.
Finally, if callers wish to speak to an SSA representative, they
can choose to hold the line until one becomes available.
Method B offices operate the same way as method A offices except that
one additional feature is available. Method B offices have an
additional general inquiry telephone line to play a message that
advises callers that all available lines are busy. This message also
states that the caller should either call at a later time or may call
SSA's toll-free 800 number. Callers are only connected to this line
when all the other general inquiry lines are already in use.
For the demonstration, method C offices do not have any additional
telephone lines, automated attendant, or the related
question-and-answer mailboxes on their general inquiry lines. They
have only voice mail capability at the desks of staff members.
The underlying objective of the demonstration project is to improve
the public's access by making more telephone lines available to
handle phone calls at local offices. The demonstration project
equipment configurations have also extended service hours for method
A and B offices because, with automated attendant, after-hours calls
can be answered and callers can leave voice mail messages.
Most method A and B offices received additional general inquiry
telephone lines when SSA installed the new equipment in their
offices. Local managers in some participating offices, however, did
not want additional lines because they believed that they could not
handle additional telephone calls without increased staffing. Table
1 shows each method A and B office and the number of general inquiry
lines each had before and after equipment was installed for the
demonstration project. As shown, five method A and eight method B
offices received at least one additional general inquiry line.
Table 1
List of Method A and B Office General
Inquiry Line Increases--Before and After
Equipment Installation for Demonstration
Project
Genera Genera
l l
inquir inquir
y y
lines lines
Office name before after
------------------------------------------------------ ------ ------
Method A
----------------------------------------------------------------------
American River, Cal. 2 2
Manchester, N.H. 2 2
Flatbush, N.Y. 3 3
Newport News, Va. 2 3
Knoxville, Tenn. 3 13
Las Vegas, Nev. 10 15
Harlingen, Tex. 1 4
Sioux City, Ia. 2 4
Cheyenne, Wyo. 3 3
Oakland, Cal. 4 4
Method B
----------------------------------------------------------------------
Pocatello, Ida. 1 3
Attleboro, Mass. 1 5
Albany, N.Y. 2 3
Petersburg, Va. 2 4
Asheville, N.C. 4 5
West Indianapolis, Ind. 2 5
El Dorado, Ark. 2 6
Norfolk, Neb. 3 3
Stockton, Cal. 4 5
----------------------------------------------------------------------
--------------------
\5 The selection of offices to participate in the demonstration
project was not done randomly.
TELEPHONE ACCESS HAS IMPROVED,
BUT MORE CALLS ARE BEING PLACED
ON HOLD
------------------------------------------------------------ Letter :3
We found statistically significant improvement in access under method
B, while method A showed no statistically significant change in
access. Under method B, busy-signal rates dropped greatly, but more
calls were being placed on hold. Because method C did not involve
any change to the general inquiry lines, we did not consider its
effect on access to the local lines.
When examining how telephone access changed by the individual offices
in the demonstration, we found mixed results among both methods A and
B. We also found that SSA staff in the demonstration offices
strongly believe that the voice mail equipment on their desk phones
enhanced efficiency and public service.
To measure changes in access for evaluation purposes, we grouped the
call outcomes into two categories: access and no access. We
considered access to consist of two call outcomes: calls in which we
spoke to an SSA employee without spending any time on hold and calls
in which we were on hold for less than 2 minutes before speaking to
an SSA employee.\6 We considered no access to consist of five
different call outcomes: busy signals, no answer after the phone
rang 10 times, recorded messages directing us to call at a later
time, calls that were disconnected before we had a chance to speak
with an SSA representative, and all calls in which we were placed on
hold for more than 2 minutes.
We selected 2 minutes as the time we would wait on hold before
hanging up because we thought it was a reasonable expectation. In
addition, our definition is consistent with information SSA obtained
from a survey of its clients. In July 1994, SSA reported that 90
percent of the respondents who used the 800 number said that being on
hold for no more than 2.3 minutes would be good service.
--------------------
\6 This definition of access actually encompasses other possible call
outcomes that occur under the demonstration: callers who connect but
hang up after hearing the greeting, callers who dial the extension of
an SSA staff member they know, and callers who connect and select the
option to use the automated services. Because the system being
tested requires callers to make a choice in service options, it was
not possible to measure the occurrence of these individual events.
MORE CALLS HAVE REACHED SSA
AT METHOD B OFFICES
---------------------------------------------------------- Letter :3.1
Table 2 compares how telephone access changed with the installation
of new equipment at method A and B offices. It shows that method B
offices had an improvement of 23 percentage points in the calls
reaching SSA and that this change was large enough to be
statistically significant. The method A configuration did not
produce a statistically significant change in access under our test.
Table 2
Changes in Telephone Access Under
Methods A and B Before and After
Equipment Installation for Demonstration
Project
Change
(after
minus
before
Call outcome Before After )
---------------------------------------------- ------ ------ ------
Method A
----------------------------------------------------------------------
Access 22.1 27.2 5.1
No access 77.9 72.8 (5.1)
Method B
----------------------------------------------------------------------
Access 23.5 46.5 23.0\a
No access 76.5 53.5 (23.0)
\a
----------------------------------------------------------------------
\a Indicates that the change is statistically significant. This
means that we are 95 percent confident that an actual change occurred
and that it was in the direction indicated.
Examining the results of our analysis by call outcomes provides a
better understanding of the changes occurring under the demonstration
project. As shown in table 3, the installation of the new equipment
and additional telephone lines has resulted in a large drop in busy
signals. After installation, busy signals dropped at method B
offices by 55.2 percentage points. The large increase in the number
of callers receiving the "call later message" after installation of
the new equipment probably accounts, in part, for the drop in
busy-signal rates.
Table 3
Call Outcomes Under Each Tested Method
by GAO's Definition of Access Before and
After Equipment Installation for
Demonstration Project
Change Change
(after (after
minus minus
before before
Call outcome Before After ) Before After )
---------------------- ------ ------ ------ ------ ------ ------
Access
----------------------------------------------------------------------
No time spent on hold 20.1 1.4 (18.7) 21.5 9.4 (12.1)
\a \a
On hold less than 2 2.0 25.8 23.8\a 1.9 37.1 35.2\a
minutes
======================================================================
Subtotal 22.1 27.2 5.1 23.5\b 46.5 23.0\a
,b
No access
----------------------------------------------------------------------
Busy signals 60.3 54.1 (6.2) 67.8 12.6 (55.2)
\a
No answer 10 rings 12.2 (12.2) 0.3 0.6 0.3
\a
Call later message 2.3 (2.3) 6.4 23.9 17.5\a
Disconnected 1.4 1.4 1.0 1.0
On hold more than 2 3.1 17.3 14.2\a 1.9 15.5 13.6\a
minutes
======================================================================
Subtotal 77.9 72.8 (5.1) 76.5 53.5 (23.0)
\a,b
----------------------------------------------------------------------
\a Indicates that the change is statistically significant. This
means that we are 95 percent confident an actual change occurred and
that it was in the direction indicated.
\b Does not add because of rounding.
The other substantial change shown in table 3 relates to calls placed
on hold. The table shows two categories for calls placed on hold:
on hold less than 2 minutes and on hold more than 2 minutes. The
percent of calls in both of these categories increased greatly under
the demonstration. With newer equipment, more telephone lines, and a
constant level of staff assigned to answer these calls, the
additional calls reaching SSA are being placed on hold until staff
can answer them.
Examining how access changed at each office within methods A and B
showed mixed results. For example, tables 4 and 5 show that 3 of the
10 method A offices and 4 of the 9 method B offices showed
statistically significant improvement in access. However, five of
the method A offices and the five remaining method B offices showed
no significant change in access. Furthermore, two method A offices
also showed statistically significant declines in telephone access
rates. Local factors such as call volumes, the number of telephone
lines available, and staffing issues may account for the wide
variation in access rates at the office level.
Table 4
Analysis of Changes in Access Rate at
Method A Offices Before and After
Equipment Installation for Demonstration
Project
Change
(after
Access Access minus
rate rate before
Method A office before after )
---------------------------------------------- ------ ------ ------
American River, Cal. 0.0 0.0 0.0
Manchester, N.H. 71.9 21.9 (50.0)
\a
Flatbush, N.Y. 6.1 3.0 (3.1)
Newport News, Va. 2.9 2.9 0.0
Knoxville, Tenn. 6.7 43.3 36.6\a
Las Vegas, Nev. 0.0 2.6 2.6
Harlingen, Tex. 21.1 52.6 31.5\a
Sioux City, Ia. 47.5 65.0 17.5
Cheyenne, Wyo. 33.3 61.5 28.2\a
Oakland, Cal. 33.3 10.0 (23.3)
\a
======================================================================
Aggregate total 22.1 27.2 5.1
----------------------------------------------------------------------
\a Indicates that the change is statistically significant. This
means that we are 95 percent confident that an actual change occurred
and that it was in the direction indicated.
Table 5
Analysis of Changes in Access Rate at
Method B Offices Before and After
Equipment Installation for Demonstration
Project
Change
(after
Access Access minus
rate rate before
Method B office before after )
---------------------------------------------- ------ ------ ------
Pocatello, Ida. 21.1 78.9 57.8\a
Attleboro, Mass. 9.4 6.3 (3.1)
Albany, N.Y. 12.1 57.6 45.5\a
Petersburg, Va. 0.0 48.6 48.6\a
Asheville, N.C. 0.0 20.0 20.0\a
West Indianapolis, Ind. 60.0 73.5 13.5
El Dorado, Ark. 31.6 36.8 5.2
Norfolk, Neb. 60.0 70.0 10.0
Stockton, Cal. 3.3 10.0 6.7
======================================================================
Aggregate total 23.5 46.5 23.0\a
----------------------------------------------------------------------
\a Indicates that the change is statistically significant. This
means that we are 95 percent confident that an actual change occurred
and that it was in the direction indicated.
We recognize that a caller placed on hold (rather than receiving a
busy signal) can be considered successful access to SSA. In fact,
SSA considers access to its 800 number successful when a caller is
connected to SSA regardless of whether the caller has spoken with a
representative, heard a recorded message, spent a long period of time
on hold, or hung up while on hold. Analyzing our data using this
broader interpretation of access, we found that statistically
significant improvement occurred under both methods A and B. These
results are shown in table 6.
Table 6
Changes in Telephone Access Under
Methods A and B When All Calls Placed on
Hold Are Considered Successful Access to
an SSA Representative
Change
(after
minus
before
Call outcome Before After )
---------------------------------------------- ------ ------ ------
Method A
----------------------------------------------------------------------
Access 25.2 44.5 19.3\a
No access 74.8 55.5 (19.3)
\a
Method B
----------------------------------------------------------------------
Access 25.4 61.9 36.5\a
No access 74.6 38.1 (36.5)
\a
----------------------------------------------------------------------
\a Indicates that the change is statistically significant. This
means that we are 95 percent confident that an actual change occurred
and that it was in the direction indicated.
Using this definition of access, on an office level, we noted
additional improvements. Among method A offices, significant
improvement in access occurred in 1 more office--4 of the 10 offices
instead of 3 of the 10 offices improved. Among method B offices,
significant improvement in access occurred in three additional
offices--seven of the nine method B offices improved instead of four
of the nine offices.
VOICE MAIL EQUIPMENT HAS
IMPROVED OFFICE EFFICIENCY
AND PUBLIC SERVICE
---------------------------------------------------------- Letter :3.2
Staff at all demonstration offices had voice mail installed on their
desk telephones. We visited 12 of the 30 demonstration offices and
met with office managers and staff using the new equipment. Overall,
we heard almost universal praise about how the voice mail feature
improved office operations and enhanced customer service.
All 12 of the office managers we interviewed were enthusiastic about
the new equipment's voice mail feature. Seven of the 12 managers
told us that the voice mail equipment increased their claims
representatives' efficiency. Other managers told us that the voice
mail equipment added flexibility to their offices and improved
customer service. Finally, all of these managers told us that
feedback they have received from the public about the new voice mail
equipment has been positive.
We also interviewed 71 staff members who use the voice mail
equipment. Most of these staff members told us that the new
equipment has improved service to the public by making it easier to
reach SSA. They said that when a caller tries to reach a specific
SSA representative who is not at his or her desk, the caller can
leave a message on the staff person's voice mail. Furthermore, many
of the staff members we interviewed told us that voice mail has
enabled them to manage their workload better and has increased their
productivity. Some of these staff also told us that they no longer
worry about losing messages or receiving inaccurately recorded
messages. Others said that with voice mail, callers can leave
messages and information needed for processing a claim. This
eliminates the need for repeated calls between SSA and the public,
speeding up the claims process.
SSA'S INTERNAL EVALUATIONS OF
THE DEMONSTRATION PROJECT
------------------------------------------------------------ Letter :4
Two separate SSA organizational entities are evaluating the telephone
demonstration project. SSA's Office of Workforce Analysis (OWA) is
evaluating the equipment's effect on office productivity and employee
reactions. The Office of Program Integrity Reviews (OPIR) is
evaluating public reaction to the equipment. Neither SSA study had
been finished as of early February 1996.
OWA STUDY: OBJECTIVES AND
METHODOLOGY
---------------------------------------------------------- Letter :4.1
OWA's study has two basic objectives, determining the equipment's
effect on productivity levels and identifying employee experiences
and reactions to using the equipment. To measure the new equipment's
effect on productivity, OWA planned to gather and compare certain
data. For example, OWA planned to examine how busy-signal rates and
call volumes have changed using data obtained from the telephone
companies servicing the demonstration offices. OWA also planned to
measure the amount of work generated by callers using the automated
services option (reporting address changes or missing checks). It
has directed local offices to prepare weekly reports on the number of
callers using these services.
To examine employee reactions, OWA has planned to have field office
managers and staff who answer the telephones fill out a short
questionnaire. The questionnaire is soliciting information about how
well the system has performed and respondents' views on ease of use
and training adequacy.
OPIR STUDY: OBJECTIVES AND
METHODOLOGY
---------------------------------------------------------- Letter :4.2
To obtain information about the public's reaction to the new
equipment, OPIR planned to install caller ID equipment at 19 of the
30 demonstration offices. Offices with caller ID are to record the
phone numbers on certain dates. OPIR prepared several different
questionnaires for its staff to use when contacting callers. OPIR
planned to contact 1,500 callers, 500 for each equipment
configuration but has encountered complications. Its report is to be
finished in February 1996.
CONCLUSIONS
------------------------------------------------------------ Letter :5
Overall, the addition of new equipment and telephone lines has
demonstrated that access to SSA offices can be improved. Even if SSA
does not devote additional staff to answering telephones in local
offices, this technology may help improve the efficiency and
effectiveness of the agency's service to the public. To fully
evaluate whether to install the demonstration phone equipment in
other locations, however, an important consideration for SSA will be
the public's and SSA employees' views along with the equipment's
relative costs and contributions to meeting SSA's public service
goals.
AGENCY COMMENTS
------------------------------------------------------------ Letter :6
SSA commented on a draft of this report in a letter dated January 29,
1996 (see app. II). SSA agreed with our findings that enhanced
technology has increased the public's telephone access to field
offices. It also agreed with our view that a full evaluation of
productivity issues and employee acceptance of and public reaction to
the new equipment is needed before installation of this equipment
across the board. SSA noted that its internal studies on these
issues will be completed by the end of February 1996.
---------------------------------------------------------- Letter :6.1
Copies of this report are being sent today to SSA and parties
interested in Social Security matters. Copies will be made available
to others upon request. If you have any further questions, please
contact me on (202) 512-7215. GAO contacts and staff who prepared
this report are listed in appendix III.
Jane L. Ross
Director, Income Security Issues
OBJECTIVES, SCOPE, AND METHODOLOGY
=========================================================== Appendix I
The objective of our review was to determine if the installation of
the new telephone equipment has improved the public's access to the
participating offices in SSA's demonstration project. To do this, we
placed phone calls to offices before and after installation of the
new equipment being tested and recorded outcomes of these calls (busy
signal, placed on hold, and the like). From these outcomes, we then
calculated access rates.
As noted earlier in this report, SSA installed two types of new
equipment at 30 field offices: automated attendant and voice mail.
The equipment was installed in three different configurations. We
labeled these configurations methods A, B, and C. SSA designated 10
offices to test each method. Table I.1 shows these office locations.
Table I.1
SSA Offices Participating in the
Demonstration Project by Method
Method A Method B Method C
------------------------- ------------------------- --------------------------
Manchester, N.H. Attleboro, Mass. Bangor, Me.
Flatbush, N.Y. Albany, N.Y. Geneva, N.Y.
Newport News, Va. Petersburg, Va. Reading, Penn.
Knoxville, Tenn. Asheville, N.C. Charleston, S.C.
St. Paul, Minn. West Indianapolis, Ind. Cedartown, Ga.
Harlingen, Tex. El Dorado, Ark. Champaign, Ill.
Sioux City, Ia. Norfolk, Neb. Oklahoma City, Okla.
Cheyenne, Wyo. Murray, Utah Roswell, N.M.
Oakland, Cal. (D/T) Stockton, Cal. Winfield, Kans.
American River, Cal. Pocatello, Ida. Las Vegas, Nev.
--------------------------------------------------------------------------------
We conducted the preinstallation phase of the test from mid-January
through the end of February 1995, placing our calls on what we
believed to be the 8 busiest days during that period. We reasoned
that the best way to measure changes in phone access was to test
performance on the busiest calling days rather than on average
calling days.
To identify the busiest calling days, we used information on
telephone call volume to the 800 number during the same period in
1994. SSA has information that tracks the busy-signal rate for the
800 number. Using these data, we identified the 8 busiest days from
mid-January through the end of February in 1994. We chose this
period because SSA began installation of the new equipment at the 30
offices during the last week of February 1995.
The busiest days tended to be Mondays, Fridays, the third of the
month (when Social Security checks are normally delivered), and the
day after a holiday. The exact days we chose for study were January
17 and 30 and February 1, 3, 6, 7, 21, and 27.
SSA had planned to complete installation of the phone service by June
1995. However, it encountered several installation problems. By
late July, only one office did not yet have the equipment installed.
We decided to give the field offices some time to become acquainted
with the equipment. By using the 1994 call log for SSA's 800 number,
we selected the following 8 days on which to conduct the
postinstallation phase calls: August 22, 29, and 30 and September 5,
6, 8, 11, and 13.
SAMPLING PROCEDURE
--------------------------------------------------------- Appendix I:1
We designed the test using statistical sampling principles so that
calls would be randomly distributed throughout the day and across the
30 SSA offices during each of the two 8-day test periods. To provide
an adequate level of precision for our estimates of the busy-signal
rates, we made 350 preinstallation calls and 350 postinstallation
calls for each of the three methods being tested.
To determine the time of the calls, we divided the workday into 28
15-minute segments (beginning at 9 a.m. and ending at 4 p.m.). This
created 224 time periods over the 8-day test period (8 days times 28
time periods per day). Since 10 locations could be called during
each of the 224 time periods, we had a total of 2,240 possible
time/location combinations, with each representing a possible
telephone call.
We numbered these combinations 1 through 2,240, with number 1
assigned to the combination of the first location and the first time
period (9:00 to 9:15 a.m.) of the first of the 8 days, and number
2,240 assigned to the combination of the tenth location and the last
time period (3:45 to 4:00 p.m.) on the eighth day. We then picked at
random 350 of the numbers from 1 to 2,240. For each number picked,
we looked up the corresponding time/location combination that had
been assigned that number and placed a telephone call at that time to
that location.
For example, one of the random numbers we picked was 572. We had
assigned that number to location number 2 during the 9:15 to 9:30
a.m. period on the third day. As shown in table I.1, location
number 2 for method A is the Flatbush office. Therefore, we placed a
call to Flatbush during the 9:15 to 9:30 a.m. period on the third
day. We also placed calls during the same period on the same day to
the Albany and Geneva offices, locations number 2 under methods B and
C.
For the postinstallation period, we placed an identical set of calls,
in time and location, to those placed to estimate the busy-signal
rates before installation of the new equipment. For example, since
we had picked the number 572 we again placed calls to the Flatbush,
Albany, and Geneva offices during the 9:15 to 9:30 a.m. period on
the third day of our postinstallation test.
We used the same set of 350 random numbers for both our pre- and
postinstallation tests of the equipment to make our comparisons of
changes in the three methods' access rates as fair as possible. By
placing the preinstallation test calls on the same days and at the
same times to each of the three groups of 10 locations, we hoped to
minimize the effect on our estimates of variation among locations in
the volume of calls received on particular days or during particular
hours. Similarly, by placing our postinstallation test calls to the
same locations and at the same times as those of our preinstallation
test calls, we attempted to minimize the effect of variation among
locations in the general call volume between the mid-January through
February period and the period of our postinstallation test in August
and September.
ADJUSTMENTS TO OUR SAMPLING
PLANS
--------------------------------------------------------- Appendix I:2
Several events arose during our analysis that necessitated adjusting
the data for study purposes. Table I.2 summarizes these adjustments.
Table I.2
Adjustments to SSA Offices Participating
in the Demonstration Project by Method
Method A Method B Method C
------------------------- ------------------------- --------------------------
Manchester, N.H. Attleboro, Mass. Bangor, Me.\a
Flatbush, N.Y. Albany, N.Y. Geneva, N.Y.
Newport News, Va. Petersburg, Va. Reading, Penn.
Knoxville, Tenn. Asheville, N.C. Charleston, W.Va.
St. Paul, Minn.\a West Indianapolis, Ind. Cedartown, Ga.
Harlingen, Tex. El Dorado, Ark. Champaign, Ill.
Sioux City, Ia. Norfolk, Neb. Oklahoma City, Okla.
Cheyenne, Wyo. Murray, Utah\a Roswell, N.M.
Oakland, Cal. (D/T) Stockton, Cal. Winfield, Kans.
American River, Cal. Pocatello, Ida. Las Vegas, Nev.\a
--------------------------------------------------------------------------------
\a Affected demonstration office.
Due to unforeseen events, we could not complete our comparison
exactly as planned. Some of the field offices had to be dropped from
the study or moved to another method.
SSA did not install new equipment in the St. Paul or Bangor field
offices as had been planned. Therefore, we excluded St. Paul and
Bangor from our study. We also discovered that the phone number we
had used in the first phase of the study for the Murray field office
was incorrect so we excluded this office from our analysis. Finally,
the Las Vegas field office, which was to receive equipment for method
C, instead received the equipment for method A. These adjustments
resulted in 10 field offices using method A, 9 field offices using
method B, and 8 field offices using method C in our analyses.
For each method, we estimated the proportion of times that the public
would have accessed SSA when calling the offices in the test during
the 8 days on which we placed calls. Because our estimates--which
apply only to the 8 days on which we placed calls--are based on a
limited number of phone calls, each estimate has an associated
sampling error. At the 95-percent confidence level, sampling errors
for our estimates of access rates under each method (both pre- and
postinstallation) are about 5 percentage points. Sampling errors for
our estimates of changes in access rates under each method are about
7 percentage points. In many instances, sampling errors for
estimates of access rates at individual offices are substantially
higher.
QUESTIONNAIRE
--------------------------------------------------------- Appendix I:3
We designed a simple computer-assisted telephone interview to collect
the data on the outcome of each telephone call attempt. The
information collected included whether (1) we got a busy signal, (2)
the phone rang without being answered (we hung up after 10 rings),
(3) a person answered, (4) we were placed on hold (we waited 2
minutes before hanging up), and (5) we were disconnected.
(See figure in printed edition.)APPENDIX II
COMMENTS FROM THE SOCIAL SECURITY
ADMINISTRATION
=========================================================== Appendix I
(See figure in printed edition.)
GAO CONTACTS AND STAFF
ACKNOWLEDGMENTS
========================================================= Appendix III
GAO CONTACTS
Roland H. Miller III, Assistant Director, (202) 512-7246
William J. Staab, Evaluator-in-Charge, (202) 512-6814
STAFF ACKNOWLEDGMENTS
In addition to those named above, the following individuals made
important contributions to this report: Jim Wright and Jay Smale
developed our sample design and the computer-assisted interview
instrument used to record telephone call outcomes; Inez Azcona, and
Jeffrey Bernstein collected the data, visited local SSA offices and
helped prepare this report; Wayne Turowski and Steve Machlin did the
computer programming and analysis of data.
*** End of document. ***