SSA Disability Redesign: Actions Needed To Enhance Future Progress
(Chapter Report, 03/12/99, GAO/HEHS-99-25).

Pursuant to a congressional request, GAO: (1) reviewed the Social
Security Administration's (SSA) efforts to redesign its disability
claims process; and (2) identified actions that SSA could take to better
ensure future progress.

GAO noted that: (1) even with its scaled-back plan, SSA has been unable
to keep its redesign activities on schedule and to demonstrate that its
proposed changes will significantly improve the claims process; (2)
while SSA has made some progress, overall, it has not met most of the
milestones for testing or implementing its five near-term initiatives;
(3) moreover, its stand-alone tests of the two new decisionmaker
positions consumed valuable resources, and provided marginal or
inconclusive results; (4) as a result, the tests did not support the
wider implementation of the positions; (5) the inability to keep on
schedule was caused, in part, by SSA's overly ambitious plan and its
strategy for testing proposed changes; (6) other problems with the
design of its tests weakened SSA's ability to predict how the
initiatives would operate if implemented; (7) the problems that led to
SSA's redesign effort persist, and as SSA continues its efforts to
improve the disability claims process the agency has had an opportunity
to learn from its experience and the best practices of other
organizations with reengineering experience; (8) to its credit, SSA has
already taken steps in response to the problems encountered in its
redesign effort by, among other things, strengthening its executive
oversight; (9) however, delays and limited progress to date suggest that
these steps, while important, are not enough to ensure success; (10) SSA
could improve its chances of making future progress by further scaling
back its near-term efforts to include only initiatives that are critical
to improving the disability claims process; (11) in addition, by testing
related process changes together, rather than on a stand-alone basis,
and at a smaller number of sites, SSA could free up resources while
still obtaining valuable data; (12) experiences of other organizations
suggest that SSA faces perhaps its greatest challenge after it completes
testing of a process change and begins to implement it in a real world
environment; (13) because a process change might function differently
under actual operational conditions than it did in a test environment,
SSA will need to take additional action to ensure that it achieves
hoped-for results; and (14) SSA will need to ensure that an adequate
quality assurance process is in place so that any changes SSA makes to
the process do not compromise the quality of decisions.

--------------------------- Indexing Terms -----------------------------

 REPORTNUM:  HEHS-99-25
     TITLE:  SSA Disability Redesign: Actions Needed To Enhance Future 
             Progress
      DATE:  03/12/99
   SUBJECT:  Claims processing
             Social security benefits
             Performance measures
             Reengineering (management)
             Disability benefits
             Eligibility determinations
IDENTIFIER:  Social Security Disability Insurance Program
             SSI
             SSA Disability Process Redesign
             SSA Program Operations Manual System
             Supplemental Security Income Program
             
******************************************************************
** This file contains an ASCII representation of the text of a  **
** GAO report.  Delineations within the text indicating chapter **
** titles, headings, and bullets are preserved.  Major          **
** divisions and subdivisions of the text, such as Chapters,    **
** Sections, and Appendixes, are identified by double and       **
** single lines.  The numbers on the right end of these lines   **
** indicate the position of each of the subsections in the      **
** document outline.  These numbers do NOT correspond with the  **
** page numbers of the printed product.                         **
**                                                              **
** No attempt has been made to display graphic images, although **
** figure captions are reproduced.  Tables are included, but    **
** may not resemble those in the printed version.               **
**                                                              **
** Please see the PDF (Portable Document Format) file, when     **
** available, for a complete electronic file of the printed     **
** document's contents.                                         **
**                                                              **
** A printed copy of this report may be obtained from the GAO   **
** Document Distribution Center.  For further details, please   **
** send an e-mail message to:                                   **
**                                                              **
**                                            **
**                                                              **
** with the message 'info' in the body.                         **
******************************************************************


Cover
================================================================ COVER


Report to the Chairman, Subcommittee on Social Security, Committee on
Ways and Means, House of Representatives

March 1999

SSA DISABILITY REDESIGN - ACTIONS
NEEDED TO ENHANCE FUTURE PROGRESS

GAO/HEHS-99-25

Disability Claims Process Redesign

(207007)


Abbreviations
=============================================================== ABBREV

  AC - Appeals Council
  ALJ - administrative law judge
  AO - adjudication officer
  DCM - disability claims manager
  DDS - disability determination service
  DI - Disability Insurance
  FPM - full process model
  HHS - Department of Health and Human Services
  HO - hearing office
  NPRM - notice of proposed rulemaking
  OHA - Office of Hearings and Appeals
  OQA - Office of Quality Assurance
  PDI - predecision interview
  POMS - Program Operations Manual System
  QA - quality assurance
  RDS - Redesigned Disability System
  RFC - residual functional capacity
  SDM - single decision maker
  SSA - Social Security Administration
  SSI - Supplemental Security Income
  SSR - Social Security Ruling

Letter
=============================================================== LETTER


B-277774

March 12, 1999

The Honorable E.  Clay Shaw
Chairman
Subcommittee on Social Security
Committee on Ways and Means
House of Representatives

Dear Mr.  Chairman: 

This report responds to your Subcommittee's request that we (1)
assess SSA's efforts to redesign its disability claims process and
(2) identify actions that SSA could take to better ensure future
progress. 

We are sending copies of this report to The Honorable Kenneth S. 
Apfel, Commissioner of Social Security; appropriate congressional
committees; and other interested parties.  We will also make copies
available to others upon request. 

Please contact me on (202) 512-7215 if you or your staff have any
questions concerning this report.  Other GAO staff who contributed to
this report are listed in appendix V. 

Sincerely yours,

Cynthia M.  Fagnoni
Director, Income Security Issues


EXECUTIVE SUMMARY
============================================================ Chapter 0


   PURPOSE
---------------------------------------------------------- Chapter 0:1

In administering its disability programs, the Social Security
Administration (SSA) is still struggling to cope with workloads that
resulted, in part, from a dramatic growth in the number of
applications for disability benefits in the early 1990s.  In the
3-year period between fiscal years 1991 and 1993, initial claims for
these benefits climbed by almost one-third--from 3 million to 3.9
million.  As a result, SSA began experiencing increased difficulty
processing disability claims in a timely manner.  For example,
claimants who were dissatisfied with their initial determination and
filed an appeal often had to wait as long as 1-1/2 years for a final
decision.  Moreover, as many as two-thirds of claimants who filed an
appeal eventually received a favorable decision at the hearing level,
which indicates potential problems with either initial or appellate
decisions and raises questions about the fairness and efficiency of
the process.  SSA concluded that the only way to effectively respond
to these problems was to fundamentally overhaul the process for
deciding whether or not a claimant is eligible for disability
benefits.  In 1994, SSA set forth an ambitious plan to redesign the
process over a 6-year period. 

Concerned about the need to improve SSA's disability claims process,
the Chairman of the Subcommittee on Social Security, House Committee
on Ways and Means, asked GAO to (1) assess SSA's efforts to redesign
its disability claims process and (2) identify actions that SSA could
take to better ensure future progress. 


   BACKGROUND
---------------------------------------------------------- Chapter 0:2

Depending on the number of times a claimant files an appeal, SSA's
process can include up to four decision points:  (1) the initial
disability determination; (2) a second independent review, called
reconsideration; (3) a hearing decision by an administrative law
judge; and (4) a review of the hearing decision by SSA's Appeals
Council--an independent review group composed of administrative
appeal judges.  The process involves a large number of diverse
components, including SSA's 1,298 field offices, where initial claims
are taken; 54 state agencies, where doctors and examiners work as
teams to make initial and reconsidered determinations on medical
eligibility; and 140 hearing offices, where attorneys and
administrative law judges consider appeals. 

With its September 1994 redesign plan, SSA hoped to achieve five
goals that would improve the process, including making it more
efficient and user-friendly.  The plan originally included 83
initiatives to be accomplished over 6 years--38 of which were to be
completed within the first 2 years.  GAO concluded in a 1996 report
that SSA's plan was overly ambitious.  At that time, SSA had made
little progress toward meeting its goals, lacked demonstrable
results, and faced difficulties obtaining and keeping the support of
some stakeholders.  In response to these and other concerns, SSA
issued a scaled-back redesign plan in February 1997.  The new plan
focused on testing and implementing eight key initiatives--each
representing a major change to the system--within 9 years instead of
the original 6. 

Five of the eight initiatives had near-term milestones; that is, they
were to be tested, implemented, or both by the close of fiscal year
1998.  Two of these initiatives involved establishing new
decisionmaker positions intended to help make disability decisions
faster and more efficiently.  Each of these new positions was to be
tested in a "stand-alone" fashion--that is, not in the context of
other proposed and related changes.  A third initiative combined in a
single test the two new decisionmaker positions and several other
process changes, including a proposal to eliminate the
reconsideration and Appeals Council review levels.  The two remaining
initiatives with near-term milestones aimed to provide essential
supports for the entire disability claims process--one by improving
the consistency of disability decisions between the initial and
appellate levels of the process and the other by improving SSA's
quality assurance process. 


   RESULTS IN BRIEF
---------------------------------------------------------- Chapter 0:3

Even with its scaled-back plan, SSA has been unable to keep its
redesign activities on schedule and to demonstrate that its proposed
changes will significantly improve the claims process.  While SSA has
made some progress, overall, it has not met most of the milestones
for testing or implementing its five near-term initiatives. 
Moreover, its stand-alone tests of the two new decisionmaker
positions consumed valuable resources and provided marginal or
inconclusive results.  As a result, the tests did not support the
wider implementation of the positions.  The inability to keep on
schedule was caused, in part, by SSA's overly ambitious plan, which
involved numerous large tests and required SSA to move forward on
many fronts simultaneously.  In addition, SSA's strategy for testing
proposed changes independently, rather than together with other
related changes and key supports, contributed to the disappointing
test results.  Finally, other problems with the design of its tests
further weakened the agency's ability to predict how the initiatives
would operate if implemented. 

The problems that led to SSA's redesign effort persist, and as SSA
continues its efforts to improve the disability claims process the
agency has an opportunity to learn from its experience and the best
practices of other organizations with reengineering experience.  To
its credit, SSA has already taken steps in response to the problems
encountered in its redesign effort by, among other things,
strengthening its executive oversight.  However, delays and limited
progress to date suggest that these steps, while important, are not
enough to ensure success.  SSA could improve its chances of making
future progress by further scaling back its near-term efforts to
include only initiatives that are critical to improving the
disability claims process.  In addition, by testing related process
changes together, rather than on a stand-alone basis, and at a
smaller number of sites, SSA could free up resources while still
obtaining valuable data.  Moreover, experiences of other
organizations suggest that SSA faces perhaps its greatest challenge
after it completes testing of a process change and begins to
implement it in a "real world" environment.  Because a process change
might function differently under actual operational conditions than
it did in a test environment, SSA will need to take additional action
to ensure that it achieves hoped-for results.  For example, SSA will
need to revise its performance measures to better monitor and more
fully assess the impact of changes on the entire process.  Further,
SSA will need to ensure that an adequate quality assurance process is
in place so that any changes SSA makes to the process do not
compromise the quality of decisions. 


   PRINCIPAL FINDINGS
---------------------------------------------------------- Chapter 0:4


      PROGRESS HAS BEEN HINDERED
      BY REDESIGN STRATEGY
-------------------------------------------------------- Chapter 0:4.1

Even under its revised plan, SSA has made limited progress in
redesigning its disability claims process.  As of October 1998, the
agency was behind schedule on all five of its near-term initiatives. 
SSA postponed implementation of its two new decisionmaker positions,
is behind schedule with its evaluation of its integrated test, and
was behind on its efforts to improve its quality assurance process. 
While the agency is also behind schedule in addressing the high
number of claims that are allowed on appeal, it has made some
progress in this area.  For example, SSA provided uniform training to
over 15,000 decisionmakers from all components of the disability
claims process.  Agency officials believe this training and other
related efforts have contributed to 90,000 people receiving deserved
benefits earlier in the process.  However, other factors can also
influence the decision-making process, and it is difficult to isolate
the effects of SSA's efforts with certainty. 

In addition to missing milestones, SSA has not clearly demonstrated
the efficacy of its proposed changes.  Its stand-alone tests of the
two proposed decisionmaker positions did not demonstrate clear
improvements to the process and, in one case, produced unreliable
results.  Consequently, SSA decided to wait for data from its
integrated test--which will show the efficacy of these two positions
in combination with other process changes--before making
implementation decisions.  Preliminary results from this test are
more promising for some, but not all, of the proposed changes. 

SSA's limited progress under its revised plan is due at least in part
to the plan's overly ambitious scope and the agency's strategy for
testing its proposed process changes.  SSA's approach of moving ahead
on many fronts simultaneously--including conducting several large
tests--was difficult to manage within established time frames.  For
example, in fiscal year 1998, SSA had five tests ongoing at over 100
sites involving over 1,000 test participants.  Our field visits to a
variety of sites and our discussions with stakeholder groups
underscored the challenges of keeping the redesign effort on
schedule.  Each test included time-consuming activities, such as
coordinating the activities of many state and federal offices and
building consensus among stakeholder groups.  By diluting the
redesign team's energies among so many different tests, SSA limited
its ability to keep its plan on track.  In addition, SSA's decision
to conduct stand-alone tests contributed to disappointing and
inconclusive results because key supports and related initiatives
were not in place during the tests.  Finally, other limitations in
SSA's test design and management made it difficult for SSA to assess
how an initiative would operate if actually implemented.  For
example, in one test of a new decisionmaker position, SSA did not
ensure that test cases and control cases were similarly handled; as a
result, SSA's test results were not meaningful. 


      SSA'S EXPERIENCE AND BEST
      PRACTICES SUGGEST ACTIONS TO
      INCREASE CHANCES OF FUTURE
      SUCCESS
-------------------------------------------------------- Chapter 0:4.2

The problems that led to SSA's redesign effort persist, and future
workloads are likely to increase as a result of demographic changes. 
As a result, the agency must continue to seek ways to improve its
work processes, enhance its workforce skills, and make more effective
use of technology.  As it proceeds with steps to improve its
disability claims process, it has an opportunity to learn from its
past mistakes.  To its credit, SSA has begun to apply some lessons it
has learned.  For example, it created a new executive-level committee
to oversee progress and direct resources and energies where they are
most needed.  But the extent of the problems experienced up to this
point suggest that this and other steps taken by SSA will probably
not be enough to keep its redesign plan on track, and more
significant changes in its reengineering approach will probably be
needed.  Relatedly, SSA will need more concrete and positive results
to assuage the persistent concerns of stakeholders regarding the
wisdom of specific process changes. 

Reengineering experts agree that successful reengineering efforts
require sharp focus, and SSA's own experience confirms that
additional focus is needed.  As with its original redesign plan,
SSA's 1997 redesign plan called for moving forward on so many fronts
that SSA continued to miss milestones and experience difficulty in
making concrete improvements to the process.  Prioritizing SSA's five
key redesign objectives would provide additional focus, and
concentrating the agency's efforts on the initiatives most likely to
achieve critical redesign objectives would improve its chances of
making concrete progress in crucial areas. 

SSA's testing approach has not been adjusted to capture lessons
learned from SSA's previous experiences and commonly accepted
reegineering practices.  For example, SSA's planned test of yet a
third decision-making position involves many sites and does not
include a number of key supporting initiatives, such as the
simplified decision methodology and redesigned computer system. 
Conducting this test at fewer sites would free up resources and help
SSA keep its initiatives on track.  SSA could also reduce the
likelihood of disappointing test results by developing these key
supports before testing the initiative.  Also, before investing
significant resources on testing this particular initiative, SSA
could explore alternatives on a small scale, so that feasible options
would be available if this proposed initiative proved ineffective or
difficult to implement. 

SSA is currently considering whether to broadly implement certain
proposed changes to the claims process.  The experiences of other
public and private organizations that have attempted to significantly
change a process indicate that implementing change is more difficult
than testing it.  Organizations naturally resist change, and new
processes may function differently when they are no longer operating
in a test environment.  Therefore, SSA will need to closely monitor
the results of changes that are implemented to determine whether the
changes are achieving the intended improvements to the process.  To
do so, SSA will need a more complete set of performance goals and
measures than it currently has.  It will also need a fully developed
quality assurance process before it implements major system changes. 
The quality assurance process is especially important because of
uncertainty that the newly tested decisionmaker positions will
perform as effectively outside the test environment as in it. 


   RECOMMENDATIONS
---------------------------------------------------------- Chapter 0:5

This report makes a number of recommendations to improve SSA's
approach to redesigning its disability claims process by focusing its
resources on those initiatives that are most likely to achieve SSA's
most critical redesign objectives, modifying its testing approach to
avoid previously experienced pitfalls, and taking steps to put in
place a comprehensive set of performance goals and measures and
adequate quality assurance procedures.  The full set of
recommendations can be found in chapter 4. 


   AGENCY COMMENTS
---------------------------------------------------------- Chapter 0:6

SSA agreed with GAO's observations that the redesign efforts to date
have not resulted in the magnitude of benefits projected in the
original redesign plan and with GAO's recommendation that SSA focus
on those areas that will make the greatest contributions to the
agency's redesign objectives.  On the other hand, SSA maintains that
its large-scale, stand-alone testing strategy was both necessary and
effective, and that its current approach to testing a third position
is consistent with GAO's concerns and recommendations.  However, GAO
continues to believe that SSA could make better use of limited
resources by first exploring the efficacy of this third position, and
alternative concepts, on a smaller scale.  SSA also stated that it
will continue to use its current approach and measures to monitor and
evaluate redesign changes, whereas GAO believes that a more carefully
crafted set of performance measures and goals is needed to
effectively monitor and evaluate changes.  SSA also provided
technical comments, which were incorporated as appropriate.  The full
text of SSA's comments is included as appendix IV. 


INTRODUCTION
============================================================ Chapter 1

The Social Security Administration (SSA) manages two major federal
disability programs that provide cash benefits to people with
long-term disabilities:  the Disability Insurance (DI) and
Supplemental Security Income (SSI) programs.  The DI program was
enacted in 1954 and provides monthly cash benefits to severely
disabled workers.  SSI was enacted in 1972 as an income assistance
program for aged, blind, or disabled individuals whose income and
resources fall below a certain threshold.  For both programs,
disability for adults is defined as an inability to engage in any
substantial gainful activity because of a severe physical or mental
impairment.\1 Both programs also use the same procedures for
determining whether the severity of an applicant's impairment
qualifies him or her for disability benefits.  In 1998, almost 11
million people received a total of over $73 billion in disability
benefits from these programs. 

SSA's complex process for determining whether an individual qualifies
for a disability benefit--the disability claims process--has been
plagued by a number of long-standing problems.  For example,
claimants who have been dissatisfied with the initial determination
and have filed an appeal frequently have had to wait more than 1-1/2
years for a final decision.  Moreover, as many as two-thirds of these
determinations were subsequently allowed by an administrative law
judge (ALJ).  In the early 1990s, SSA had difficulty keeping up with
a rapidly growing workload, and backlogs of appealed cases waiting
for a hearing grew.  In response to these problems, SSA concluded
that minor improvements to the disability claims process would be
insufficient and embarked on an effort to fundamentally reengineer,
or redesign, its process.\2 In 1994, the agency issued an ambitious
plan for redesigning the process within 6 years.  However, 2 years
into implementing the redesign plan, SSA had not made much progress,
and we and SSA concluded that the scope of the plan was too large. 
The agency reevaluated its approach and, in February 1997, issued a
scaled-back plan with revised milestones. 


--------------------
\1 The legal definition of disability for adults is the inability to
engage in any substantial gainful activity by reason of any medically
determinable physical or mental impairment that can be expected to
result in death or to last for at least 12 months.  A child is
considered disabled if the child has a medically determinable
physical or mental impairment that results in marked and severe
functional limitations that can be expected to result in death or to
last for at least 12 months. 

\2 SSA consistently uses "redesign" to describe its efforts to
improve the disability claims process, although it considers this
term synonymous with "reengineer." In this report, we also use
"redesign" to refer to SSA's reengineering effort. 


   PROBLEMS WITH SSA'S DISABILITY
   CLAIMS PROCESS CAME TO THE FORE
   IN THE EARLY 1990S
---------------------------------------------------------- Chapter 1:1

SSA's disability claims process has long been recognized as complex
and fragmented.  The decision about whether an individual is disabled
is based on standards set forth in the Social Security Act and
extensive SSA regulations and rulings.  Moreover, disability
decisions involve a multilevel process that spans many diverse
components, including SSA's 1,298 field offices, 54 state agencies,
and 140 hearing offices.  This organizationally complex structure has
contributed to a number of problems.  For example, through the years
a high percentage of claimants who were dissatisfied with their
initial determinations received favorable decisions on appeal. 
Claimants have also waited a long time for final decisions on their
eligibility.  In the early 1990s, these problems were aggravated by
mounting workloads, as applications for disability benefits escalated
at the same time that SSA was experiencing a decline in its
workforce.  This, in turn, caused workloads to back up and increased
the time it took claimants to receive decisions on their claims. 


      THE DISABILITY CLAIMS
      PROCESS IS COMPLICATED AND
      FRAGMENTED
-------------------------------------------------------- Chapter 1:1.1

SSA's disability claims process, which has not changed fundamentally
in over 40 years, is inherently complex and fragmented.  The process
contains several opportunities for appeal, and the organizational
unit involved, professional background of the adjudicator, and
procedures for making a decision on appeal are all different from
those of the initial determination.  Each organizational unit has
separate lines of authority and goals without responsibility for the
overall outcome of the process. 

The claims process starts when an individual contacts one of SSA's
1,298 field offices across the country to apply for benefits.  Field
office personnel help claimants complete their applications; obtain a
detailed medical and work history; and identify other, nonmedical
eligibility factors.\3 Field office personnel then forward the claims
to one of 54 disability determination service (DDS) agencies that are
administered by the 50 states and the District of Columbia, Guam,
Puerto Rico, and the Virgin Islands. 

Under a unique federal-state arrangement, SSA pays state DDSs to
determine whether claimants are disabled.\4 At the DDS, a team
consisting of a specially trained disability examiner and an agency
physician or psychologist reviews the available medical evidence and
gathers additional medical evidence, if necessary.  In making the
disability determination, the team follows official guidance found in
SSA's Program Operations Manual System (POMS), which is based on
applicable laws and SSA's regulations and rulings and also includes
detailed instructions for processing cases.  If the claimant is
dissatisfied with the initial determination, the claimant may request
a reconsideration review within 60 days of receiving the
determination.  Reconsideration is also performed by the DDSs and is
based on the same guidance as the initial determination but is
carried out by a new adjudicative team. 

If the claimant is dissatisfied with the determination, he or she has
60 days to appeal and request a hearing before an ALJ.  ALJs are
hearing officers located at 140 hearing offices around the country
that are administered by SSA's Office of Hearings and Appeals (OHA). 
ALJs review the file to determine if additional medical evidence is
needed, conduct a hearing, and render a decision.\5 ALJs conduct de
novo hearings; that is, ALJs may consider or develop new evidence and
are not bound by DDS determinations.  These hearings often present
the first opportunity for face-to-face contact between claimants and
the individuals deciding their eligibility.  In rendering a decision,
ALJs do not follow the POMS but rely directly on applicable laws and
SSA regulations and rulings.  ALJs are subject to the Administrative
Procedure Act, which affords them some independence in making a
disability decision.\6

Finally, if the ALJ denies the claim, the claimant has 60 days to
request a review by the Appeals Council, an independent review group
attached to the OHA and composed of administrative appeals judges. 
The Appeals Council may decide to dismiss the request for review,
grant the request and issue its own decision, or remand the case back
to the ALJ.  The Appeals Council is the claimant's fourth and final
level of administrative review.  Upon exhausting these administrative
remedies, the claimant may file a complaint with a federal court. 
Figure 1.1 shows the four decision points in SSA's current disability
claims process. 


   Figure 1.1:  SSA's Disability
   Claims Process as of October
   1998

   (See figure in printed
   edition.)



   (See figure in printed
   edition.)

SSA's approach to reviewing the quality of the disability decision
reflects the complex and fragmented nature of the process.  As we
have previously reported, current quality assurance reviews focus on
DDS determinations and ALJ decisions in isolation from one another,
and the approach for reviewing DDS determinations differs from the
approach for reviewing ALJ decisions.\7 Reviews of DDS determinations
are conducted by staff from SSA's Office of Quality Assurance (OQA). 
These reviews focus heavily on DI claims that have been allowed.\8 In
conducting their quality review, OQA staff use the same approach,
policy, and procedures that the DDSs use in reaching a determination;
that is, they rely on the POMS.  In contrast, only a small number of
ALJ allowance decisions are selected for review by SSA's Appeals
Council.  For the most part, reviews of ALJ decisions predominantly
consist of reviews of claims denied by ALJs and appealed by claimants
to the Appeals Council.  In reviewing ALJ decisions, the Appeals
Council relies on the same laws and SSA regulations and rulings as
those used by ALJs. 


--------------------
\3 Nonmedical eligibility factors include such considerations as
whether the applicant has paid Social Security taxes for enough years
and recently enough to be covered under Social Security for DI
benefits and whether the applicant has sufficiently low income or
resources to be eligible for SSI benefits. 

\4 This arrangement was instituted, in part, because the states had
prior experience in administering various disability-related programs
and had established working relationships with the medical community. 
DDSs are required to follow SSA policy guidelines but are not under
the direct administrative control of SSA. 

\5 Under a temporary measure directed at reducing hearing office
workloads, a request for a hearing may be reviewed and the claim
fully allowed by a senior staff attorney. 

\6 The Administrative Procedure Act protects the ALJ's independence
by restricting the extent to which SSA management can exert control
over these adjudicators.  For example, ALJ salaries are determined by
the Office of Personnel Management, not by SSA. 

\7 Social Security Disability:  SSA Must Hold Itself Accountable for
Continued Improvement in Decision-Making (GAO/HEHS-97-102, Aug.  12,
1997). 

\8 On the basis of statutory requirements, OQA staff review 50
percent of all allowed DI claims and, if they find errors, return
these claims to the DDSs for correction before the determination
becomes final.  These reviews--called pre-effectuation reviews--are
conducted in order to avoid erroneously awarded claims and to protect
the DI trust fund.  Also, SSA's OQA staff randomly select a small
percentage of claims awarded or denied by the DDSs and use the
results to compute the accuracy rates of DDS offices. 


      LONG-STANDING PROBLEMS ARE
      ASSOCIATED WITH THE CLAIMS
      PROCESS
-------------------------------------------------------- Chapter 1:1.2

SSA's disability claims process has long suffered from problems
associated with its complexity and fragmentation.  Among these
problems are the high allowance rates by ALJs of appealed DDS
determinations.  In fiscal year 1993, before SSA issued its redesign
plan, 68 percent of determinations that were appealed received
favorable decisions at the hearing level.  High ALJ allowance rates
have been attributed to a number of factors.  According to SSA, an
ALJ might arrive at a different decision than a DDS because the
claimant's condition has worsened, or because ALJs are more likely
than DDS decisionmakers to meet with claimants face-to-face, and thus
have access to more or different information.  However, SSA studies
have also found that DDS and ALJ adjudicators often arrive at
different conclusions even when presented with the same evidence.\9
Disability decisions require difficult judgments, and adjudicators
sometimes reach different conclusions.  Further, DDS and ALJ
adjudicators use medical expertise differently and rely on different
documents for guidance when making decisions.  Finally, training has
not been delivered consistently or simultaneously to all groups of
decisionmakers. 

This high rate of allowances at the hearing level has raised
questions about the fairness, integrity, and cost of SSA's disability
program.  In fiscal year 1998, the cost of making a determination at
the DDS level was $547 per case, while the cost of an ALJ decision
was an additional $1,385.  In general, the costs of administering
these disability programs reflect the demanding nature of the
process:  in fiscal year 1998, SSA spent about $4.3 billion, or
almost 66 percent of its administrative budget, on the disability
programs, even though disability beneficiaries are only 21 percent of
the agency's total number of beneficiaries. 

Another long-recognized problem with SSA's claims process is that
many claimants must wait a long time for their final decisions. 
Because of the multiple levels and decision points in the process, a
great deal of time passes while a claimant's file is passed from one
employee or office to another.  Delays are also caused by the need to
obtain extensive medical evidence from health care providers to
document the basis for disability.\10 One SSA study conducted in 1993
showed that an average claimant waited up to 155 days from initial
contact with SSA until receiving an initial determination notice,
during which time 16 to 26 employees might have handled the claim. 
Only 13 hours of these 155 days were spent on "task time"--that is,
time spent working directly on the case.  Further, the study found
that it could take up to 550 days from initial contact to receipt of
a hearing decision, with only 32 hours of this time spent on task
time.  As a result of these multiple handoffs and the general
complexity of the process, SSA believes claimants do not understand
the process and have had difficulty obtaining meaningful information
about the status of their claims. 


--------------------
\9 SSA, Office of Program and Integrity Reviews, Findings of the
Disability Hearings Quality Review Process (Washington, D.C.:  SSA,
Sept.  1994) and Secretary of Health and Human Services,
Implementation of Section 304 (g) of Public Law 96-265, Social
Security Disability Amendments of 1980 (the Bellmon Report)
(Washington, D.C.:  HHS, Jan.  1982). 

\10 According to SSA, these providers often do not understand the
requirements, find the forms confusing, or feel burdened by the
requests for evidence. 


      THE MOUNTING DISABILITY
      WORKLOAD IN THE EARLY 1990S
      EXACERBATED PROBLEMS
-------------------------------------------------------- Chapter 1:1.3

In the early 1990s, SSA's problems with its disability claims process
came to the fore as the growing workload placed additional pressure
on SSA's already inefficient process.  The number of initial claims
had been rising steadily, but it increased dramatically between
fiscal years 1991 and 1993--from about 3 million to 3.9 million, or
almost 32 percent.\11 Moreover, future increases were expected.  At
the same time, SSA had to manage this growing workload with staffing
levels that had been falling since the 1980s.  As a result, SSA's
disability workload began to accumulate during this period.  Most
dramatically, the number of pending hearings almost doubled between
1991 and 1993--from 183,471 to 357,564. 


--------------------
\11 This increase does not include applications for SSI by aged
claimants. 


   SSA'S INITIAL EFFORTS TO
   REDESIGN THE PROCESS MOVED
   SLOWLY
---------------------------------------------------------- Chapter 1:2

To address these long-standing problems and dramatically improve
customer service, SSA embarked on a plan in 1994 to radically
redesign its disability claims process by completing 83 initiatives
over 6 years.  We concluded in a 1996 report, however, that 2 years
into the plan, SSA had yet to achieve significant progress.\12 SSA's
slow progress was due in part to the overly ambitious nature of the
redesign plan, the complexity of the redesign initiatives, and
inconsistent stakeholder support and cooperation. 


--------------------
\12 SSA Disability Redesign:  Focus Needed on Initiatives Most
Crucial to Reducing Costs and Time (GAO/HEHS-97-20, Dec.  20, 1996). 


      SSA'S INITIAL REDESIGN
      EFFORTS
-------------------------------------------------------- Chapter 1:2.1

Concerned about the inefficiency of the disability claims process and
its effect on the quality of service to the public, SSA's leadership
decided in 1993 that the agency needed a strategy for radically
improving the process.  SSA reviewed reengineering efforts and
approaches in other organizations and concluded that process
reengineering was critical to achieving its strategic objective of
providing world-class service.\13 SSA then created a Disability
Process Redesign Team composed of 18 SSA and state DDS employees with
varied experience and backgrounds and charged it with fundamentally
rethinking and redesigning SSA's claims process from start to
finish.\14 Consistent with commonly held reengineering principles,
the team collected extensive information on the process itself and
options for improving it.\15 These efforts culminated in a redesign
proposal that was widely distributed throughout SSA and the state
DDSs and to interested public and private individuals and
organizations to solicit comments, concerns, and ideas for
improvement.  The proposal was also published in the Federal
Register, and a comment period elicited 6,000 written responses,
which were considered as SSA finalized its initial redesign proposal. 

In September 1994, SSA issued its vision for fundamentally
redesigning the disability claims process.  SSA's vision included
five objectives for the redesigned process:  (1) making the process
"user-friendly," (2) allowing claims that should be allowed at the
earliest possible level, (3) making the disability decision as
quickly as possible, (4) making the process efficient, and (5)
providing a satisfying work environment for employees.  SSA's vision
was based on more consistent guidance and training for all
adjudicators; an automated and simpler claim intake and appeal
process; a single, comprehensive quality review process; and a
simplified method for making disability decisions.  From the
claimant's perspective, the redesigned process was to offer a range
of options for filing a claim; provide a single point of contact; and
have fewer decision points, as shown in figure 1.2. 

   Figure 1.2:  SSA's Proposed
   Disability Claims Process

   (See figure in printed
   edition.)



   (See figure in printed
   edition.)

SSA had high expectations for its proposed redesigned process.  The
agency projected that the combined changes to the process would, by
fiscal year 1997, result in a 25-percent improvement in productivity
and customer service over projected fiscal year 1994 levels, and a
further 25 percent by the end of fiscal year 2000--all without a
decrease in decisional accuracy.  SSA did not expect the overall
redesigned process to alter total benefits paid to claimants, but it
estimated that the changes would result in administrative cost
savings of $704 million through fiscal year 2001, and an additional
$305 million annually thereafter. 

After putting forth its broad vision, SSA issued in November 1994 a
more detailed plan for developing, testing, and finally implementing
proposed disability process improvements.  The plan originally
included 83 initiatives to be accomplished over 6 years.  SSA
recognized in its implementation plan that most, if not all, of the
proposed process changes were interdependent, and that the
development, testing, and implementation of related changes would
need to be properly sequenced.  For example, SSA recognized that all
activities and associated benefits were dependent on improvements to
its computer system, which were not expected to be completed until
the end of the 6-year time frame. 


--------------------
\13 The agency decided to follow reengineering principles championed
by Michael Hammer and others at the time.  Reengineering was,
therefore, defined as the fundamental rethinking and radical
redesigning of business processes to bring about dramatic
improvements in performance. 

\14 Several aspects of the process were designated to be outside the
scope of this reengineering project--for example, the use of an ALJ
as the presiding officer at administrative hearings. 

\15 The team received briefings from staff in all components involved
in the disability claims process, visited and interviewed experts and
interested parties both within and outside SSA, held focus group
sessions with claimants, studied the claims process in successful
organizations to identify best practices and process innovations,
researched and documented the current process, and used computer
models to compare the current processes with possible alternatives. 


      PROGRESS ON THE INITIAL
      REDESIGN PLAN WAS SLOW
-------------------------------------------------------- Chapter 1:2.2

In 1996 and 1997, we issued several reports that raised concerns
regarding SSA's redesign effort.  These concerns included, among
other things, a lack of progress and demonstrable results.  For
example, we reported that SSA had not fully completed any of the 38
near-term initiatives it had hoped to accomplish in the first 2
years.\16 As a result, SSA did not have any concrete results
available to demonstrate the efficacy of its proposed initiatives. 

SSA's slow progress was due in part to its overly ambitious redesign
plan and the complexity of some of its redesign initiatives.  We
reported that SSA did not follow best practices when it decided to
take on a large number of initiatives concurrently.\17 Specifically,
we reported that successful reengineering calls for focusing on a
small number of initiatives at one time, whereas SSA decided to
tackle 38 initiatives in the first 2 years of its redesign effort. 
Moreover, some of these initiatives were large in scope and very
complex.  For example, scheduled implementation of SSA's large and
complicated initiative for redesigning its computer system was
delayed because of problems identified during testing. 

Some aspects of SSA's redesign plan faced considerable opposition. 
As part of its redesign effort, SSA had identified over 100
individual groups--both internal and external to SSA--as having a
stake in the process and whose involvement was, in many cases,
critical to the entire disability claims process.  These stakeholder
groups--which included various SSA employee unions and associations,
state entities and organizations, congressional committees, other
federal agencies, and advocacy groups--had a wide variety of views on
SSA's plan, and some opposed specific initiatives.  For example,
SSA's plan called for a new position--a disability claims manager
(DCM)--that would combine the duties of field office and DDS
personnel into one position.  The DCM represented significant change
to the current process, and SSA faced numerous challenges in
obtaining stakeholder cooperation for this key initiative. 

In light of these difficulties and in order to increase SSA's chance
of success, we recommended in our December 1996 report that SSA
reduce the scope of its redesign effort by focusing on those
initiatives considered most crucial to improving the process and
testing those initiatives together, in an integrated fashion, at a
few sites.  In another 1996 report, we recommended that, concurrent
with the first phase of its DCM test, SSA test alternatives that we
believed were more feasible and compare their relative costs and
benefits with those of the DCM before deciding to increase the number
of DCM test positions.\18

Later, we supported SSA's redesign efforts associated with its
initiative to improve the consistency of disability decision-making
and recommended, among other things, that SSA establish a performance
goal for this key redesign initiative.\19


--------------------
\16 GAO/HEHS-97-20, Dec.  20, 1996. 

\17 GAO/HEHS-97-20, Dec.  20, 1996. 

\18 SSA Disability Redesign:  More Testing Needed to Assess
Feasibility of New Claim Manager Position (GAO/HEHS-96-170, Sept. 
27, 1996). 

\19 GAO/HEHS-97-102, Aug.  12, 1997. 


   SSA SCALED BACK ITS REDESIGN
   PLAN
---------------------------------------------------------- Chapter 1:3

As a result of our input, the overall lack of progress, and
stakeholder concerns, SSA reassessed its approach to redesign and
issued a revised plan in February 1997.  The new plan focused on
eight key initiatives, each one intended to effect a major change to
the system.\20 The plan also included updated tasks and milestones
for each key initiative and expanded the time frame for the entire
redesign project from 6 to 9 years, ending in 2003.  The eight
initiatives and their milestones are described in figure 1.3. 


   Figure 1.3:  The Eight
   Initiatives and Milestones
   Under the February 1997
   Redesign Plan

   (See figure in printed
   edition.)



   (See figure in printed
   edition.)

\a See app.  I for a complete description of the eight
subinitiatives. 

As shown in figure 1.3, five of the eight initiatives had relatively
near-term deadlines--that is, before the end of fiscal year 1998--for
completing a key test or beginning implementation.  Two of these
initiatives involve testing new positions and, if test results
warrant, implementing new positions on a stand-alone basis--that is,
independently of other, related initiatives.  One new position, the
single decision maker (SDM), would expand the DDS disability
examiner's authority to determine certain claims without relying on
the DDS physician; the SDM would instead use the physician as a
consultant on an as-needed basis.  The SDM was expected to make the
initial determination process faster and more efficient by
eliminating handoffs to DDS physicians in those cases in which the
appropriate determination was clear.\21 Another new position, the
adjudication officer (AO), would review cases that were appealed to
the hearing level.  The AO was to help claimants understand the
appeals process and would have authority to grant disability benefits
in cases in which it was clear that the claim merited a fully
favorable decision.  In all other cases, the AO was to make sure that
all pertinent information was included in the case file and was fully
explained, thus facilitating its use by the ALJ at the next level of
appeal.  By performing these tasks, the AO was expected to improve
customer service and make the appeals process faster and more
efficient. 

A third near-term initiative is the full process model (FPM) test. 
The FPM combines five proposed changes into a single test to
investigate their interactive effects on creating a more efficient
process and better customer service.  The five tested changes are (1)
creating the SDM position; (2) creating the AO position; (3)
establishing a new predecision interview, in which the SDM would
interview claimants when the evidence did not support a fully
favorable determination in order to obtain any additional information
before making the final determination; (4) eliminating the
reconsideration step; and (5) eliminating the Appeals Council
step--that is, removing the claimant's option to request a review by
the Appeals Council of an ALJ decision. 

The two other near-term initiatives--process unification and quality
assurance--are considered essential elements for achieving correct
decisions in the new disability claims process.  The intent of the
process unification initiative was to achieve similar results on
similar cases at all stages of the process.  To this end, SSA planned
a number of activities, including conducting ongoing training;
clarifying policies; and developing unified guidance, called a single
presentation of policy, for making disability decisions across all
levels of the process.  SSA also planned to complete eight additional
subinitiatives--all designed to help reduce inconsistencies in
decision-making between the DDS and ALJ levels. 

SSA's quality assurance initiative included near-term activities in
two areas.  First, as part of each of the other major redesign
initiatives, SSA planned to develop and test "in-line" quality
assurance approaches--such as training, mentoring, and peer
review--in order to build quality into the process before decisions
are made.  Second, SSA planned to develop and test a single
"end-of-line" quality review mechanism that covered the entire
adjudicatory process from beginning to end and provided data on
problems or failures in a component's in-line quality assurance
process.  Appendix I provides additional information on SSA's five
near-term redesign initiatives. 


--------------------
\20 Some initiatives in the original implementation plan were
deferred.  Still others, considered to be good business practices,
were "institutionalized"; that is, SSA shifted responsibility for
implementing them from the Disability Process Redesign Team to
front-line components without further testing or development. 

\21 Under both SSA's original and revised plans, the responsibilities
of two positions, the SDM and the field office representative, would
eventually be combined into a new position, the DCM.  This position
would be the initial point of contact for claimants, as depicted in
fig.  1.2. 


   OBJECTIVES, SCOPE, AND
   METHODOLOGY
---------------------------------------------------------- Chapter 1:4

The Chairman of the House Subcommittee on Social Security, Committee
on Ways and Means, asked us to (1) assess SSA's efforts to redesign
its disability claims process and (2) identify any actions needed to
better ensure future progress.  We agreed to focus our work on the
five initiatives in SSA's scaled-back plan that have relatively
near-term dates for testing, implementation, or both:  the SDM, AO,
FPM, process unification, and quality assurance initiatives. 

In assessing SSA's redesign experience, we obtained documents from
and interviewed SSA officials responsible for planning, managing, and
evaluating redesign efforts.  We visited several DDS and hearing
office test sites and interviewed test participants and managers in
Richmond, California; Brooklyn, New York; Raleigh, North Carolina;
and Providence and Warwick, Rhode Island.  We also interviewed SSA
regional officials with responsibility for overseeing or coordinating
redesign efforts within their regions as well as representatives of
nine major stakeholder groups to obtain their views on SSA's specific
initiatives and general approach for redesign.  Finally, we reviewed
the literature and interviewed experts on business process
reengineering. 

We conducted our work between August 1997 and November 1998 in
accordance with generally accepted government auditing standards with
the following exception:  we did not independently verify agency
data, including test data on redesign initiatives.  We did obtain
information from SSA on steps it took to obtain and verify the test
data and any problems associated with them.  We have noted our
concerns regarding the validity and reliability of the data in the
report, where appropriate.  We obtained comments from SSA officials
responsible for the redesign tests, which we have summarized in
chapter 4. 


PROGRESS UNDER REVISED PLAN
HINDERED BY OVERLY AMBITIOUS
STRATEGY AND PROBLEMS WITH TEST
DESIGN
============================================================ Chapter 2

After 4 years of redesign efforts, SSA has made only limited progress
toward improving its disability claims process.  While narrowing the
focus of its redesign plan has helped, SSA has continued to miss
milestones and has not clearly demonstrated that the proposed
initiatives would significantly improve the current process.  As a
result, SSA has had to defer service improvements and reduce
estimated savings.  The agency's limited progress has resulted, in
part, from SSA's overly ambitious strategy for testing and
implementing its redesign initiatives.  Conducting a number of tests
and other redesign activities simultaneously proved to be too
difficult to keep on track.  In addition, problems with SSA's
approach to designing and managing its tests of new initiatives
contributed to marginal and inconclusive test results and made it
more difficult for SSA to discern how a tested initiative would
operate if implemented on a widespread basis. 


   PROGRESS UNDER REVISED PLAN HAS
   BEEN LIMITED
---------------------------------------------------------- Chapter 2:1

SSA has made only limited progress in improving its disability claims
process, despite having fewer initiatives in its revised redesign
plan than in the original plan.  The agency has not met most of its
adjusted milestones for testing and implementing its five near-term
initiatives.\22 Moreover, results from SSA's stand-alone tests of two
new decisionmaker positions, the SDM and the AO, were not compelling
and did not support broader implementation.  Therefore, SSA decided
to wait for preliminary results of its integrated test, which has in
fact produced some promising results.  In addition, SSA has made
progress under its process unification initiative, such as providing
training and clarifying policy, and agency officials believe the
actions taken thus far have had a positive effect on customer
service.  Overall, however, as a result of missed milestones and
disappointing test results, SSA has deferred many other process
improvements and reduced its redesign expectations for administrative
savings. 


--------------------
\22 While SSA has issued various testing and implementation schedules
over time, this report compares SSA's progress against the dates for
testing and implementing initiatives laid out in its February 1997
plan.  In many cases, these revised dates provided the agency with
additional time to complete its planned activities. 


      DELAYS CONTINUE UNDER
      REVISED PLAN
-------------------------------------------------------- Chapter 2:1.1

Even under its scaled-back plan, SSA continues to experience delays. 
As of October 1998, the agency was behind schedule on all five of the
plan's near-term initiatives.  After more than 3 years of testing,
SSA had yet to complete its test of the AO decisionmaker position
and, for reasons discussed in the next section, SSA has delayed its
decision on whether to implement both the SDM and AO decisionmaker
positions.  Also, SSA did not complete its assessment of the FPM test
results in fiscal year 1998 as scheduled. 

SSA has completed some of its planned activities under its process
unification initiative, but it has missed other key implementation
deadlines.  The agency has clarified key policies and, since 1996,
has issued policy instructions in the same format for all
adjudicators.  SSA has also provided an unprecedented training
program involving 15,000 decisionmakers and quality reviewers from
key components of the disability claims process and has adopted
process unification principles for its ongoing training program by
providing the same training to all adjudicators.  However, SSA has
experienced delays in several other planned activities.  For example,
the agency is behind schedule on a test to study the effect of
requiring DDS adjudicators to more fully document the rationale they
used in making particular disability determinations.  SSA hopes this
more detailed explanation will reduce decisional inconsistencies. 

SSA has begun work on its quality assurance initiative, but this
effort has also been delayed.  As part of its tests of other redesign
initiatives, SSA has been exploring "in-line" quality assurance
approaches--such as training and mentoring--that are intended to
build quality into the process before decisions are made.  SSA
planned to institute these practices nationwide when it implemented
the other redesign initiatives; however, delays in implementing the
initiatives have delayed the widespread use of these quality
assurance practices.  In addition, SSA is more than a year behind in
developing a single end-of-line review mechanism.\23

The agency planned to develop one quality standard for its
end-of-line reviews in fiscal year 1997 and to test its use in fiscal
year 1998.  However, as of the end of fiscal year 1998, SSA had not
reached internal agreement on what that single quality review
standard should be. 

Key milestones and the status of SSA's five near-term initiatives are
summarized in table 2.1.  Additional information on SSA's efforts to
meet near-term milestones--including specific actions taken to date
and the nature and extent of delays--can be found in appendix II. 



                         Table 2.1
          
            Milestones and Status for Near-Term
                        Initiatives

                       Milestones (through FY 1998)
                ------------------------------------------
Date for                              Status as of October
planned action  Planned action        1998
--------------  --------------------  --------------------
SDM initiative
----------------------------------------------------------
4/1997          Complete phase I      Completed 11/1997
                test and evaluation.

FY 1998         Begin                 Not started
                implementation.

FY 1998/1999    Complete phase II     Ongoing
                test.


AO initiative
----------------------------------------------------------
FY 1997         Complete test.        Not completed

1/1998          Begin nationwide      Not started
                rollout.


FPM initiative
----------------------------------------------------------
1/1998          Complete test case    Completed 1/1998
                selection.

FY 1998         Complete assessment   Not completed
                of four of five
                process changes.\a


Process unification initiative
----------------------------------------------------------
FY 1997         Complete training of  Completed FY 1997
                15,000 adjudicators
                and quality
                reviewers.

FY 1997         Implement eight       Three completed (FY
                subinitiatives.       1997 and 1998); five
                                      not completed


Quality assurance initiative
----------------------------------------------------------
FY 1997         Develop end-of-line   Not completed
                review mechanism.

FY 1997         Develop in-line QA    Completed FY 1997
                procedures to
                support testing of
                AO and SDM
                initiatives.

FY 1998         Test review           Not started
                mechanism.

FY 1998         Develop QA            Not completed
                procedures to
                support testing of
                FPM and DCM redesign
                initiatives.
----------------------------------------------------------
\a The four changes being assessed under FPM are the SDM, AO,
predecision interview, and elimination of reconsideration. 

Sources:  SSA planning documents and officials. 


--------------------
\23 As discussed in ch.  1, SSA quality reviewers currently use
different criteria for reviewing disability decisions depending on
the level at which the decision was made. 


      EFFICACY OF REDESIGN
      INITIATIVES HAS NOT YET BEEN
      FULLY DEMONSTRATED
-------------------------------------------------------- Chapter 2:1.2

As of October 1998, SSA had not clearly demonstrated that its
proposed changes would achieve the desired improvements in the
disability claims process.  SSA had expected the new SDM and AO
positions to significantly improve the efficiency and processing time
without sacrificing the quality of decision-making.  However, results
from the stand-alone tests of these positions have been largely
disappointing and, in some cases, inconclusive.  As a result, SSA
decided to postpone implementation decisions on these two initiatives
until results from the agency's integrated FPM test were available. 

As an example, SSA had hoped that permitting the SDM to make
disability determinations independently, using the DDS physician only
on an as-needed basis, would reduce the time spent on the
determination process.  However, early test results revealed that the
SDM position would, on average, reduce by only 1 day the time
claimants waited for an initial determination and by only 3.6 minutes
the time personnel actually spent working on the case.  Moreover, SDM
determinations for certain impairment categories were less accurate
than under the current system.  However, early results from the test
of the FPM initiative, where the SDM was tested with other process
changes, have shown more promise for the SDM.  SSA's final evaluation
of the FPM test for four of the five process changes will not be
available before October 1999. 

Table 2.2 shows the final or most recent results of the tests of the
three initiatives.  Appendix III contains more detailed information
on test results for these three initiatives. 



                         Table 2.2
          
           Results of Tested Redesign Initiatives

                               Test results
                ------------------------------------------
Redesign
objectives      SDM           AO            FPM
--------------  ------------  ------------  --------------
Allow claims    Increased     Increase in   Increased
that should be  allowances    allowances    allowances (by
allowed at the  (by 1.2       made earlier  4.7 percent)
earliest        percent),     at the        and improved
possible        but reduced   appellate     accuracy
level.          accuracy      level was     (mostly denial
                (mostly       not           determinations
                denial        demonstrated  ) at the
                determinatio  , but         initial level,
                ns) at the    improved      but
                initial       accuracy of   improvements
                level.        some          at the
                              allowance     appellate
                              decisions     level not
                              was           demonstrated.
                              demonstrated
                              .

Make decisions  Marginal      Did not       Improvement
as quickly as   improvement   achieve       demonstrated
possible (that  (1 day        targets;      at the DDS
is, reduce      faster) at    improvement   level due to
days needed to  initial       over the      elimination of
make a final    level.        current       reconsideratio
decision).                    process not   n step, but
                              demonstrated  extent of this
                              .             improvement
                                            has not been
                                            determined.
                                            Improvement at
                                            the appellate
                                            level, or
                                            overall, not
                                            demonstrated.

Make process    Marginal      Did not       Improvement
more efficient  improvement   achieve       demonstrated
(for example,   (3.6-minute   targets;      at the DDS
by reducing     reduction in  improvement   level due to
task time,      task time)    over the      elimination of
administrative  at the        current       reconsideratio
costs, or       initial       process not   n step, but
both).          decision      demonstrated  extent of this
                level.        .             improvement
                                            has not been
                                            determined.
                                            Improvement at
                                            the appellate
                                            level, or
                                            overall, not
                                            demonstrated.

Make process    Not tested.   Mixed         Customer input
user-                         results:      neutral
friendly.                     customer      regarding
                              input         predecision
                              shifted from  interview.
                              positive to
                              neutral over
                              time.

Provide a       Generally     Mixed         Generally
satisfying      positive for  results:      positive for
work            SDM test      test          SDM and AO
environment     participants  participants  test
for employees.  .             were more     participants.
                              satisfied
                              with AO
                              position
                              than OHA
                              staff were.
----------------------------------------------------------
Note:  SDM test results were reported in Nov.  1997 and are final. 
AO test results are from different time periods and generally reflect
the most recent and complete data available.  FPM test results were
reported in Oct.  1998 and are preliminary, because not all of the
cases have been processed through the AO and ALJ levels. 

Source:  SSA test evaluation documents. 

SSA has not made enough progress on its two other near-term
initiatives, process unification and quality assurance, to fully
assess their efficacy.  Although SSA has not completed many of its
planned measures for these two initiatives, some of the early process
unification actions may have had a positive effect on customer
service.  SSA reported that it has accurately paid benefits to
approximately 90,000 people 500 days earlier in the process than
otherwise might have been the case.  While SSA generally did not test
these process unification initiatives before implementing them,
officials believe that the increase in allowances made earlier in the
process is in large part due to the agency's process unification
efforts.  At the same time, these officials noted that other factors
can influence allowance rates.  Therefore, without conducting
carefully structured tests, it is difficult to isolate the effects of
actions taken by the agency. 


      DELAYS AND DISAPPOINTING
      TEST RESULTS LED TO
      DECREASED EXPECTATIONS FOR
      REDESIGN
-------------------------------------------------------- Chapter 2:1.3

As a result of the delays and disappointing test results, SSA has
decreased projected administrative savings and postponed the
anticipated date for realizing any savings.  In 1997, SSA projected
savings of 12,086 staff-years for fiscal years 1998 through 2002,
resulting from implementing several process changes.  SSA planned to
use some of these staff-year savings to help with other workloads. 
Instead, in 1998, SSA both decreased its savings projections and
postponed the date it expected to realize savings, which changed its
projected staff-year savings to 7,207 through fiscal year 2003. 
Table 2.3 shows how SSA's projected staff-year savings changed from
its 1998 to its 1999 President's budget. 



                         Table 2.3
          
            Projected Redesign Savings in Staff-
            Years for 1998 and 1999 President's
                          Budgets

                      Staff-year savings
                  --------------------------
                          1998          1999    Difference
                   President's   President's      (1998 to
Fiscal year             budget        budget         1999)
----------------  ------------  ------------  ------------
1998                   (861)\a         (654)           207
1999                     1,684         (741)       (2,425)
2000                     3,551         1,850       (1,701)
2001                     3,683         2,236       (1,447)
2002                     4,029         2,187       (1,842)
2003                        \b         2,329         2,329
==========================================================
Total                   12,086         7,207       (4,879)
----------------------------------------------------------
\a Numbers in parentheses represent additional staff-years needed. 

\b The 1998 President's budget provided projected redesign savings
for fiscal years 1998 through 2002. 

Source:  SSA budget documents. 

More importantly, test results have not provided a compelling case
for SSA to make these changes and thereby improve customer service as
quickly as it had hoped.  Overall processing times have not
significantly changed since the beginning of redesign; that is, while
processing times have decreased at the initial level, they have
increased at the ALJ level.\24 On the other hand, more allowances are
being made earlier in the process, and SSA attributes this to its
process unification efforts, which were planned to improve customer
service without significantly increasing the overall cost of
providing benefits. 


--------------------
\24 Between fiscal years 1994 and 1998, processing time decreased at
the DDS level from 86.9 and 109.6 days (for the DI and SSI programs,
respectively) to 72.6 and 75.4 days and increased at the ALJ level,
where data for the two programs are combined, from 305 to 341 days. 
SSA does not currently measure overall processing time. 


   LIMITED PROGRESS IS RELATED TO
   SSA'S TESTING AND
   IMPLEMENTATION STRATEGY
---------------------------------------------------------- Chapter 2:2

SSA's difficulties in achieving appreciable improvements in its
disability claims process have been caused, in part, by the scope of
SSA's revised plan and the agency's strategy for testing its proposed
process changes.  Much like its original plan, SSA's February 1997
plan was designed to achieve quick and major improvements on many
fronts simultaneously in response to the pressing problems with the
claims process.  However, as with the original plan, SSA's revised
plan proved to be too ambitious and difficult to manage within
established time frames.  Moreover, SSA's decision to conduct
stand-alone tests contributed to marginal SDM and AO test results,
and weaknesses in how SSA designed and managed its AO test
contributed to unreliable AO test results.  Finally, difficulties SSA
experienced with testing changes in an operational environment raised
questions about how a tested initiative would operate if implemented. 


      SSA'S REVISED PLAN STILL
      PROVED TOO AMBITIOUS
-------------------------------------------------------- Chapter 2:2.1

Like its original plan, SSA's revised plan was designed to provide a
number of near-term, visible improvements, while also laying the
foundation for long-term changes.  To accomplish this, SSA acted to
make progress on a number of fronts simultaneously.  For example,
hoping to alleviate growing workloads at the appellate level, SSA
began testing and planned to implement the AO position independently
of other initiatives, even though certain changes that could support
the position, such as the redesigned computer system, were still
being developed.  SSA also began testing and planned to implement the
SDM position by itself because SSA believed it could achieve quick
and decisive improvements through this position.  The agency believed
these quick improvements would build momentum for redesign and
increase stakeholder support.  While the AO and SDM tests were still
ongoing, SSA began its FPM test, which investigated the interactive
effects of five process changes together:  the new SDM and AO
decisionmaker positions, the predecision interview with the claimant,
eliminating the reconsideration step, and eliminating the claimant's
option to request a review by the Appeals Council.  In addition, SSA
began testing and developing subinitiatives under its process
unification, quality assurance, and the three remaining longer-term
initiatives.  Given the urgent need to fix the process, SSA
considered this ambitious approach appropriate as well as consistent
with reengineering theory.  At that time, reengineering theory
generally called for short time lines for testing and implementing
major process changes. 

In addition to this multifaceted approach, SSA decided for several
reasons to conduct its tests of the proposed redesign changes at many
sites and to involve numerous participants.  First, officials
believed this approach would build trust among employees and other
stakeholders, who feared that redesign would negatively affect them. 
Second, SSA believed it needed to use a large number of test cases to
produce statistically valid information in key areas.  For example,
SSA wanted sufficient data to determine the impact of redesign
initiatives on the accuracy of SDM determinations in each major
category of impairment.  Finally, SSA wanted to have enough data to
demonstrate the impact that changes to the process would have on
benefit outlays.  SSA officials told us that Office of Management and
Budget officials were concerned that the proposed changes to the
claims process could result in large, unanticipated increases in
benefit outlays.  Because of the size of the disability programs,
even a small increase in the percentage of claimants awarded benefits
can result in a significant increase in program costs.  For example,
SSA officials have roughly estimated that a 1-percent increase in
allowances in the disability programs for a period of 10 years could
result in an increase of $11 billion in the total benefits paid to
beneficiaries--that is, program costs--during that same period. 

As a result of SSA's decision to conduct many tests simultaneously,
at one point SSA was testing four near-term initiatives and training
test participants for another, longer-term initiative, the DCM. 
These tests were being conducted at more than 100 sites and involved
over 1,000 participants.  Table 2.4 shows SSA's testing schedule,
including numbers of sites and participants. 



                         Table 2.4
          
          SSA's Test Schedule and Number of Sites
                      and Participants

            Time period
             (in fiscal                   Number of
               years)                   participants\
          ----------------          ----------------------
Initiati                    Number
ve and                          of  Decisionmake  Clerical
phase     1996  1997  1998   sites            rs     staff
--------  ----  ----  ----  ------  ------------  --------
SDM,         X     X            10            93        12
 phase I
SDM,               X     X      30           337        44
 phase
 II
AO           X     X     X   23-26           191       110
FPM                X     X      11           134        22
Process                  X      10           209      93\a
 unifica
 tion,
 expande
 d
 rationa
 les
DCM,                     X      33           213        75
 phase I
==========================================================
Total        2     4     5    117-         1,177       263
                               120
----------------------------------------------------------
\a The amount of clerical staff time devoted to this test varied. 

Note:  While some tests were conducted in the same fiscal year, the
exact starting and ending dates of each test varied.  The number of
test sites and participants also varied over time.  The number of
participants represents SSA's best estimate and does not include
indirect support staff, such as supervisors and case consultants. 
The expanded rationales test also included physician and supervisor
participants, which are not shown in the table.  According to SSA,
test participants were doing work comparable to what they would have
been doing if they had not been in a test. 

Source:  GAO analysis of data provided by SSA officials. 

Despite SSA's good intentions, its scaled-back plan still proved to
be too ambitious, and the agency had difficulty keeping it on track. 
Conducting several large tests that overlapped in time consumed a
great deal of management attention and resources.  In addition to
developing the test plan, implementing and monitoring the test, and
collecting and analyzing data, each test involved negotiating and
coordinating activities with test sites, test participants, employee
unions, and other stakeholders.  This large array of testing and
evaluation activities made it difficult for SSA to stay on schedule
and simultaneously maintain sufficient focus on other redesign
efforts--such as process unification and quality assurance. 

Unrealistic milestones for specific initiatives also contributed to
missed deadlines.  For example, SSA allowed itself only 17 months to
conduct the FPM test and assess the results, even though it can take
up to 21 months for a test case to make its way through the entire
disability claims process.  In addition, SSA's milestones for the
eight process unification subinitiatives were probably too ambitious
(they did not include sufficient time to conduct needed tests or make
procedural changes), especially given the overall magnitude of SSA's
redesign efforts and the complexity of the problems these
subinitiatives are intended to address.  Moreover, some of the
factors that contributed to differences between decisions made by the
DDS adjudicators and the ALJs have evolved over a number of years and
involve sensitive legal issues. 

Finally, other competing workloads placed considerable strain on
SSA's ability to manage the overall redesign effort.  Besides the
redesign initiatives, disability program officials and staff had to
cope with additional unanticipated duties and responsibilities.  For
example, legislation that reformed the nation's welfare program in
1996 also required that children receiving benefits under the SSI
program meet a stricter definition of disability than had been
applied in the past.  As a result, during fiscal year 1997, when many
redesign initiatives were being tested, SSA's disability staff also
had to plan and execute a review of the eligibility of over 288,000
children receiving SSI benefits. 


      DECISION TO CONDUCT
      STAND-ALONE TESTS
      CONTRIBUTED TO MARGINAL
      RESULTS
-------------------------------------------------------- Chapter 2:2.2

SSA's decision to test its AO and SDM initiatives independently from
related initiatives contributed to the disappointing test results. 
SSA conducted these stand-alone tests because it wanted to institute
these two positions quickly.  However, as initially envisioned, these
initiatives were expected to result in process improvements and
administrative savings in concert with other initiatives.  Tested
alone, these positions did not demonstrate potential for
significantly improving the process. 

To illustrate, at the very early stages of its redesign effort, SSA
developed expectations for AO productivity assuming that the AO would
be operating in a completely redesigned environment.  However, the AO
test did not include supporting initiatives, such as a redesigned
computer system, and, consequently, AO productivity was far below
SSA's expectations.  Similarly, the SDM was expected to be operating
in a redesigned environment that included, among other changes, the
new responsibility of conducting a predecision interview with
claimants.  The results of the stand-alone SDM test indicated a
decline in the accuracy of initial determinations; on the other hand,
the integrated FPM test indicated that adding the predecision
interview to the SDM's responsibilities may improve accuracy, as
compared with the current process.\25 This improved accuracy may have
resulted because SDMs collected more or better data during the
predecision interview or because SDMs performed their job more
thoroughly in preparation for a meeting with the claimant.  While SSA
could not have predicted the precise impact of not including a
particular process change in its stand-alone tests, the agency
understood from the outset of its redesign effort that proposed
changes were closely linked and that they depended on each
other--especially on computer supports--to dramatically improve the
process. 

Overall, the decision to conduct stand-alone tests caused delays, did
not result in the efficient use of resources, and did not achieve the
agency's goal of quickly building trust and enthusiasm among those
who resisted the changes.  For example, despite the improved
performance of the SDM in the FPM test, pockets of opposition to the
SDM, particularly among groups representing some DDS physicians,
still existed.  While groups that perceive themselves to be
negatively affected by change may not be swayed on the basis of clear
and positive test results, marginal or inconclusive results provide
detractors with a firmer basis to oppose change. 


--------------------
\25 The AO position continued to perform poorly under the integrated
FPM test, which did not include the redesigned computer system
either. 


      TEST DESIGN AND MANAGEMENT
      PROBLEMS CONTRIBUTED TO
      UNRELIABLE AO TEST RESULTS
-------------------------------------------------------- Chapter 2:2.3

The AO test suffered from a number of design and management flaws
that raised questions about the reliability of certain test results. 
For example, to ensure that AO sites were staffed with the best
employees possible, SSA selected test participants from a national
pool and temporarily relocated them to their preferred locations. 
Since the test lasted some time, many of these employees decided to
return to their home units, and SSA had to replace them with new,
less experienced employees.  Replacing test participants created
instability in the test environment that negatively affected the test
results.  In addition, SSA did not arrange for AOs to have necessary
supports (such as computers, clerical assistance, supervisors, or
feedback from ALJs), which contributed to poor results. 
Consequently, SSA took steps to refine the AO test and provided
additional supports, including training and feedback, to test
participants.  Accuracy and productivity subsequently improved,
although productivity has not improved to the level originally
expected by SSA. 

Despite these improvements, other problems with how the test cases
were handled made it difficult for SSA to assess the efficacy of the
AO position.  Under the proposed process, an AO cannot deny a claim,
so, when an AO does not allow a case, the AO is then required to make
sure that all pertinent information is properly arranged in the case
file and to prepare a thorough explanation of all medical evidence so
that the case can move expeditiously to an ALJ hearing.  To fairly
assess the impact that the AO had on processing time at the appellate
level, SSA planned to compare cases prepared by AOs with a small
group of control cases in which no AO had been involved.  The two
groups were to be handled in a comparable manner; for example, both
sets of cases were to be promptly scheduled for hearings.  However,
in many instances OHA staff did not follow instructions concerning
how the cases were to be handled.  Since the number of control cases
was relatively small, when the improperly handled cases were excluded
from the analysis, the number of useable control cases was too small
to permit a valid comparison.  In addition, SSA did not design its
test to determine the overall impact of the AO-prepared cases on the
quality of decisions at the next appellate level.\26 Without reliable
data on its control group or sufficient data on the impact on
quality, SSA could not fully assess the effect of the AO position on
the claims process. 


--------------------
\26 In cases in which the AO had developed the case but could not
allow the claim, the AO would certify that it was fully developed and
ready for hearing.  Although SSA surveyed ALJ views on the quality of
AO certifications, SSA did not assess the overall impact of AO
decisions and certifications on the quality of appellate decisions. 


      VALID TESTS WERE DIFFICULT
      TO CONDUCT IN SSA'S
      OPERATIONAL ENVIRONMENT
-------------------------------------------------------- Chapter 2:2.4

SSA's other tests--of the SDM and FPM--suffered from design problems
that stemmed largely from difficulties with trying to conduct a test
in a "real world" operational environment.  While the SDM and FPM
tests provided information and insight into the efficacy of these two
concepts, operational limitations made it difficult for SSA to
conduct a statistically valid test and conclusively demonstrate how a
tested initiative would operate if implemented. 

To a lesser extent than with the AO test, SSA's test of its SDM
initiative also provided incomplete information and limited assurance
that the initiative would perform as tested.  For example, under the
current process, 50 percent of DDS allowance determinations are
reviewed by regional quality assurance staff, and errors are returned
to the adjudicator for correction.  However, under the SDM
stand-alone test, 100 percent of all determinations--allowances and
denials--were reviewed by SSA quality assurance staff and returned
for correction.  As a result, a large number of cases were returned
to SDM adjudicators even though, on average, there was not a large
difference in error rates between the SDM and the current process. 
SDM test participants and other DDS officials told us that this
100-percent review probably caused test participants to rely more
heavily on agency physicians than they might have otherwise.  In
addition, because SSA does not have administrative control over state
DDS programs--which are under the direction of state governors--the
agency was not able to select a strictly random group of test sites
or participants; nevertheless, SSA officials believe that the
participant selection methods they used came as close to random as
possible, given the present constraints.  Moreover, because workloads
and production capacity varied at the sites, SSA could not dictate
the number of test cases at each site and was therefore unable to
distribute the test caseload in a representative manner.  Finally,
the test was not initially designed to collect data on test cases as
they moved beyond the initial determination level to the appeals
level--data that would have helped determine the impact of the SDM on
overall appeal rates, processing time, efficiency, and quality of
determinations.  SSA has since modified its approach to collect some
of this information. 

In designing the FPM test, SSA overcame some, but not all, of the
problems experienced with previous tests.  For example, SSA was able
to persuade states that it believed were nationally representative,
on the basis of an analysis of state characteristics, to participate
in the test.  SSA also decided to track cases through the entire
disability claims process, rather than through the initial
determination level only.  To further ensure a sound test design, SSA
hired a consulting firm to independently evaluate the design of the
test.  While the firm found the test design to be basically sound, it
made several suggestions to improve the test and better ensure
stakeholder confidence in the validity of the test results.  SSA was
not able to make all the recommended changes, however.  For example,
because of state union-management agreements, SSA was unable to
obtain data on the qualifications of employees to ensure that test
participants were representative of all employees, as recommended by
the consultant.  In addition, contrary to the consultant's
recommendation, SSA did not mitigate the impact of the 100-percent
review of SDM determinations for quality, which may lead to some of
the same problems experienced with the SDM stand-alone test. 


ACTIONS NEEDED TO IMPROVE CHANCES
FOR FUTURE PROGRESS
============================================================ Chapter 3

While SSA has experienced, and continues to face, many difficulties
with its redesign effort, the agency can still take actions to
increase its chances for future progress.  As SSA continues its
redesign work, it has an opportunity to apply lessons learned from
its 4 years of reengineering experience, as well as from other
commonly accepted reengineering and management best practices.  SSA
has already begun to apply some lessons it has learned, such as
strengthening executive oversight of its redesign effort.  However,
the extraordinary difficulty of the task at hand and the performance
shortcomings previously experienced suggest that these steps might
not be enough.  Other fundamental changes in SSA's approach will
probably be necessary.  In particular, although more focused than its
original plan, SSA's current redesign plan is still very large in
scope and difficult to manage, and the successful completion of key
initiatives will likely require that SSA scale back its near-term
efforts even further.  SSA can also modify its testing approach to
avoid pitfalls encountered in the past.  As it moves to implement
changes that appeared efficacious in a testing environment, SSA can
ensure that it has adequate performance measures and goals to assess
changes to the process and to provide early warning of problems as
well as adequate quality assurance processes to guard against
unanticipated results. 


   MANY STEPS REMAIN UNDER THE
   REVISED PLAN
---------------------------------------------------------- Chapter 3:1

The need for SSA to improve its disability claims process continues
today.  SSA's large pending workload persists, especially at the
hearing level.  The pending workload at the hearing level grew from
357,564 in 1993 to about 483,712 in 1997.  In addition, the average
length of time it took to receive a hearing decision upon appeal also
grew in the 1990s--from 238 days in 1993 to 386 in 1997.  The
dramatic growth in initial applications for disability benefits that
contributed to these increases and exacerbated long-standing problems
has ended.  In fact, in recent years the number of individuals
applying for disability benefits has declined, which has in turn
helped reduce the 1998 appellate backlog to 384,313 and appellate
processing time to 371 days.  However, no one knows how long this
decline will last.  An economic downturn could increase unemployment
and drive up demand for disability benefits, and the number of
applications, at any time.  Moreover, the number of applications for
disability benefits can be dramatically affected by court cases and
changes in the law, such as the possibility of congressional action
to increase the retirement age.  Finally, SSA expects claims to
increase again beginning in 2000, when the eligibility age for full
Social Security retirement benefits changes from 65 to 67 years, and
more dramatically by 2010, as the baby boom generation approaches its
disability-prone years.  Taken together, present and future workloads
highlight the continuing pressure on SSA to move expeditiously to
improve its disability claims process. 

Many steps remain to be taken under the agency's February 1997
redesign plan.  As of October 1998, the agency was continuing to test
the AO, SDM, and FPM initiatives.\27 Should SSA decide to implement
any of these positions or process changes, it will face innumerable
steps ahead.  For example, SSA will need to seek changes in the law
or develop new regulations for many of the changes it is considering,
a time-consuming and multistep process.  For some of the initiatives,
such as the SDM, SSA will also need to provide for training,
facilities, equipment, and various clerical and managerial supports. 
In some cases, SSA will need to develop plans for implementing
changes in phases, such as installing new positions at a small number
of sites each month.  In addition, SSA must guard against unwanted
effects that could result from making changes to one part of the
process without adequately addressing their impact upon other parts. 
For example, should SSA decide to eliminate the reconsideration step,
SSA will need to be aware of the possibility of, and take steps to
guard against, the development of more backlogs at OHA caused by the
speedier movement of cases through the process to that level. 

While SSA has made important progress, much remains to be
accomplished on two other important near-term initiatives:  process
unification and quality assurance.  For example, under its process
unification initiative, SSA intends to review and revise established
regulations to develop its planned single presentation of policy--a
time-consuming task to which SSA has not yet been able to devote
adequate resources.  Also under process unification, SSA intends to
continue providing systematic ongoing training to adjudicators at all
decision-making levels and to continue work on several remaining
subinitiatives.  Under its quality assurance initiative, SSA still
needs to ensure that adequate in-line quality assurance procedures
are in place for any changes it makes to the process.  SSA is also
still trying to reach an agreement on a single "end-of-line" quality
review mechanism for the whole disability claims process.  Once
agreement is reached, SSA will need to test this mechanism.  As
discussed in chapter 2, developing and testing initiatives can
involve a substantial amount of time and effort and require the
cooperation of numerous stakeholders. 

If SSA continues its redesign effort as planned, the agency has even
more matters to contend with for its three longer-term initiatives: 
the DCM; a simplified methodology for making disability decisions;
and the Redesigned Disability System (RDS), SSA's new disability
computer system.  All three involve major operational changes and are
the furthest from implementation.  The DCM combines the duties of SSA
field office personnel and state disability examiners and will
require legislative changes before it can be implemented.  As of
October 1998, SSA was still conducting the first of three lengthy
test phases that precede full implementation of the DCM.\28 The
simplified decision methodology initiative is still in the
developmental stage, and much more research needs to be accomplished
before SSA can begin laboratory testing.  Finally, SSA is
experiencing a number of problems with its proposed RDS, a system
that is viewed as key to many of the planned process efficiencies. 
In January 1998, we reported that software development problems had
delayed the scheduled implementation of RDS by more than 2 years.\29
Later in 1998, we reported that SSA had experienced problems and
delays in its RDS pilot effort initiated in August 1997 to assess the
performance, costs, and benefits of RDS.\30 For example, systems
officials stated that, using RDS, the reported productivity of claims
representatives in the SSA field office dropped.  Systems officials
also stated that because the RDS software had not performed as
anticipated, SSA had engaged an independent contractor to evaluate
and recommend options for proceeding with RDS.  This effort is
expected to further delay SSA's national rollout of the new
disability computer system.  See table 3.1 for the steps that have
yet to be taken under the revised plan. 



                         Table 3.1
          
               Steps Remaining for the Eight
          Initiatives in SSA's February 1997 Plan

Initiative    Steps remaining
------------  --------------------------------------------
SDM           Complete test and evaluation, prepare cost-
              benefit analysis, and make go/no go decision
              (also see FPM). If warranted, prepare a
              notice of proposed rulemaking (NPRM) and
              final regulations and conduct phased rollout
              (including training, facilities, equipment,
              and other support). To extend SDM to all
              cases, make legislative changes.

AO            Complete test and evaluation, prepare cost-
              benefit analysis, and make go/no go decision
              (also see FPM). If warranted, prepare NPRM
              and final regulations; make organizational
              decisions; and conduct phased rollout,
              including training, facilities, equipment,
              and other support.

FPM           For AO, SDM, PDI, and eliminating
              reconsideration, complete test and
              evaluation and make go/no go decisions. If
              warranted, prepare NPRM and final
              regulations for eliminating reconsideration
              and conduct phased rollout of PDI (including
              training and other support) and elimination
              of reconsideration. Complete test and
              evaluation for eliminating Appeals Council
              review. (Also see AO and SDM.)

Process       For single presentation of policy, begin
unification   consolidating established policies and
              regulations. For training, continue to
              develop and provide ongoing training to
              support process unification. For four
              pending subinitiatives: (1) clarify and
              finalize policies and procedures for
              weighing treating source opinion; (2)
              complete testing and implement expanded
              rationales; (3) draft final rules for weight
              of DDS medical consultant opinion; and (4)
              clarify and finalize procedures for
              assessing residual functional capacity for
              less than the full range of sedentary work.
              For two subinitiatives that SSA has to some
              extent implemented, the following steps
              remain: (1) continue oversight of new
              procedure for quality review of hearing
              decisions and (2) monitor new acquiescence
              action plan.

Quality       For in-line QA, complete development and
assurance     implement in-line QA processes for redesign
              initiatives. For end-of-line QA, reach
              agreement on a single standard for an end-
              of-line review mechanism and test and
              evaluate review mechanism. If warranted,
              implement end-of-line quality review
              mechanism.

Simplified    Complete development and studies of new
decision      methodology, conduct lab tests of functional
methodology   assessment instruments, integrate various
              research projects into a single paradigm for
              disability decision-making purposes, and
              establish "methodology lab" studies to
              determine the administrative impact of
              recommendations on the results of disability
              decisions. If warranted, publish NPRM and
              final rules and implement new decision
              methodology.

RDS           Contingent on results of consultant review,
              continue piloting and enhancing
              functionality of RDS to support redesign
              vision, incorporate disability applications
              and forms into SSA's expanded Internet
              capabilities for use with third parties,
              identify additional opportunities for
              electronic medical evidence, and roll out
              RDS.

DCM           Complete first phase of testing. Prepare and
              develop an evaluation plan for second,
              formal test phase. Make go/no go decision on
              whether to enter into a third or
              preimplementation phase of testing with RDS
              and decision methodology supports. If
              warranted, incorporate RDS and decision
              support, prepare final regulations, and make
              any necessary legislative changes.
----------------------------------------------------------
Source:  SSA. 

Even if SSA successfully tests and implements all of the redesign
initiatives included in the February 1997 updated plan, it is
unlikely that all of the problems that gave rise to SSA's redesign
effort in the first place will be satisfactorily resolved.  As we
have noted, test results to date show only modest improvements in
operations, and budgetary savings will not be as large as originally
anticipated.  Moreover, except for the AO initiative, most of SSA's
redesign efforts to date are focused on improving the process at the
initial determination level, leaving problems at the ALJ level
largely unresolved.  These problems include length of processing
times and the large number of backlogged cases at hearing offices,
which are among the most pressing problems that SSA faces with the
claims process and which require additional solutions. 


--------------------
\27 As discussed in chapter 2, stand-alone testing for the AO has
been extended to work on improving upon initial disappointing
results; stand-alone testing continues for the SDM as a step toward
implementation.  In addition, evaluation of FPM test results has not
been completed. 

\28 The three phases are a preparation phase involving classroom and
on-the-job training (Nov.  1997 to June 1999); a phase to formally
test the concept without the support of RDS and the simplified
decision methodology (June 1999 to Oct.  2000); and a
preimplementation phase to assess the impact of implementation with
RDS and the simplified decision methodology supports (Oct.  2000 to
Oct.  2002). 

\29 Social Security Administration:  Software Development Process
Improvements Started but Work Remains (GAO/AIMD-98-39, Jan.  28,
1998). 

\30 Social Security Administration:  Technical Performance Challenges
Threaten Progress of Modernization (GAO/AIMD-98-136, June 19, 1998). 


   SSA'S ACTIONS TO REVISE ITS
   APPROACH HAVE BEEN HELPFUL BUT
   INSUFFICIENT TO ENSURE SUCCESS
---------------------------------------------------------- Chapter 3:2

SSA has already taken actions to revise its approach and apply some
lessons learned from its early efforts with redesign, including
formalizing high-level executive oversight, working to improve test
design, and rethinking its strategy for communicating with
stakeholders.  However, these efforts may not be enough to ensure
success.  Because of the unique barriers to change inherent in
governmental operations, redesign is particularly challenging for
government agencies, and SSA may need to consider additional changes
in its approach to improve its chances of making tangible future
progress. 

In its 1997 plan, SSA established a new management structure to
oversee redesign efforts in order to make its senior managers more
accountable and involved.  Specifically, SSA centralized authority
for redesign efforts by creating an Executive Committee for
Disability Redesign, chaired by the principal deputy commissioner. 
Such high-level oversight is critical, given the organizational
complexity of the disability claims process.  It is also consistent
with government and industry best practices, which provide that the
individual in charge of a reengineering effort be responsible for the
entire process and its performance.\31 Strengthening executive
oversight has already had a positive effect on the progress of
redesign.  For example, by promoting timely processing of cases for
the FPM test, the Executive Committee has helped to expedite analysis
of test results.  SSA officials told us they believe that Executive
Committee oversight has helped provide a new momentum by working to
ensure that activities stay on schedule and that critical policy
decisions receive sufficient and early high-level attention. 

SSA is also applying some valuable lessons learned from conducting
the AO test.  Because the AO test results were inconclusive as a
result of problems with the design and management of the test, SSA
has taken greater care with the design and management of subsequent
tests.  For example, SSA enlisted the services of an independent
consulting firm to review its proposals for both the FPM and DCM
tests.  Also, for the DCM test, which is similar to the AO test in
that it is lengthy and involves testing the efficacy of a new
position, SSA is taking steps to ensure test participants receive
adequate training and support, and that the testing environment
remains stable. 

SSA officials told us they have also learned a great deal about
balancing the need for open communication with stakeholders with the
need to keep initiatives on track and make tough and sometimes
unpopular decisions.  Effective stakeholder communication is an
important area according to reengineering experts.  Its importance
was recently noted in a private sector survey of 102 private and
government organizations that found that sending inconsistent signals
and not communicating enough with stakeholders were among the five
most serious mistakes top management sponsors made during a major
change.\32 However, communicating with stakeholders is different from
obtaining consensus on proposed changes, a practice that can
sometimes lead to management paralysis.  The proposed changes in
SSA's redesign plan affect most aspects of the disability claims
process, and it is unlikely that the agency can achieve
across-the-board support from all parties affected by the change. 
Early on in its redesign efforts, SSA leadership took extraordinary
steps to reach out to key stakeholders to build acceptance and
consensus for its redesign initiatives.  SSA officials told us they
now understand that they cannot expect to satisfy all stakeholders
and believe they lost valuable momentum early in the redesign effort
trying to do so.  Agency officials have continued with their efforts
to communicate with various stakeholder groups, however, and our
review showed that, although stakeholders do not unanimously support
all of SSA's redesign initiatives, many of the stakeholders we
contacted were satisfied with the level of communication from SSA. 

Nevertheless, these positive efforts and lessons learned may prove
insufficient for achieving appreciable progress.  Even with
strengthened executive oversight since February 1997, milestones have
continued to slip.  Compelling test results and improvements to the
disability claims process have also proven elusive.  With so much
remaining to be accomplished, and many barriers to overcome, SSA will
need to take additional steps to keep its redesign effort on track
and achieve further improvements to the disability claims process. 

SSA is not the only government agency that has had trouble
reengineering its operations.  According to reengineering experts,
many federal, state, and local agencies have failed in their
reengineering efforts.  One reason for this high degree of failure is
the unique environment of the government workplace, which adds
considerable complexity.  For example, the flexibility to reengineer
a process is often constrained by laws or regulations that require
that processes follow certain procedures--such as the requirement, in
some cases, that a physician participate in disability cases
involving children or mental impairments.  Also, government agencies,
unlike their private sector counterparts, cannot choose their
customers and stakeholders.  Agencies must serve multiple customers
and stakeholders, who often have competing interests.  In addition,
following government procedures, such as drafting and issuing new
regulations and complying with civil service rules, makes it
difficult to implement changes at the quick pace often considered
vital for successful reengineering efforts.  Finally, public agencies
must also cope with frequent leadership turnover and changes in the
public policy agenda.  For example, as discussed in chapter 2, SSA
faced several policy changes during the last few years, such as the
need to redetermine the eligibility of thousands of children
receiving SSI benefits, at the same time that the agency was trying
to conduct large tests of process changes. 


--------------------
\31 Michael Hammer and Steven A.  Stanton, "The Reengineering
Revolution," Government Executive (Sept.  1995), p.  7A. 

\32 ProSci, Best Practices Report for Managing Change (Loveland,
Colo.:  ProSci, 1998). 


   SHARPER FOCUS IS NEEDED ON MOST
   CRITICAL AREAS
---------------------------------------------------------- Chapter 3:3

According to experts in the field, reengineering requires sharp focus
and enormous discipline, and organizations are more likely to succeed
if they concentrate their efforts on a small number of initiatives at
any given time.  One way of focusing a reengineering effort is by
prioritizing process improvement objectives and identifying those
initiatives most likely to achieve those objectives.  Basic
reengineering precepts suggest that an agency should decide which
process or major subprocess should have highest priority for agency
action.  This decision should be based on selecting process changes
that (1) have strong links to the agency's mission and would have a
high impact on customers, (2) are likely to provide a large return on
invested resources, (3) enjoy a strong consensus, (4) are feasible
given the available resources and infrastructure, or (5) can be
achieved within a short period of time in order to gain experience in
reengineering. 

SSA's own experience strongly underscores the need for focus.  As
discussed in chapter 1, SSA realized early on that it could not
effectively manage the large number of initiatives in its original
redesign plan within established time frames, and later that scaling
back its plan in February 1997 was a step in the right direction. 
SSA's experience was not unlike that of others.  Early reengineering
theory called for large systemwide changes over a short period of
time; but experts now suggest that achieving significant change takes
longer and costs more than generally believed several years ago. 
However, SSA has continued to miss milestones and, with much
remaining to be accomplished, additional focus may be necessary to
achieve significant and concrete improvements to the process.  As we
reported in December 1996, process unification, quality assurance,
and enhanced information systems are among those initiatives most
crucial to producing significant improvements in the process.  Other
initiatives could be explored on a limited basis or undertaken at a
later date once progress was ensured for critical initiatives or when
additional resources became available.\33

Concern over the scope of SSA's plan and the resources used for
redesign activities was similarly expressed by the independent,
bipartisan Social Security Advisory Board in an August 1998
report.\34 The Board concluded that the costs of the redesign project
were significant and could not be sustained indefinitely.  The cost
of SSA's redesign efforts is difficult to calculate.  According to
SSA officials, the agency spent approximately $16.7 million from 1995
through 1998 on redesign activities--mostly on travel associated with
relocating test participants around the country, but also on
training, rent, supplies, and equipment.\35 In addition to these
expenditures, the Advisory Board pointed out that the redesign effort
consumed the time and attention of a considerable number of the most
experienced and knowledgeable staff within both SSA and the DDSs,
diverting them from the routine disability claims process.  In the
context of constrained administrative resources, the Board advised
that resources that had been diverted be returned as soon as possible
to their usual functions so that SSA and the state agencies could
fulfill their basic program responsibilities. 

Prioritizing its key redesign objectives might help SSA to better
focus its efforts.  As discussed in chapter 1, SSA's redesign effort
currently has five key objectives:  allowing claims that should be
allowed at the earliest possible level and improving efficiency,
speed, and customer and employee satisfaction.  However, these
objectives can work at cross purposes; an improvement in one area can
result in a deterioration of performance in another.  For example,
focusing on efforts that speed up the process and improve efficiency
might reduce the amount of attention given to developing evidence and
documenting decisions.  This, in turn, might result in incorrect
allowances (or denials) earlier in the process.  On the other hand,
focusing on the objective of making the right decision at the
earliest possible level could add time at the initial level, which
might result in more accurate initial determinations and fewer
appeals, which in turn might improve the speed and efficiency of the
overall process. 

SSA officials told us that if they were to begin again, they would
consider dividing the redesign effort into smaller, more manageable
segments.  This would be one way for the agency to better focus on
specific initiatives and perhaps be able to achieve more visible
near-term gains.  In fact, SSA may end up taking this approach during
the implementation phase by rolling out small segments of the
redesign plan one at a time. 


--------------------
\33 GAO/HEHS-97-20, Dec.  20, 1996. 

\34 Social Security Advisory Board, How SSA's Disability Programs Can
Be Improved (Washington, D.C.:  Social Security Advisory Board, Aug. 
1998). 

\35 This amount does not include the salaries of personnel
participating in or managing the tests or the cost of developing the
new decision methodology. 


   A MODIFIED TESTING APPROACH
   COULD MORE EFFICIENTLY PROVIDE
   VALUABLE INFORMATION
---------------------------------------------------------- Chapter 3:4

Reengineering best practices, as well as SSA's own experience to
date, suggest that modifications to SSA's testing approach could help
the agency to more efficiently demonstrate the likely result of
proposed changes.  Conducting smaller and more integrated tests could
free up resources to address critical initiatives while effectively
demonstrating the efficacy of interrelated changes.  In addition,
some of SSA's redesign initiatives face considerable barriers to
implementation because they represent significant change, affect
jobs, or depend on other changes or supports to be effective.  SSA
could more effectively explore the viability of such initiatives--as
well as of alternative approaches--on a small scale or wait until
essential supports have been developed before investing significant
resources in testing these initiatives. 


      SMALL-SCALE TESTING IS AN
      EFFECTIVE WAY TO INITIALLY
      DEMONSTRATE EFFICACY
-------------------------------------------------------- Chapter 3:4.1

Many reengineering experts believe that entities undergoing
reengineering, such as SSA, should conduct small tests of proposed
initiatives.  Reengineering best practices caution against moving
directly from concept to large-scale testing or implementation and
suggest that methods such as limited pilot tests and prototyping are
cost-effective means for evaluating the effectiveness and workability
of proposed changes.  As we recommended in our 1996 report on SSA's
reengineering effort, SSA would benefit from concentrating its
efforts on first testing initiatives using a smaller, more manageable
scope at only a few sites across the country.\36

SSA's own experience with the AO and SDM tests confirmed that
small-scale testing is prudent.  Significant resources and time were
devoted to large-scale tests of the AO and SDM, only to discover that
their efficacy in a stand-alone environment was marginal.  The AO
test in particular--which lacked good design, disciplined management,
and key supports--proved costly and ineffective in proving the AO
concept.  SSA moved quickly from concept to large-scale testing
because it wanted to definitively demonstrate the positive impact of
these proposed changes so they could be immediately implemented.  But
test results did not support immediate implementation.  Instead, the
outcome has been continual testing that has drained agency resources
and energies.  In hindsight, SSA could have discovered the
marginality of stand-alone initiatives with a much smaller commitment
of resources. 

As noted in chapter 2, SSA officials continue to believe that the
agency must conduct tests involving a large number of cases.  Given
SSA's desire to collect a sufficiently large amount of data and move
quickly to change the claims process, SSA officials believe their
approach for the AO and SDM tests was correct, and that if the test
results had been positive, all would be well.  However, we believe
that SSA took a costly risk that may have eroded support for the
initiatives.  SSA officials have said that in the future they would
consider reducing the number of sites that they use in tests by
concentrating test sites in a few states or within one SSA region to
permit more efficient use of resources and easier test management and
oversight. 

SSA's current plans involve testing other initiatives, such as the
DCM, on a large scale.  The DCM test currently under way has a
start-up cost of $20 million and involves 210 federal and state
participants at 33 sites across the country.  Given the uncertainties
inherent in this new position, as well as SSA's past experience with
large-scale testing of the AO and SDM initiatives, SSA runs the risk
of learning on a large and expensive scale that the DCM does not meet
the agency's redesign objectives.  It would be more cost-effective to
test this initiative on a small scale and move on to a large-scale
test only if initial results suggested the potential for significant
gains.  In the event of unforeseen difficulties or poor test results,
it would be easier and less costly to make any necessary adjustments
to a small-scale test than to a larger one. 


--------------------
\36 GAO/HEHS-97-20, Dec.  20, 1996. 


      INTEGRATED TESTING IS
      IMPORTANT FOR ASSESSING THE
      EFFECTIVENESS OF
      INTERRELATED INITIATIVES
-------------------------------------------------------- Chapter 3:4.2

According to a reengineering expert we consulted, stand-alone testing
of interrelated initiatives is inefficient and unnecessary because it
provides no synergy or learning across the whole process.  In
addition, as shown by reengineering research, effectively evaluating
the overall impact of a redesign effort requires studying the entire
business unit or process being reengineered.  In fact, we recommended
in our 1996 report that SSA combine key initiatives into an
integrated process and test that process at a few sites. 

SSA's experience confirms the importance of integrated testing. 
Projected benefits from reengineering were predicated on the
assumption that most process changes and supporting initiatives would
be operational simultaneously.\37 However, as discussed in chapter 2,
SSA has been testing initiatives independently and without the
benefit of some key supports.  SSA officials maintain that they have
learned a great deal from the large-scale, stand-alone tests, such as
how to better run a test.  They also maintain that the stand-alone
tests provided a baseline of information; for example, testing the
SDM in a stand-alone environment provided data to compare with the
SDM performance in an integrated environment.  SSA officials also
believe that the tests contributed to improved communication among
operational units and opened the door for important cultural changes
needed to support redesign.  Although SSA may have learned from its
stand-alone tests, these tests did not demonstrate dramatic
improvements to the process or provide valuable insight on how the AO
and SDM would ultimately work in concert with other initiatives.  For
example, only when SSA began the FPM test did it become apparent that
the SDM might have performed differently if it had been tested in an
integrated environment.  Rather than conducting large-scale testing
of individual initiatives, such as the SDM and AO, moving directly
into integrated testing, even on a small scale, or waiting until key
supports were in place, might have been more efficient. 

SSA has the opportunity to apply these lessons in future tests of
initiatives.  For example, the agency recently began testing expanded
rationales---an effort designed to more fully document, at the
initial level, the reasons a claim has been denied.  These tests have
been conducted outside of the FPM test, even though expanded
rationales are closely related to other process changes in the FPM. 
Officials are now taking steps to incorporate this feature into the
FPM.  SSA will be conducting small pilot tests in four states to
gather information regarding the impact of expanded rationales when
they are added to other FPM process changes.  Folding the expanded
rationales test into the FPM test will provide more valuable
information on the efficacy of this change in the environment in
which it was intended to be implemented.  Similarly, a new simplified
disability decision methodology and computer software support are
considered essential to the success of the DCM position.  However,
since these important support initiatives are not scheduled to be
available in time to meet the current schedule for testing the DCM,
it is not clear what or how much SSA will learn from this test about
the viability or effectiveness of the DCM in a redesigned
environment. 


--------------------
\37 According to SSA's Nov.  1994 plan, most key initiatives were to
be operational around 1998, except for the DCM, simplified decision
methodology, and some features of RDS, because these initiatives
needed much longer lead times. 


      ALTERNATIVE REDESIGN
      PROCESSES SHOULD BE FULLY
      EXPLORED
-------------------------------------------------------- Chapter 3:4.3

Reengineering best practices suggest that, before selecting a
specific process change for implementation, an organization should
develop several possible alternatives to the existing work process
and consider the costs and benefits of each.  These alternatives
should then be explored in order to (1) convincingly demonstrate the
potential of each option to achieve the desired performance goals;
(2) fully describe the types of technical and organizational changes
necessary to support each goal; and (3) if possible, test key
assumptions.  Also, as part of a cost-benefit analysis, an agency
should take into consideration any barriers to and risks in
implementing each alternative. 

SSA might have avoided some of the problems currently being
experienced with the AO initiative, which has engendered strong
opposition, had other alternative work subprocesses also been
explored on a small-scale basis before large-scale AO testing. 
Alternatives to the AO initiative for improving the appellate process
exist, such as SSA's temporary program to permit senior staff
attorneys in hearing offices to allow benefits in clear-cut cases.\38
However, SSA did not adequately assess the merits of the alternatives
by obtaining concrete and comparable data on their relative costs,
benefits, and risks.  After 3 years of testing, SSA must decide
whether to abandon the AO initiative, begin seriously exploring other
solutions to pressing problems at the appellate level, or both. 
Compounding matters, opponents of the AO concept have pointed to its
marginal test results to support their own favored, albeit untested,
alternatives.  SSA officials agreed that they did not fully prepare
themselves for the possibility that their proposed changes might not
work and thus did not adequately pursue alternatives earlier in the
redesign process or develop contingency plans. 

SSA may still be able to apply this important lesson in a remaining
area by more fully exploring feasible alternatives to the DCM
initiative.  As with the AO, the DCM initiative is facing some strong
opposition and has perhaps even more barriers to full implementation
standing in its way.  According to one high-level SSA official, test
results would have to be very compelling to support implementation of
the DCM initiative.  Nevertheless, SSA has begun a 3-year large-scale
test of the DCM without adequately exploring feasible alternatives. 
For example, SSA could have--as we recommended in our 1996
report--systematically tested alternatives such as sequential
interviewing to compare their relative effects on the process before
beginning the large-scale DCM test.\39 Instead of testing this
concept, SSA allowed the individual operating units to decide whether
or not they would adopt this approach.  SSA officials believe few, if
any, units are actively pursuing it.  There is still time for SSA to
explore such alternatives to the DCM while the agency conducts its
protracted test. 


--------------------
\38 In 1994, SSA began a plan to reduce the backlog of appealed
cases.  As part of this plan, SSA established an expanded prehearing
conference, at which OHA's senior staff attorneys were given
quasi-judicial powers, or the authority to issue allowance decisions
without an ALJ's involvement or approval. 

\39 Under sequential interviewing, after SSA field office personnel
obtained the necessary nonmedical information from the claimant, they
would refer the claimant to disability examiners, who would complete
the medical portion of the application, often during the same
interview.  Claimants would be able to talk to both parties that
worked on their case, and time lost in hand-offs between the two
units could be minimized. 


   IMPLEMENTATION WILL REQUIRE
   CAREFUL MONITORING
---------------------------------------------------------- Chapter 3:5

As of October 1998, SSA was considering widespread implementation of
several changes to the disability claims process on the basis of some
promising results from its FPM test.  While SSA has encountered
considerable challenges in testing its initiatives, the risk of
further difficulty during their implementation is very high.  The
experience of other public and private organizations that have
attempted business process reengineering strongly indicates that,
when compared with developing or testing possible changes to a
process, implementation of those changes is more difficult. 
Moreover, it is possible that certain process changes may not perform
as expected outside the test environment.  SSA, therefore, needs
adequate performance goals and measures for key initiatives and
objectives to monitor and assess the impact of any changes made to
the process.  SSA also needs an adequate quality assurance process in
place to ensure the quality and accuracy of decisions. 


      IMPLEMENTING A NEW PROCESS
      IS DIFFICULT
-------------------------------------------------------- Chapter 3:5.1

Experience has shown that implementation of a new process is
extremely difficult and, compared with development and testing, is
the most failure-prone phase of a reengineering effort.  During
implementation, an organization's natural resistance to change must
be overcome.  According to a reengineering expert we consulted, many
reengineering efforts fail because too little time and effort are
allotted to implementation.  The numerous issues that need to be
considered and planned for include

  -- identifying all tasks, time frames, and needed resources for an
     orderly transition;

  -- structuring the rollout of the new process in a way reasonably
     suited to the nature of the process and the work and structure
     of the organization;

  -- assigning roles and responsibilities for implementation to the
     individuals who will do the work of the new process;

  -- providing a means for collecting and sharing information on
     implementation problems and solutions; and

  -- providing for close monitoring during implementation. 

SSA's implementation plans issued in 1994 and 1997 do not address
many of the above considerations.  For example, the plans do not
address the key roles, responsibilities, and reporting relationships
required by the new process.  In our discussions with stakeholders,
we found increasing anxiety over the fact that some key
organizational decisions related to work space, which unit would be
responsible for managing the proposed AO positions, and other
infrastructure issues had not yet been made.  Nor do the
implementation plans address how SSA will monitor the process to
ensure successful implementation and optimum improvements. 
Recognizing that its current implementation plan is lacking in many
specifics, SSA plans to develop more detailed implementation plans as
key decisions are made. 


      ADEQUATE PERFORMANCE GOALS
      AND MEASURES ARE NEEDED TO
      CLOSELY MONITOR RESULTS OF
      PROCESS CHANGES
-------------------------------------------------------- Chapter 3:5.2

In order to be able to effectively monitor the results of its process
changes during implementation, SSA will need adequate performance
goals and measures.  Researchers for the Harvard Business Review
found that failure to measure a new process can be particularly
damaging to a reengineering effort because, without a comprehensive
measurement system that can track the new process' performance, it is
impossible to tell if implementation is succeeding or failing.\40 A
National Academy of Public Administration report similarly found that
measuring and tracking performance continuously was one of six
critical success factors in reengineering in the government
sector.\41 The report cites performance management as a key
characteristic in successful organizations because it offers the only
way for them to assess whether or not reengineering is achieving the
results they desire. 

SSA currently collects a large amount of data related to the
disability claims process, but these data could be improved or better
tracked for the purpose of determining progress toward redesign
goals.  Key indicators that SSA uses or could use to measure progress
are fragmented, incomplete, or entirely missing.  For example, for
its agencywide performance plan, SSA is using separate performance
measures for disability claims processing times at the initial and
appeal levels.  This fragmented approach ignores the
interrelationship between the two levels; that is, reducing
processing time at the initial level might result in premature or
poor determinations; cause more cases to be appealed; and, thus,
cause overall processing times to increase.  Conversely, implementing
steps that result in a longer initial processing time but also permit
earlier correct allowances could shorten the overall average
processing time by reducing appeals.  In addition, although SSA has
said that process unification is the "cornerstone" in the foundation
of the redesigned disability claims process, SSA's performance plan
does not contain a goal for this important initiative; rather, SSA
continues to measure performance in a disjointed manner. 

SSA is collecting some appropriate data for its tests but still needs
to make sure they are linked to the agency's strategic goals and
integrated into the agencywide performance measurement system.  As
stressed by the Chief Financial Officers Council, an organization
composed of representatives of federal departments and agencies,
government entities should integrate all reform activities, including
reengineering, into the framework of the Government Performance and
Results Act (commonly known as the Results Act).  According to the
Council, one of the reasons this is important is to ensure
consistency and reduce duplication of effort.\42 Our review of SSA's
fiscal year 1999 performance plan pointed out that SSA's
reengineering effort is not fully integrated into its Annual
Performance Plan.\43 Although the Plan noted SSA's efforts to improve
the disability claims process, the Plan did not include any useful
discussion of SSA's major initiative to completely redesign its
disability claims process, nor did it indicate whether changes or
improvements expected to result from this effort were factored into
the performance measures or goals. 


--------------------
\40 Gene Hall, Jim Rosenthal, and Judy Wade, "How to Make
Reengineering Really Work," Harvard Business Review (Nov.-Dec. 
1993), p.  129. 

\41 Sharon L.  Caudle, Reengineering for Results:  Keys to Success
From Government Experience (Washington, D.C.:  National Academy of
Public Administration, 1995), p.  53. 

\42 Executive Guide:  Effectively Implementing the Government
Performance and Results Act (GAO/GGD-96-118, June 1996). 

\43 The Results Act:  Observations on the Social Security
Administration's Fiscal Year 1999 Performance Plan (GAO/HEHS-98-178R,
June 9, 1998). 


      GUARDING AGAINST UNINTENDED
      RESULTS IS IMPORTANT
-------------------------------------------------------- Chapter 3:5.3

SSA cannot be certain that its initiatives will perform the same
under "real world" conditions as they did in an artificial test
environment, and the agency will need to take additional steps to
guard against the possibility of unintended results.  For example,
SSA's test of the SDM included a quality review of all cases decided
under the test, whereas currently, far fewer cases, most of which
involve allowance determinations, are reviewed.  In the absence of
this 100-percent review, the SDM might perform differently, which
could have a significant effect on the accuracy of determinations,
the number of allowances and appeals, and overall benefit outlays. 
Possible unintended results could include inaccurate disability
determinations, unanticipated increases in benefit outlays, and
increased appellate workloads.  When test results are marginal, there
is a greater chance that expected process improvements might not
materialize. 

SSA needs to be sure that, when implementing a change in the process
such as the SDM, an adequate quality assurance process is in place to
ensure that benefit eligibility decisions are accurate.  Accuracy is
important because incorrect decisions can result in wrongful benefit
payments, unnecessary appeals, or hardship to the claimants caused by
incorrect denials.  Under its quality assurance initiative, SSA is
seeking to build quality into the decision-making process using tools
such as training, mentoring, peer review, and feedback.  SSA has been
exploring approaches to in-line quality assurance as part of its SDM
phase II test, allowing individual test sites to set up their own
processes.  During implementation of the SDM, and in the absence of a
uniform approach, SSA will need to take steps to ensure that
individual state processes are sufficient to maintain quality. 

Ultimately, SSA will need to establish a final quality assurance
process that will both identify systemic problems with case decisions
and measure the success of SSA's efforts to build quality into the
process.  As discussed in chapter 1, current reviews of DDS
determinations and ALJ decisions are conducted in isolation from each
other.  SSA has recently instituted a review of ALJ decisions that
will help identify inconsistencies in decision-making between the two
levels.  However, SSA has yet to develop a single quality review
mechanism applicable to both levels.  SSA has had particular
difficulty getting its initial and appellate decision-making levels
to agree on a consistent quality assurance process that cuts across
all phases of the decision-making process, including reaching
agreement on what constitutes a correct decision. 


CONCLUSIONS, RECOMMENDATIONS, AND
AGENCY COMMENTS AND OUR EVALUATION
============================================================ Chapter 4


   CONCLUSIONS
---------------------------------------------------------- Chapter 4:1

More than 4 years after releasing its original redesign blueprint,
SSA is still struggling to make significant improvements to its
disability claims process.  While the agency has made some progress
with process unification, SSA has missed many of its redesign
milestones, and the results of early tests did not support
implementation of specific proposed changes.  The agency is still
conducting a number of tests, including yet another large,
nonintegrated test at numerous sites.  Also, top agency officials
would like to begin making some implementation decisions about new
decisionmaker positions and other proposed changes.  With so much
left to do, SSA still has a window of opportunity, which will not be
open for long, to apply some lessons learned to help the agency
achieve important improvements to its disability claims process. 
(SSA is no longer experiencing a dramatic growth in applications for
disability benefits, but the agency can expect applications to
increase again as the baby boom generation ages or if the economy
suffers a downturn.)

SSA's ability to learn from past experience will be an important
ingredient in the success of future efforts.  For example, the size
of SSA's tests and the scope of redesign initiatives slowed SSA's
progress under its original 1994 redesign plan.  When the agency
revised its redesign plan in 1997 to include fewer initiatives and
increased executive oversight, similar problems continued to limit
progress.  Even this revised plan required the agency to move forward
on a number of varied fronts simultaneously, and SSA continued to
miss key milestones.  Again, the agency may have underestimated the
challenges of managing stakeholder input and keeping such an
ambitious effort on course.  Strong project oversight should
continue, but it will probably not be enough to ensure timely
progress.  Therefore, SSA needs to further focus its efforts by
prioritizing its objectives and concentrating its resources on the
efforts most likely to achieve those objectives.  Such efforts should
include those that help to improve consistency in decision-making,
ensure accurate results, and achieve large efficiencies through the
use of technology. 

Past experience has shown that a large-scale test of an individual
initiative, while providing an abundance of information on how well
that initiative performs in isolation from other changes, does not
clearly demonstrate how the initiative would function in a redesigned
process and is not the most efficient and effective use of resources. 
Moreover, while SSA hoped that this testing approach would help gain
the support of key stakeholders likely to be affected by the changes,
it has not done so.  To help free up resources and effectively
demonstrate the efficacy of proposed changes, SSA should conduct
relatively small tests that integrate several of the proposed changes
to the process.  Smaller tests will allow SSA to more efficiently
identify promising concepts before moving to larger-scale testing or
implementation.  Integrated testing--testing related concepts
together and with key supports in place--will help SSA to demonstrate
whether proposed changes will perform as intended under the new
process. 

SSA's experience with the AO test has also shown the risks inherent
in devoting considerable time and resources to a single unproven
approach or change.  Results of AO tests have been consistently
disappointing, and SSA now finds itself faced with the same
long-standing problems the AO was intended to remedy without a tested
alternative solution.  Therefore, in the future, before investing
significant time and resources on any initiative, SSA should explore
feasible alternatives for changing the process on a small scale.  For
example, as we have recommended before, SSA should explore sequential
interviewing as a feasible and less risky alternative to the
controversial DCM position.  Exploring alternatives and conducting
small, integrated tests of related initiatives before making large
investments are sound reengineering and management practices, the
wisdom of which has been underscored by SSA's experience to date. 

Since other organizations have found implementation of process
changes to be the most failure-prone phase of a redesign effort, SSA
is also likely to encounter numerous pitfalls as it attempts to
effect process changes in such a complex environment.  As a result,
it is especially important for SSA to take action to closely monitor
the results of changes it makes to the process and watch for early
warnings of problems.  It is possible that process changes may not
operate as expected outside the test environment.  It is also
possible that some stakeholders who do not support specific changes
may act to undermine their success.  If process changes do not
operate as expected, the results could include inaccurate decisions,
unanticipated program costs, increased appellate workloads, and lack
of improvement in service to the claimant.  Therefore, SSA should
immediately establish a comprehensive set of performance goals and
measures--a set that cuts across the whole process and is also linked
to SSA's overall strategic and performance plans--in order to assess
and monitor the results of changes to the process. 

Finally, SSA's tests of process changes have provided only limited
assurance that these changes would not degrade the quality of
disability decisions.  Specifically, SSA's tests included artificial
steps, such as a quality review of all test cases, that are not
likely to be used outside the test environment.  Quality is perhaps
the most critical aspect of the decision-making process because each
inappropriate disability decision does a disservice to claimants,
taxpayers, or both.  A wrongful denial burdens the claimant and could
result in unnecessary administrative costs if the claimant appeals
the decision, whereas a wrongful allowance results in a continuous
stream of inappropriate benefit payments.  Therefore, as changes are
made to the process, SSA should ensure that it has a quality
assurance process in place that both promotes and monitors the
quality of disability decisions. 


   RECOMMENDATIONS
---------------------------------------------------------- Chapter 4:2

As SSA proceeds with further exploration and testing of redesign
initiatives and considers implementation options, it should take the
following steps to improve the likelihood of making key improvements
to the disability claims process: 

  -- further focus resources on those initiatives, such as process
     unification, quality assurance, and computer support systems,
     that offer the greatest potential for achieving SSA's most
     critical redesign objectives;

  -- test promising concepts at a few sites in an integrated fashion;

  -- establish key supports and explore feasible alternatives before
     committing significant resources toward the testing of specific
     initiatives, such as the DCM;

  -- develop a comprehensive set of performance goals and measures to
     assess and monitor changes in the disability claims process; and

  -- ensure that quality assurance processes are in place that both
     monitor and promote the quality of disability decisions. 


   AGENCY COMMENTS AND OUR
   EVALUATION
---------------------------------------------------------- Chapter 4:3

SSA mostly agreed with our report's observations and the thrust of
its recommendations.  Specifically, SSA agreed that the tests
conducted took longer than anticipated and did not result in the
budgetary and operational efficiencies originally hoped for in the
1994 redesign plan.  SSA also agreed that it should focus on those
areas that will make the greatest contributions to improving the
quality and timeliness of decisions.  As we have recommended, SSA
intends to pursue additional process unification and quality
assurance activities.  The agency also indicated it will pursue
elements of the FPM that will significantly improve customer service. 
Finally, SSA agreed that systems technology must continue to be an
important focus of resources. 

SSA took issue with our critique of its testing strategy.  SSA
believes that stand-alone and FPM testing were both needed to gather
data and experience that are essential for making responsible
decisions.  Moreover, SSA believes that testing at fewer sites would
not have provided the required information or allowed the agency to
complete the tests in less time.  While we understand the agency's
desire to conduct large tests in order to obtain statistically valid
results, we continue to believe that exploring the efficacy of
initiatives initially on a smaller scale before moving to large-scale
testing or implementation would result in a better use of resources. 
Also, because the various initiatives are interdependent, we believe
that integrated testing would provide more complete and useful
information on how the initiatives will perform in the new process. 

SSA also stated that its current approach to testing the DCM is
consistent with our concerns and recommendations, in that it
recognizes and builds upon what SSA has learned from previous testing
experiences.  However, we still have reservations about SSA's current
approach to testing the DCM.  First, SSA continues to test this new
position on a rather large scale without having explored the
position's potential efficacy through prototyping or limited pilot
testing.  Second, SSA is testing this initiative without the benefit
of the key supports (such as a new simplified disability decision
methodology and computer software support) upon which its efficacy
relies.  Finally, SSA is moving forward with the DCM test without
having explored the feasibility of alternative approaches. 

While agreeing to focus on certain key initiatives, SSA believes that
changes to the decision-making process should precede major computer
system changes to enable technological developments to be crafted in
the manner most supportive of the new process.  Similarly, SSA stated
that changes to the decision-making process should precede the
development of a new quality assurance process, the purpose of which
will be to evaluate the quality of the new process.  However, we
believe that SSA can make substantial progress toward developing
these critical supports before finalizing the process changes.  For
example, certain key aspects of SSA's quality assurance
initiative--such as ensuring the consistent application of policy
across all levels of the process and developing agreement on what
constitutes a correct decision--need not rely on final changes to the
process of making a decision. 

Finally, SSA pointed out, and we agree, that the agency's monitoring
and evaluation systems currently capture a significant amount of data
related to the disability claims process.  However, as our report
indicates, these data are not always translated into comprehensive
and complete performance goals and measures that look at the
efficiency and effectiveness of the process as a whole.  As we
asserted in our report, SSA's use of separate performance measures
for disability claims processing times at the initial and appeals
levels in its agencywide performance plan ignores the
interrelationship between the two levels, thereby reducing the
usefulness of the performance measures.  We also noted the lack of
integration of SSA's redesign objectives with those found in the
agencywide performance plan.  We believe SSA can do more to make
better use of the large amount of data it collects through a
carefully crafted set of performance goals and measures. 


DESCRIPTIONS OF NEAR-TERM REDESIGN
INITIATIVES AND THEIR TEST AND
IMPLEMENTATION SCHEDULES
=========================================================== Appendix I

Five of the eight key initiatives in the Social Security
Administration's (SSA) February 1997 plan had near-term milestones;
that is, those initiatives were to be tested or implemented before
fiscal year 1999.  At the time the February 1997 plan was issued, SSA
had already begun testing two of the five initiatives--the
adjudication officer (AO) and single decision maker (SDM)
positions--and planned to begin implementing them in fiscal year
1998.  Another initiative--the full process model (FPM)--involved a
more complicated and lengthy test of several proposed changes to the
process; SSA expected to assess and implement some of these changes
earlier than others.  The remaining two near-term
initiatives--process unification and quality assurance--involve a
number of actions aimed at improving the consistency and quality of
decisions at all stages of the process.  SSA expected to complete
many planned activities under these two initiatives in time to
support the other proposed changes to the process as they were
implemented. 


   ADJUDICATION OFFICER
--------------------------------------------------------- Appendix I:1

The AO is a new position created both to facilitate the appeals
process and to serve as the primary focal point for all prehearing
activities when a hearing request is filed.  For appealed cases, the
AO would be responsible for (1) providing the claimants, claimants'
representatives, or both with an in-depth understanding of the
hearing process; (2) providing unrepresented claimants with
information about the hearing and their right to representation; (3)
developing the record for a hearing (that is, obtaining additional
evidence as warranted); (4) issuing fully favorable on-the-record
decisions when supported by the evidence; and (5) narrowing issues
and fully developing cases that are forwarded to the ALJ for hearing. 

According to the February 1997 plan, SSA planned to test the AO both
in a stand-alone fashion, without system supports such as the new
computer system in place, and, later, as part of the FPM test, which
combined several redesign features into a single test.  The AO was
the first initiative to be tested because SSA felt that the AO
position, if successfully tested and implemented, would help reduce
the backlogs and lengthy processing times at the appellate level. 
Initially, the purpose of the stand-alone test was to quickly
demonstrate the viability of the concept.  However, SSA increased the
number of test cases and added a control group for comparison
purposes after the Office of Management and Budget raised concerns
regarding the impact of process changes on benefit outlays. 

The objectives for the stand-alone test were to determine whether the
AO was an effective and cost-efficient method for (1) improving case
processing time, (2) accelerating case development, and (3)
increasing the ability of administrative law judges (ALJ) to
adjudicate claims that require a hearing.  In addition, testing was
expected to demonstrate that the AO would not adversely affect
benefit costs.  Specifically, the test would determine the AO's
impact on benefit costs (measured by allowance and certification
rates), decisional accuracy, productivity (measured by task time),
customer service (measured by processing time), and employee reaction
to the new process. 

The stand-alone AO test began in November 1995.  Test sites included
nine state disability determination service (DDS) sites and 16
federal sites spread among Office of Hearings and Appeals (OHA)
hearing offices, SSA field offices, and processing centers.  However,
SSA was not satisfied with early test results and decided in February
1997 to postpone implementation while it refined and extended the
test.  Test refinements, designed to improve efficiency, processing
time, and quality, included strengthening site management, improving
clerical and computer support, and providing additional feedback and
training to AOs.  Refinements were to be accomplished within 120 days
and to be followed by a second monitoring phase. 

According to the February 1997 plan, SSA expected to complete its
evaluation of test data, make a decision for rollout, and develop and
publish regulations for implementing the AO nationwide in fiscal year
1997.  SSA planned to begin nationwide implementation in January 1998
and to complete the implementation process in fiscal year 1999. 


   SINGLE DECISION MAKER
--------------------------------------------------------- Appendix I:2

The SDM position was designed to make the initial disability
determination process more effective and efficient by allowing the
disability examiner to be solely responsible for making the
determination.  Currently, disability examiners and DDS physicians
are jointly responsible for making the disability determination, and
DDS physicians are required to sign disability determination forms
and to certify all determinations.  Under the SDM model, physicians
would function as true consultants, providing information and advice
only on cases referred to them by the SDM. 

Under its February 1997 plan, SSA planned to test the SDM position in
both a stand-alone environment (without other, related process
changes included in the test) and as part of the FPM (which would
integrate several redesign features into a single test).  The SDM
test was conducted early in SSA's redesign effort, and in a
stand-alone fashion, because SSA felt that with the SDM in place SSA
could make some quick improvements to the process and thus build
momentum and support for the overall redesign effort. 

SSA planned to test the SDM in a stand-alone fashion in two phases. 
The first phase would test the viability of the concept--that is,
whether the SDM could improve productivity by reducing processing
time and the cost of processing cases while maintaining the current
level of accuracy in making disability determinations.  More
specifically, SSA would determine the impact of the SDM on allowance
rates, decisional accuracy, processing time, productivity, and
administrative costs.  SSA planned to conduct this test in eight
states in a tightly controlled test environment (that is, with
100-percent quality review of the test cases).  As specified in the
February 1997 plan, testing began in May 1996, and test evaluation
was to be completed in fiscal year 1997.  A decision to implement the
SDM would be based on the results of this test. 

The second phase would test approaches to implementation under more
"real world" circumstances, such as with lower levels of quality
assurance review.  Specifically, SSA was looking for approaches to
implementation that would ensure the highest quality of customer
service in terms of accuracy, productivity, processing time, and
costs.  In the February 1997 plan, the second phase was scheduled to
begin in May 1997 and, according to other planning documents, to
continue for at least 12 months. 

Contingent on positive test results, the February 1997 plan called
for implementation of the SDM position by the close of fiscal year
1998.  The SDM was not expected to handle all cases initially,
because statutes and regulations currently require a DDS physician to
review denials of claims involving mental impairments or children. 
SSA planned to pursue legislative changes in fiscal year 1999 that
would permit the SDM to handle such cases and to fully implement the
SDM position in fiscal year 2000. 


   FULL PROCESS MODEL
--------------------------------------------------------- Appendix I:3

The FPM is not a single initiative but a combination of several
redesign features to be tested as an integrated process.  As noted by
SSA, most, if not all, of the proposed process changes are
interdependent, and this test was designed to assess the impact of
combined changes.  The redesign features included in the FPM test are
(1) the SDM position, (2) a predecision interview by the SDM of the
claimants in cases in which the evidence obtained does not support a
fully favorable determination, (3) the elimination of the
reconsideration step, (4) the AO position, and (5) the elimination of
the Appeals Council review step. 

The objective of the FPM test was to determine whether, or the extent
to which, these redesign features could together produce an accurate
and appropriately documented disability determination more
efficiently than the current process.  More specifically, the test
would evaluate the impact of the combined features on allowance
rates, decisional accuracy, appeal time, processing time,
productivity, administrative costs, and employee satisfaction.  The
evaluation would also compare the performance of the SDM and the AO
in an integrated test versus a stand-alone test. 

SSA's February 1997 plan called for the FPM test to be conducted at
11 DDS sites in eight states and at two federal sites.  SSA planned
to complete its selection of 30,000 test cases for the FPM test by
January 1998.  Also in fiscal year 1998, SSA expected to complete its
assessment of all redesign features except the elimination of the
Appeals Council review step and to decide whether to institute these
four features if test results supported these changes.  In fiscal
year 1999, SSA planned to implement the predecision interview and
eliminate the reconsideration step.  Also in fiscal year 1999, SSA
expected to complete its evaluation of eliminating the Appeals
Council step--which would take much longer because cases need time to
get to and through the appellate process.  SSA did not expect to
fully implement the elimination of the Appeals Council review step
until fiscal year 2001. 


   PROCESS UNIFICATION
--------------------------------------------------------- Appendix I:4

The objective of this initiative was to achieve similar results on
similar cases at all stages of the process, through the consistent
application of laws, regulations, and rulings.  Through this
initiative, SSA hoped to address long-standing inconsistencies in
decision-making between the DDS and ALJ levels that have contributed
to high rates of appeal and allowances at the hearing level.  High
appeal and ALJ allowance rates have, in turn, increased case backlogs
and processing times at the hearing level.  In addition, ALJ
allowances have been used in class action law suits as an indication
of DDS errors in initial determinations.  Process unification is
considered by many in SSA to be the most critical of SSA's redesign
initiatives. 

In SSA's early redesign plan, the process unification initiative
consisted of developing a single presentation of policy to replace
the many vehicles used by SSA to convey policy.  After further review
of the problem, SSA expanded the scope of this initiative to include
developing Social Security Rulings (SSR)\44 to clarify policy,
providing training, and taking action on eight subinitiatives that
address key differences between the DDS and ALJ decision-making
processes.  The objectives and planned activities for these process
unification efforts follow. 


--------------------
\44 SSRs are precedential court decisions and policy statements or
interpretations that SSA has adopted as binding policy.  SSRs are
published under the authority of the commissioner of Social Security
and are binding on all components of SSA.  SSRs do not have the
effect of law or regulations, and they may be based on case decisions
made at any administrative level of adjudication, federal court
decisions, commissioner's decisions, opinions of the Office of the
General Counsel, and other policy interpretations of the law and
regulations. 


      SINGLE PRESENTATION OF
      POLICY
------------------------------------------------------- Appendix I:4.1

Currently, different vehicles exist for conveying policy and
procedural guidance to decisionmakers at the different levels of the
process.  The objective of this effort is to develop a single
presentation of policy that is binding on all decisionmakers and
helps to ensure consistent application of policy.  In July 1996, SSA
began issuing all policy instructions in the exact same language to
all adjudicators.  SSA's next steps involve addressing differences in
policy instructions issued before July 1996.  Under its February 1997
plan, SSA intended to develop a plan and procedures for combining
existing adjudicative policy documents into a "single book."
According to one SSA official, integrating policy documents will be a
lengthy process because SSA will want to review and revise some of
its policies before integrating them. 


      SOCIAL SECURITY RULINGS
------------------------------------------------------- Appendix I:4.2

The objectives of this effort are to clarify policy in areas
contributing to inconsistent decision-making and to support planned
process unification training.  In July 1996, in preparation for its
first training initiative, SSA issued nine SSRs.  SSA believes these
SSRs address the most significant adjudicative issues: 

  -- applying federal circuit court and district court decisions,

  -- clarifying when adjudicators must give the opinion of a treating
     physician controlling weight,

  -- considering allegations of pain and other symptoms in
     determining whether an impairment is severe,

  -- evaluating symptoms,

  -- considering medical source's opinions on issues that are the
     purview of only SSA,

  -- considering DDS physicians' findings of fact at the ALJ and
     Appeals Council levels,

  -- assessing the credibility of an individual's statements about
     pain or other symptoms,

  -- assessing the residual functional capacity (RFC) for initial
     claims;\45 and

  -- explaining the impact of an RFC assessment of less than the full
     range of sedentary work on an individual's ability to do other
     work. 


--------------------
\45 When medical evidence does not show that an applicant's condition
meets or equals the severity criteria in SSA's Listing of
Impairments, adjudicators assess the applicant's RFC to determine
whether he or she can perform past relevant or other work. 


      TRAINING
------------------------------------------------------- Appendix I:4.3

The objective of this effort is to provide consistent training to
adjudicators at all levels of the process in three of the most
complex disability areas--assessment of symptoms, treatment of
opinion evidence, and assessment of RFC--in order to help ensure
consistent application of policy.  At the time SSA issued its
February 1997 plan, SSA had already begun training 15,000 disability
adjudicators (including DDS examiners, physicians, ALJs, and quality
assurance staff).  SSA planned to conduct ongoing process unification
training. 


      EIGHT SUBINITIATIVES
------------------------------------------------------- Appendix I:4.4

SSA approved a second series of recommendations for process
unification in April 1996 and a set of action plans for implementing
these recommendations in July 1996.  These recommendations involved
eight subinitiatives.  According to its February 1997 plan, SSA
expected to implement the subinitiatives in fiscal year 1997. 
Although the February 1997 plan did not cite specific activities for
these subinitiatives, some planned activities were specified in other
planning documents.\46 The purpose of these subinitiatives and a
description of any planned activities relating to them are provided
below. 


--------------------
\46 These planning documents are the July 1996 action plans and SSA's
February 1997 tactical plan. 


         SUBINITIATIVE 1: 
         TREATING PHYSICIANS'
         OPINIONS
----------------------------------------------------- Appendix I:4.4.1

The objective of this subinitiative is to ensure that, in rendering a
decision, adjudicators give appropriate weight to the opinions of the
claimants' treating physicians.  Under SSA regulations, adjudicators
must consider the opinions of treating physicians and must give
controlling weight to those opinions regarding the nature and
severity of the claimant's impairment(s), provided that the opinions
are well-supported by medically acceptable clinical and laboratory
diagnostic techniques and not inconsistent with other substantial
evidence in the record.  However, an SSA study found that DDS
adjudicators and ALJs respond differently to opinions of claimants'
physicians:  ALJs appear to give more weight to the opinions of the
treating physicians than do DDS adjudicators, who are more likely to
assess these opinions in conjunction with other medical evidence in
the file. 

As discussed above, SSA issued a ruling in July 1996 clarifying its
policy on assigning appropriate weight to the opinions of treating
physicians.  In its planning documents, SSA indicated that it
intended to develop other options for achieving the goal of this
initiative. 


         SUBINITIATIVE 2: 
         COMPLETE DOCUMENTATION IN
         DDS DENIALS
----------------------------------------------------- Appendix I:4.4.2

The purpose of subinitiative 2 is to more completely document
subjective and objective evidence in cases in which a DDS denies a
claim.  SSA requires DDS physicians to record explanations of their
reasoning in assessing a claimant's RFC.  However, SSA has found that
such explanations often are not well-documented.  Improved
explanations could be more useful to ALJ decision-making, especially
since it is SSA's policy that ALJs consider the opinion of the DDS
physician in making an RFC assessment. 

To address subinitiative 2, SSA intended to prepare a program
circular on documentation requirements and develop a revised quality
assurance process to enforce those requirements by October 1996. 


         SUBINITIATIVE 3:  MORE
         DETAILED EXPLANATIONS IN
         RECONSIDERATION DENIALS
----------------------------------------------------- Appendix I:4.4.3

The purpose of subinitiative 3 is to require more detailed
explanations in all reconsideration denials--that is, more detail on
how the determination was made, especially addressing issues of
treating physicians' opinions, subjective symptoms (pain), and RFC. 
As with initiative 2, improved explanations of reconsideration
denials could be more useful to ALJ decision-making. 

For subinitiative 3, SSA intended to provide training and implement
new instructions on providing more detailed explanations of
reconsideration denials by October 1996.  Upon implementing this
initiative, SSA also planned to evaluate these changes and their
implications for other redesign initiatives. 


         SUBINITIATIVE 4:  REMAND
         SELECTED HEARING CASES
----------------------------------------------------- Appendix I:4.4.4

The purpose of this subinitiative is to return or remand selected
appealed cases to the DDS for consideration of new evidence
introduced at ALJ hearings.  One SSA study indicated that 76 percent
of appealed cases contain new evidence--generally medical in
nature--and that 27 percent of hearing awards hinge on the additional
evidence.  For cases in which the DDS determined that benefits were
indeed warranted, remanding cases with new evidence would avoid the
need for a more costly and time-consuming ALJ decision.  For cases in
which the DDS did not believe an award was justified, the DDS
physician would provide a revised assessment of the case's medical
facts and return the case to the ALJ for hearing. 

In its February 1997 tactical plan, SSA expected to return 100,000
cases--out of roughly 500,000 appealed cases per year--to the DDS for
consideration of new evidence. 


         SUBINITIATIVE 5:  WEIGHT
         OF THE DDS PHYSICIANS'
         OPINIONS
----------------------------------------------------- Appendix I:4.4.5

The purpose of this subinitiative is to issue guidelines defining the
specific weight to be given by ALJs to the opinions or conclusions of
the DDS physicians.  SSA policy already requires ALJs to consider the
opinions of DDS physicians when making RFC assessments.  However,
perceptions exist that ALJs have not sufficiently done so. 

SSA issued a ruling in July 1996 clarifying that ALJs must consider
the findings of fact made by DDS physicians as expert opinion
evidence.  In its planning documents, SSA indicated that it intended
to issue final regulations by April 1997 clarifying the weight to be
given by ALJs to the DDS physicians' opinions.  SSA also intended to
issue implementing instructions and conduct training on this subject
by April 1997. 


         SUBINITIATIVE 6:  RFC FOR
         LESS THAN SEDENTARY WORK
----------------------------------------------------- Appendix I:4.4.6

The objective of this subinitiative is to clarify regulatory language
regarding assessing a claimant's RFC in cases in which the claimant
is found capable of less than a full range of sedentary work.  SSA
has determined that differing DDS and ALJ assessments of a claimant's
capacity to function in the workplace are the primary reason for most
ALJ awards.  For example, for ALJ awards involving physical
impairments, ALJs were significantly more likely than DDS physicians
to find that applicants had very limited work capacity--that is, they
could do "less than the full range of sedentary work."\47 Moreover,
according to one study, ALJs were likely to arrive at different
conclusions in this regard than DDS adjudicators even when presented
with the same evidence. 

SSA issued a ruling in July 1996 clarifying SSA's policies regarding
RFCs of less than a full range of sedentary work.  In its planning
document, SSA stated that it intended to publish final regulations
and issue operating instructions on handling RFCs of less than the
full range of sedentary work by April 1997. 


--------------------
\47 From Sept.  1992 through Apr.  1995, two-thirds of ALJ awards
involved cases that merited a "less than the full range of sedentary
work" assessment--a classification that often leads to an award. 


         SUBINITIATIVE 7: 
         ACQUIESCENCE POLICY
----------------------------------------------------- Appendix I:4.4.7

The objective of this subinitiative is to issue an SSR clarifying
SSA's acquiescence policy.  The Social Security Act provides
procedures for claimants to appeal a final SSA decision to the
federal district court level.  In some cases, the circuit court
decision may conflict with SSA's interpretation of the act.  In such
cases, if SSA does not seek further court review or is unsuccessful
on further appellate review, SSA must acquiesce to the decision of
the circuit court.  This can result in inconsistency in policy across
the circuits if SSA does not take specific and timely steps to convey
the new policy uniformly to all SSA components. 

SSA published an SSR in July 1996 clarifying its policy on
acquiescence.  In its planning documents, SSA indicated that it
intended to develop an action plan for accelerated preparation and
clearance of acquiescence rulings by October 1996. 


         SUBINITIATIVE 8:  QUALITY
         REVIEW OF HEARING
         DECISIONS
----------------------------------------------------- Appendix I:4.4.8

The objective of this subinitiative is to implement a quality review
of ALJ allowances under the Appeals Council's own motion review
authority.  The current quality review mechanism for ALJs consists
largely of Appeals Council reviews of denials at the request of
claimants, which some argue causes a propensity for ALJs to allow
cases.  Moreover, quality reviews for the DDS and ALJ decision-making
processes are conducted in isolation from each other, and there are
no procedures in place for reconciling the differences between the
processes.  For example, DDS adjudicators do not receive feedback on
why cases they denied were subsequently overturned at the appellate
level.  The new quality assurance process aims to provide more
balance in reviewing ALJ awards and denials and to identify and
reconcile factors that contribute to differences between DDS and ALJ
decisions. 

In its planning documents, SSA stated that it intended to implement a
process whereby disability examiners and physicians with SSA's Office
of Program Integrity Review\48 would perform pre-effectuation reviews
of ALJ allowances and forward cases they felt were unsupported to the
Appeals Council.  The Appeals Council would then, using its own
motion review authority, review the case and decide whether to
reverse the decision or remand the case to the ALJ.  For cases in
which the Appeals Council disagreed with the conclusions of the
quality reviewers, the award would be allowed, but the case would be
referred to an intercomponent panel that considers the need to
clarify policies, issue new policies, or provide additional training. 
In its July 1996 action plan, SSA stated that it intended to
implement the quality assurance process, including conducting
training, by October 1996.  Evaluation of this process to determine,
for example, the need for regulatory change and implications for
redesign activities, would be ongoing. 


--------------------
\48 In 1998, the Office of Program Integrity Review changed its title
to the Office of Quality Assurance and Performance Assessment, also
called the Office of Quality Assurance. 


   QUALITY ASSURANCE
--------------------------------------------------------- Appendix I:5

The purpose of this initiative is twofold:  (1) to develop in-line
quality assurance processes (approaches designed to build quality
into the process) and (2) to develop a single, end-of-line review
mechanism that looks at the whole adjudicative process.  In-line
quality assurance processes would include procedures and tools--such
as regular training, mentoring, peer review, and real-time
feedback--to ensure the right decision is made the first time.  The
end-of-line quality assurance process would include a final review
and feedback mechanism that employed the same standard of review for
all cases and would become SSA's primary vehicle for monitoring and
measuring the success of each component's front-line quality
assurance process. 

For in-line quality assurance, SSA expected to develop and
incorporate processes as part of its major redesign initiatives,
including the AO position, the SDM position, other FPM concepts,
process unification, and the disability claims manager position. 
Specifically, procedures and tools would be developed during testing
of these initiatives.  For example, the February 1997 plan called for
quality assurance principles to be built into the SDM process during
the second phase of SDM testing, beginning in May 1997. 
Participating test sites were asked to develop quality assurance
programs for safeguarding the accuracy of decisions.  In addition,
SSA expected to continue development of quality assurance processes
to support the other FPM concepts--which include the AO, predecision
interview, and elimination of reconsideration--in fiscal year 1998. 
Finally, SSA planned to develop a strategy for fully deploying
initiatives, and their quality assurance processes, in fiscal year
1999. 

For end-of-line quality assurance, according to its February 1997
plan, SSA expected to develop a prototype final review and feedback
mechanism in fiscal year 1997, test this mechanism in fiscal year
1998, and evaluate and deploy it in fiscal year 1999. 


SSA'S EFFORTS TO MEET NEAR-TERM
MILESTONES
========================================================== Appendix II

SSA was not able to meet many of its near-term milestones for testing
and implementing key initiatives in its February 1997 plan.  SSA had
hoped to begin implementing both the AO and SDM positions in fiscal
year 1998 but postponed the decision of whether or not to implement
until fiscal year 1999.  SSA also is not as far along with evaluating
the results of its FPM test as it had earlier hoped, and further
implementation delays are likely.  SSA has taken some concrete steps
on its process unification initiative, but most of the activities
occurred behind schedule, and many have yet to be completed.  SSA
also missed most of its milestones under its quality assurance
initiative. 


   IMPLEMENTATION DELAYS FOR AO
   AND SDM POSITIONS
-------------------------------------------------------- Appendix II:1

SSA missed milestones in its February 1997 plan for implementing the
AO position.  SSA planned to complete its stand-alone AO test--its
test of the AO position excluding other redesign changes--in fiscal
year 1997.  If test results warranted, SSA had planned to begin
nationwide rollout of the new position in January 1998.  However, as
of October 1998, SSA was still testing the AO in a stand-alone
fashion and had not yet produced a final evaluation.  SSA was also
testing the AO as part of its FPM test--which integrates the AO and
other process changes--and waiting for results from the FPM test
before making a decision on whether to implement the AO. 

SSA also missed its planned milestones for implementing the SDM
position.  As with the AO initiative, SSA delayed SDM implementation
until it had more definitive results from the FPM test.  SSA expected
the FPM test to provide better insight into the efficacy of the SDM
position in conjunction with other, related changes. 


   DELAYS IN EVALUATING FPM TEST
   RESULTS
-------------------------------------------------------- Appendix II:2

SSA is behind schedule in assessing the results of its FPM test.  SSA
had planned to assess four of the five FPM process changes--the AO,
the SDM, predecision interview, and elimination of
reconsideration--in fiscal year 1998.  While SSA did not expect to
process all its test cases until November 1999, it believed that it
could use a smaller sample of cases to speed up some of the analysis. 
Specifically, SSA planned to use the first 2 months of test cases to
analyze the effect of the four process changes on accuracy and
allowance rates at all levels and processing time at the DDS
level.\49 While this approach expedited the analysis, SSA was not
able to complete its assessment by the close of fiscal year 1998.  As
of October 1998, SSA could assess the impact of three process
changes--the SDM, predecision interview, and eliminating
reconsideration--on the initial level of decision-making.  However,
data to assess their impact on the hearing level and for the overall
process, as well as data to assess the AO piece, were still
incomplete:  5 percent of the AO cases and 15 percent of the ALJ
cases had not yet been processed. 

Moreover, these 2 months of sample cases did not allow SSA to fully
assess the impact of the proposed changes.  SSA used another sample
of cases selected later in the test process to assess the impact of
the proposed changes on productivity (task time) at all levels and
processing time at the appellate level.  This later sample was needed
in order to give test participants time to adjust to their new
positions so that their productivity could be fairly assessed.  Data
needed for this assessment are incomplete, with approximately 15
percent of the AOs' cases, 49 percent of ALJs' control cases, and 69
percent of ALJs' test cases not yet processed.  Therefore, as of
October 1998, SSA's assessment of the impact of process changes on
productivity and processing time at the hearing level and for the
overall process was still preliminary and incomplete. 


--------------------
\49 SSA believed these 2 months of cohort data would enable it to
reliably assess the effect of FPM because each month includes a
randomly selected and representative group of cases.  Without this
approach, SSA would have had to wait for all the cases to flow
through the process to determine the ultimate impact on allowance
rates and the related cost, as was done with the tests of the AO and
SDM initiatives. 


   MIXED PROGRESS WITH PROCESS
   UNIFICATION
-------------------------------------------------------- Appendix II:3

SSA has made important but incomplete progress with its process
unification initiative to improve the consistency of disability
decisions between the initial and appellate levels.  Concrete actions
taken by SSA under this initiative included completing nationwide
process unification training for 15,000 adjudicators and quality
reviewers.\50 This training brought together staff from different
offices and adjudicative levels in an effort to give adjudicators
consistent training and an opportunity to share differing viewpoints. 
SSA has stated that it has also incorporated process unification
principles into its ongoing training program and has provided
additional cross-component training to adjudicators at all levels. 
Regarding its goal of developing a single presentation of policy, SSA
also compared the guidance used by disability examiners with that
used by ALJs and identified small differences.  Starting in July
1996, SSA has been publishing new policies that are identical for all
levels of adjudicators.  Although SSA has yet to make changes to more
closely align its older policies and procedures, SSA believes that
instructions issued since 1996 address a substantial segment of
important policy areas. 

SSA has completed three of eight subinitiatives under process
unification, although two were completed somewhat later than planned. 
For one subinitiative, SSA issued a final regulation in May 1998 that
clarified its process for acquiescing to--that is, modifying its
policies as a result of--court decisions.  In addition, SSA has
developed an action plan to implement this regulation.  For another
subinitiative, SSA issued a regulation in July 1998 establishing a
new procedure for reviewing OHA allowances in order to identify
factors that contribute to differences between DDS and ALJ decisions. 
SSA instituted these important changes 1 year behind schedule.  SSA
stated that it completed a third subinitiative--ensuring complete
documentation of evidence in cases in which the DDS denies the
claim--through its early process unification training.  However, SSA
has not completed other actions included in its original plan, such
as developing a program circular on documentation requirements and
revising its quality assurance process to support those requirements. 
SSA has been pursuing this subinitiative in concert with its
subinitiative that provides for more detailed explanations of
reconsideration denials.  The combination of these two subinitiatives
has been referred to as the "expanded rationale initiative."

As of October 1998, SSA had not completed, and had made mixed
progress with respect to, the other five subinitiatives.  Final
regulations that address two subinitiatives--concerning the weight of
DDS physicians' opinions and RFC for less than sedentary work--have
been drafted, and SSA expects to issue them in final form in the near
future.  SSA's expanded rationale subinitiative experienced a number
of delays.  After encountering difficulties with an initial pilot
test in 1997, SSA changed the focus of its expanded rationale
subinitiative from more fully documenting only denial determinations
to more fully documenting allowances as well.  The agency did not
begin testing again until early 1998.  After experiencing further
problems with deciding what information to include in the rationale,
SSA issued a notice of its intent to test this subinitiative as part
of the FPM test, beginning on or about October 29, 1998.  For another
subinitiative, SSA has been unable to achieve the goal of, within 1
year, remanding 100,000 appealed cases that had new medical
information from OHA to the DDS, where medical experts reside to
evaluate and reexamine the cases.  SSA began remanding cases on
schedule in July 1997, but after 10 months it had only remanded 8,488
cases to the DDSs.  According to SSA, new evidence was generally
received too late to warrant remanding.  Realizing that it could not
reach its goal of 100,000, SSA changed the remanding criteria to
assist OHA with processing old cases and plans to pursue the original
remanding goals through the other process unification subinitiatives. 
Regarding its subinitiative on treating physicians' opinions, SSA
held a symposium in June 1998 at which experts surfaced related
policy issues on this complex topic.  In October 1998, SSA indicated
it planned to develop a final rule. 


--------------------
\50 Before updating its plan, SSA issued in July 1996 nine SSRs to
clarify policy in areas identified as contributing to inconsistent
decisions. 


   DELAYS IN QUALITY ASSURANCE
-------------------------------------------------------- Appendix II:4

SSA missed key milestones for its quality assurance initiative as
well.  As planned, SSA is developing "in-line" quality assurance
procedures--
approaches for building quality into the process at the beginning--as
it tests various redesign process changes.  For example, as part of
its SDM preimplementation test, SSA asked state agencies to develop
and test approaches for improving the quality of SDM determinations. 
SSA planned to implement these procedures at the same time that it
implemented the SDM and other process changes.  However,
implementation of in-line quality assurance approaches for process
changes such as the SDM has been delayed along with implementation of
the process changes themselves.  Moreover, as of October 1998, SSA
had not assessed the impact that state agency approaches have had on
SDM quality. 

In addition to in-line quality assurance efforts, SSA planned to
develop one standard for its end-of-line quality reviews in fiscal
year 1997 and to test its use in fiscal year 1998.\51 However, as of
the end of fiscal year 1998, the agency had not reached agreement on
what that single standard for quality reviews should be. 

Moreover, as of October 1998, SSA had identified additional and very
important issues and questions that should be addressed in the
context of the quality assurance initiative, including when, how, and
by whom an "end-of-line" review should be conducted; whether
end-of-line quality reviews should be balanced with respect to
percentage of allowance and denial decisions selected for review;
whether the DDS performance standard for quality reviews should be
changed; and whether in-line quality assurance approaches should be
mandatory. 


--------------------
\51 As discussed in ch.  1, SSA quality reviews reflect the
fragmented approach to decision-making at the two levels. 


RESULTS OF SSA'S TESTS
========================================================= Appendix III

SSA has not yet demonstrated that tested initiatives will
significantly improve the process.  For the three tests conducted to
date, SSA collected data to assess whether the initiatives would help
achieve its five redesign objectives.\52 To test its expectation
that, overall, redesign would not change total benefits paid to
claimants, SSA also assessed the impact of tested initiatives on the
overall allowance rate.  Results from SSA's stand-alone tests of the
AO and SDM positions were disappointing and, in the case of the AO,
also inconclusive.  FPM test results are still preliminary but, as of
October 1998, indicated some potential for improving the claims
process at the DDS level.  For example, the combination of three
process changes--that is, the SDM, predecision interview, and
elimination of reconsideration--appears to improve accuracy of
initial determinations and to reduce processing time and
administrative costs at the DDS level, although SSA has not yet fully
analyzed the extent of the improvement.  On the other hand,
preliminary data on the AO's performance in the FPM context were
inconclusive, and SSA has yet to demonstrate any improvements at the
appellate level.  SSA has not yet determined the overall impact of
the four process changes together on the process. 


--------------------
\52 SSA's five redesign objectives are to (1) allow claims that
should be allowed at the earliest possible level; (2) make decisions
as quickly as possible (reduce processing time, or the number of days
needed to make a final decision); (3) make the process more efficient
(reduce task time spent working directly on claims, administrative
costs, or both); (4) make the process user-friendly; and (5) provide
a satisfying work environment for employees. 


   AO STAND-ALONE TEST RESULTS
   WERE DISAPPOINTING AND
   INCONCLUSIVE
------------------------------------------------------- Appendix III:1

Results from the stand-alone test of the AO position generally fell
short of SSA's early targets, and SSA has not been able to
demonstrate whether the AO position is meeting SSA's redesign
objectives.  Regarding SSA's objective of correctly allowing more
claims at the earliest possible level, the AO position appears to
have improved the accuracy of allowances but did not increase the
number of allowances made earlier in the process.  SSA had expected
the AO to help meet this objective because each allowance made
correctly by an AO would represent a claim that would not require a
hearing.  Early results from the AO test indicated that AOs were
allowing 24 percent of cases.  However, early AO allowances were not
clearly more accurate than comparable decisions made under the
current process.  Comparable decisions are "on-the-record" allowances
made by either ALJs or other hearing office staff for claims that are
relatively clear-cut and do not require a hearing.\53

However, after almost 3 years of testing--half of which involved
increased training and feedback for test participants--AO accuracy
has improved, and AO allowance decisions are currently more accurate
than comparable decisions made under the current process.\54

On the other hand, the percentage of claims allowed by the AOs has
declined over time, and SSA has not demonstrated that the current
level of early allowances made by the AO represents an improvement
over the current process.  According to SSA officials, as of October
1998, AOs had been allowing about 15 percent of appealed cases.  Data
we obtained from OHA indicated that, as of October 1998, ALJs and OHA
staff had together allowed over 17 percent of appealed cases without
a hearing. 

Test results for two other redesign objectives--reducing processing
time and improving efficiency--fell short of SSA's early targets and
did not clearly demonstrate an improvement over the current process. 
For example, as of April 1998, AOs needed an average of 89 days to
allow cases and 103 days to prepare them for a hearing, compared with
SSA's early targets of 45 and 60 days.  In addition, SSA originally
expected AOs to process an average of two cases a day, but instead
AOs processed, on average, less than one case per day.  Comparisons
with SSA's early targets are somewhat misleading, according to SSA,
because these targets assumed that key supports for the AO would be
in place, while the test did not include such supports.  To make a
fairer assessment, SSA selected a small group of control cases to be
handled under the current process with whose times the AO processing
and task times could be compared.  However, as discussed in chapter
2, OHA staff did not handle AO and control cases as instructed.\55
Without control cases, SSA could not determine with certainty
whether, or the extent to which, the AO position reduced overall
processing or task time.\56 On a positive note, judges considered 85
percent of cases prepared by AOs to be fully developed and ready for
a hearing. 

Test results for the other two redesign objectives--to make the
process user-friendly and provide a satisfying work environment for
employees--were somewhat mixed.  An early focus group study showed
claimants preferred the AO process, but more recent survey data did
not show any clear preference.  Two-thirds of claimants' attorneys
surveyed in December 1997 viewed the AO process as an improvement
over the traditional hearing process.\57 With respect to improving
the work environment, AO test participants surveyed by SSA generally
liked the new process, but other OHA staff were more neutral about
the AO initiative.\58

Finally, test results indicated that the AO position would
significantly increase overall allowance rates at the appellate
level.  Specifically, the combined rate of AO allowances and ALJ
allowances based on AO-prepared cases was 56 percent as of March
1998, which is 4 percent higher than the overall allowance rate for
the appellate level under the current process.  Assuming no change in
allowance rates at the initial level, such an increase in allowances
at the appellate level would translate into an increase in overall
benefit payments; however, SSA did not estimate the impact on benefit
payments. 


--------------------
\53 A comparison with all allowances made at the appeals level would
be misleading because AOs were expected to allow only those cases
that were straightforward, whereas most allowances at the appeals
level are made by ALJs and generally involve more difficult
adjudicative issues. 

\54 Different standards exist for reviewing the accuracy of
disability decisions, and opinions differ regarding which standard of
review is appropriate to apply.  SSA used three standards to assess
the relative accuracy of AO allowances and, as of September 1998, AO
allowances were higher than comparable decisions under the current
process for all three standards. 

\55 To fairly assess processing and task times, OHA staff were
instructed to promptly schedule and hold hearings for both AO and
control cases. 

\56 Comparing cases allowed or prepared by AOs with cases handled
routinely under the current system is inappropriate.  Unlike AO
cases, cases handled under the current system are generally placed in
a lengthy backlog of cases, which can increase both the processing
and task times needed to reach a final disposition.  Comparing AO
allowances with allowances for cases without a hearing under the
current process would be more appropriate; however, SSA has not
collected the data needed to make such a comparison. 

\57 However, a spokesperson representing an organization of SSA
claimants' attorneys that we contacted indicated that the group does
not support SSA's AO initiative. 

\58 Six internal stakeholder groups that we contacted had the
following views of the AO initiative:  two supported the AO concept,
two opposed it, and two did not have a position but did have concerns
about its effectiveness. 


   SDM STAND-ALONE TEST RESULTS
   WERE MARGINAL AND MIXED
------------------------------------------------------- Appendix III:2

Results of SSA's stand-alone test of the SDM initiative were marginal
and mixed and did not clearly indicate that, by itself, the SDM would
significantly aid SSA in meeting its redesign objectives.  SDM test
results for SSA's objective to correctly allow more cases at the
earliest level were mixed.  Although the SDM allowance rate was 1.2
percent higher than that of the control group representing the
current process, SDM accuracy was slightly lower overall and
significantly lower for denial determinations.  In particular,
analysts identified nine impairments that were likely to be
associated with determination errors, such as diabetes and asthma,
for which the accuracy of SDM determinations was markedly lower than
the accuracy of determinations made under the current process. 
Maintaining accuracy is critical because incorrect determinations can
result in unnecessary costs.  For example, SSA staff unofficially
estimated that the relatively high number of inaccurate allowances by
the SDM could result in inappropriate benefits payments of $65
million, which would outweigh any administrative cost savings.  And
inaccurate denials could cause unnecessary reconsiderations and
appeals, resulting in higher administrative costs--which SSA
estimated at $2 million--as well as inconvenience, or even financial
hardship, for some claimants.  On the other hand, SSA estimated that
the increase in the number of allowances that the SDM made correctly
could save $9.5 million as a result of fewer reconsiderations and
appeals. 

The SDM's contribution toward meeting two other redesign objectives
was marginal.  By making the SDM solely responsible for the initial
disability determination, instead of jointly responsible along with a
DDS physician, SSA expected the SDM to reduce processing time and
improve efficiency.\59 However, test results indicated that the SDM
process saved only 1 day, compared with the current process.  Test
results also showed a marginal decrease in task time (time spent
working directly on cases).  Specifically, the DDS physician task
time decreased by 7.1 minutes per case, offset somewhat by an
increase in SDM task time by 3.5 minutes, for an overall decrease of
3.6 minutes per case.  The potential administrative savings
associated with this process improvement was estimated by SSA to be
$16.1 million. 

Test results for the objective of providing a satisfying work
environment for employees were generally positive.  SSA's survey of
test participants indicated the following:  DDS managers believed the
SDM model to be an effective use of examiner and consultant
resources, disability examiners generally liked the process, and most
medical consultants believed that SDMs performed good work and
required less input.\60 SSA did not test whether the SDMs would make
the process user-friendly because this process change is transparent
to the customer. 

With respect to the SDM's impact on overall benefit payments, SSA
estimated that the increase in SDM allowances would translate into an
increase of $2.1 billion in program costs.  SSA did not measure the
impact of SDM on the overall allowance rates--that is, by tracking
SDM denial determinations that were appealed through to the final
decision--and the associated program costs. 

In its phase II test of SDM, SSA is exploring different approaches
for improving the accuracy, productivity, and processing time of the
SDM.  Preliminary data from this test suggest that SDM accuracy and
processing time might be improving.  However, as of June 1998, data
from the phase II test were not weighted to produce reliable
estimates of accuracy or productivity. 


--------------------
\59 SSA expected the SDM to consult with DDS physicians on an
as-needed basis so that physicians could concentrate on the more
complex cases, thereby reducing task and processing time. 

\60 Key stakeholder groups we contacted also supported the SDM
concept, with the exception of groups representing DDS physicians
from two states. 


   PRELIMINARY FPM RESULTS WERE
   PROMISING BUT INCONCLUSIVE
------------------------------------------------------- Appendix III:3

As of October 1998, preliminary results of the FPM test showed
promise for achieving some of SSA's five redesign objectives at the
DDS level of the process; however, SSA has not determined the extent
of potential improvements.  As of October 1998, SSA had not processed
enough cases, and performed sufficient analysis, to demonstrate
improvement at the appellate level. 

Preliminary results indicate that, with the addition of the
predecision interview, the FPM brings SSA closer to its objective of
correctly allowing more cases earlier in the process--that is, at the
DDS level; however, progress toward this objective has not been
demonstrated at the appellate level.  For initial determinations made
under FPM, accuracy improved and the allowance rate increased,
compared with the current process.\61 It is unclear whether the
100-percent review of initial determinations in the FPM test affected
relative accuracy and allowance rates.\62 Moreover, if allowances
made at the reconsideration step are included in the analysis, the
overall allowance rates at the DDS level under the current process
and under FPM are similar:  36.8 and 36.5, respectively.  SSA
preliminary findings are less conclusive at the appellate level.  As
of October 1998, SSA had not completely processed all the test cases
through the appellate level.\63 Also, SSA did not completely assess
the AO's performance with respect to this objective.  For example,
SSA compared the accuracy of AO allowances with that of all other
allowances made at the appellate level under the current process.  As
discussed previously, a more appropriate comparison would have been
with those cases allowed under the current process--by either ALJs or
other OHA staff--that did not require a hearing:  that is, similarly
clear-cut cases. 

With the elimination of reconsideration, the FPM appears to
contribute to reductions in processing time (measured in days) and
task time (measured in minutes or work-years saved) at the DDS level. 
This analysis is incomplete, however, and SSA has not determined the
impact of changes at the appellate level or on the process overall. 
Because of the predecision interview, the SDM required more days and
task time to reach a final determination, as compared with the
current process.\64 On the other hand, the elimination of
reconsideration under FPM has potential for significantly lowering
the overall number of days and task time needed to reach a final
determination at the DDS level.  As of October 1998, SSA estimated
that FPM would save 2,600 staff-years valued at $185 million because
of reduced task time spent on cases at the DDS level.  However, SSA's
test results did not include an assessment of overall reductions in
processing time at the DDS level.  Moreover, SSA had not demonstrated
the impact of FPM changes on processing and task time at the
appellate level and on the process overall.  SSA's test results were
incomplete, in part, because it had not processed all the test cases
through the appeals process.  In particular, SSA lacked a sufficient
number of cases to assess the impact of FPM on task time at the
appellate level.  In addition, SSA results did not include a
comparison of the overall processing and task time needed at the
appellate level to reach a final decision under FPM--either by an AO
or an ALJ using an AO-prepared case--with the time needed under the
current process. 

Test results did not demonstrate whether FPM would make the process
more user-friendly, although there were indications that FPM might
provide a more satisfying work environment.  Customer survey results
indicated that customer satisfaction was greater for those awarded
after a predecision interview than for those denied and that the
predecision interview did not alter the overall satisfaction of
claimants who were denied.  The survey also revealed that claimants
found the letter sent by SSA to inform them of the interview to be
confusing.  With respect to the work environment, a majority of FPM
test participants indicated that they believed that the new process
offered them the ability to serve the claimant better.  In addition,
most test participants felt that their new duties had a positive
effect on their job satisfaction. 

Finally, preliminary test results indicated that the FPM might result
in a higher overall allowance rate and benefit payments, although
more cases need to be processed and evaluated to reach a final
conclusion.  As of October 1998, SSA estimated an increase of 2
percent in the overall allowance rate under FPM but had not yet
estimated the impact on benefit payments. 



(See figure in printed edition.)Appendix IV

--------------------
\61 Using a predecision interview, the SDM's accuracy rate for
initial determinations that resulted in denials was 2 percent higher
than under the current process; the accuracy rate for allowances was
comparable to that of the current process.  In addition, the
allowance rate was 4.7 percent higher than that of the current
process. 

\62 All initial determinations in the FPM test were reviewed for
accuracy, whereas the comparison group received its normal, lower
level of review that focuses on allowance decisions.  The difference
in level and focus of the reviews could have affected the relative
behavior of FPM test participants. 

\63 As discussed in app.  II, SSA used two different groups of cases
for its assessment of the FPM.  As of Oct.  1998, 5 percent of the
2-month group of test cases used to assess accuracy and allowance
rates had still not been completed by AOs, and 15 percent had not
been completed by ALJs.  For the different set of cases used to
assess task time and processing time, as of Oct.  1998, 15 percent
had not been completed by AOs, and more than 50 percent had not been
completed by ALJs. 

\64 With the predecision interview, the SDM required, on average, 11
more days to make initial allowances and 24 more days to make denials
than under the current process.  Overall, the SDM's task time was
16.3 minutes longer than that of the current process for initial
determinations. 


COMMENTS FROM THE SOCIAL SECURITY
ADMINISTRATION
========================================================= Appendix III



(See figure in printed edition.)



(See figure in printed edition.)



(See figure in printed edition.)



(See figure in printed edition.)

GAO CONTACTS AND STAFF ACKNOWLEDGMENTS

GAO CONTACTS

Kay Brown, Assistant Director, (202) 512-3674
Michele Grgich, Evaluator-in-Charge, (415) 904-2183

STAFF ACKNOWLEDGMENTS

In addition to those named above, Julie M.  DeVault, Carlos J. 
Evora, John M.  Ortiz, and Robert T.  Tomco made major contributions
to collecting and synthesizing data.  Ms.  DeVault, Mr.  Tomco, and
James P.  Wright contributed significantly to writing this report,
and Sharon L.  Caudle and William E.  Hutchinson provided valuable
technical assistance throughout the review. 


BIBLIOGRAPHY
============================================================ Chapter 1

Brynjolfsson, Erik, and others.  "The Matrix of Change." Sloan
Management Review (winter 1997), pp.  37-54. 

Caudle, Sharon L.  Reengineering for Results:  Keys to Success From
Government Experience.  Washington, D.C.:  National Academy of Public
Administration, 1995. 

_____.  "Reengineering:  Avoiding Becoming Lost in Space." The Public
Manager (spring 1998), pp.  27-30. 

Guha, Subo, and others.  "Business Process Change and Organizational
Performance:  Exploring an Antecedent Model." Journal of Management
Information Systems, 14 (1) (summer 1997), pp.  119-54. 

Hall, Gene, Jim Rosenthal, and Judy Wade.  "How to Make Reengineering
Really Work." Harvard Business Review (Nov.-Dec.  1993), pp.  119-31. 

Hammer, Michael, and James Champy.  Reengineering the Corporation. 
New York:  Harper Collins Publishers, 1993. 

_____, and Steven A.  Stanton.  "The Reengineering Revolution."
Government Executive (Sept.  1995), pp.  2A-12A. 

Linden, R.M.  Seamless Government:  A Practical Guide to
Re-engineering in the Public Sector.  San Francisco:  Jossey-Bass
Publishers, 1994. 

ProSci.  Best Practices Report for Managing Change.  Loveland, Colo.: 
ProSci, 1998. 

Secretary of Health and Human Services.  Implementation of Section
304 (g) of Public Law 96-265, Social Security Disability Amendments
of 1980 (the Bellmon Report).  Washington, D.C.:  HHS, Jan.  1982. 

Social Security Advisory Board.  How SSA's Disability Programs Can Be
Improved.  Washington, D.C.:  Social Security Advisory Board, Aug. 
1998. 

SSA, Office of Program Integrity Reviews.  Findings of the Disability
Hearings Quality Review Process.  Washington, D.C.:  SSA, Sept. 
1994. 



RELATED GAO PRODUCTS
============================================================ Chapter 2

Social Security Disability:  SSA Making Progress in Conducting
Continuing Disability Reviews (GAO/HEHS-98-198, Sept.  18, 1998). 

Social Security Reform:  Raising Retirement Ages Improves Program
Solvency but May Cause Hardship for Some (GAO/T-HEHS-98-207, July 15,
1998). 

Social Security Administration:  Technical Performance Challenges
Threaten Progress of Modernization (GAO/AIMD-98-136, June 19, 1998). 

The Results Act:  Observations on the Social Security
Administration's Fiscal Year 1999 Performance Plan (GAO/HEHS-98-178R,
June 9, 1998). 

Social Security Administration:  Software Development Process
Improvements Started but Work Remains (GAO/AIMD-98-39, Jan.  28,
1998). 

Social Security Disability:  SSA Must Hold Itself Accountable for
Continued Improvement in Decision-Making (GAO/HEHS-97-102, Aug.  12,
1997). 

Business Process Reengineering Assessment Guide (GAO/AIMD-10.1.15,
Apr.  1997, Ver.  3). 

SSA:  Significant Challenges Await New Commissioner (GAO/HEHS-97-53,
Feb.  20, 1997). 

SSA Disability Redesign:  Focus Needed on Initiatives Most Crucial to
Reducing Costs and Time (GAO/HEHS-97-20, Dec.  20, 1996). 

Appealed Disability Claims:  Despite SSA's Efforts, It Will Not Reach
Backlog Reduction Goal (GAO/HEHS-97-28, Nov.  21, 1996). 

SSA Disability Redesign:  More Testing Needed to Assess Feasibility
of New Claim Manager Position (GAO/HEHS-96-170, Sept.  27, 1996). 

Social Security Disability:  Backlog Reduction Efforts Under Way;
Significant Challenges Remain (GAO/HEHS-96-87, July 11, 1996). 

Executive Guide:  Effectively Implementing the Government Performance
and Results Act (GAO/GGD-96-118, June 1996). 


*** End of document. ***