-------------------------Indexing Terms-------------------------
REPORTNUM: GAO-06-538
TITLE: Equal Employment Opportunity: DOD's EEO Pilot Program
der Way, but Improvements Needed to DOD's Evaluation Plan
DATE: 05/05/2006
-----------------------------------------------------------------
******************************************************************
** This file contains an ASCII representation of the text of a **
** GAO Product. **
** **
** No attempt has been made to display graphic images, although **
** figure captions are reproduced. Tables are included, but **
** may not resemble those in the printed version. **
** **
** Please see the PDF (Portable Document Format) file, when **
** available, for a complete electronic file of the printed **
** document's contents. **
** **
******************************************************************
GAO-06-538
* Report to Congressional Addressees
* May 2006
* equal employment opportunity
* DOD's EEO Pilot Program Under Way, but Improvements Needed to
DOD's Evaluation Plan
* Contents
* Results in Brief
* Background
* The Three Programs Emphasize ADR Techniques, Share Common
Implementation Strategies, and Report Low Case Activity
* DOD's Pilot Program Emphasizes ADR Techniques
* DLA's Pilot Program
* DeCA's Pilot Program
* USAF's Pilot Program
* DOD's EEO Pilot Programs Share Common Implementation
Strategies
* DOD's EEO Pilot Programs Report Low Case Activity
* Although Containing Some Strengths, Limitations in Its Evaluation
Plan Will Hinder DOD's Ability to Assess Pilot Program Results
* Strengths of DOD's Evaluation Plan
* Without Key Evaluation Plan Features, DOD Will Be Limited in
Its Ability to Assess Pilot Programs' Results
* Conclusions
* Recommendations for Executive Action
* Agency Comments
* EEO Laws and Regulations Applicable to Federal Employees
* Comments from the Department of Defense
* GAO Contact and Staff Acknowledgments
Report to Congressional Addressees
May 2006
EQUAL EMPLOYMENT OPPORTUNITY
DOD's EEO Pilot Program Under Way, but Improvements Needed to DOD's
Evaluation Plan
Contents
Figures
May 5, 2006Letter
Congressional Addressees
Federal employees or applicants for employment who allege that they have
been discriminated against by a federal agency may file a complaint with
that agency.1 The Equal Employment Opportunity Commission (EEOC) has
established procedures for federal agencies to process equal employment
opportunity (EEO) complaints, including time frames for taking certain
actions, and use of alternative dispute resolution (ADR) programs.2 The
federal EEO complaint process consists of two stages-informal, or
precomplaint counseling, and formal, when a complaint is filed with the
agency. Delays in the processing of EEO complaints have been a
long-standing concern of EEOC, other federal agencies, and Congress. As an
example of such delays, from fiscal years 1993 to 2003, the time the
federal government took to investigate a formal EEO complaint rose from an
average low of 171 days to an average high of 343 days.3
In 2000, as part of DOD's fiscal year 2001 authorization act,4 Congress
authorized the Department of Defense (DOD) to carry out a 3-year pilot
program for improving processes for the resolution of EEO complaints by
civilian employees of DOD. The legislation provided that the pilot program
was to include procedures to reduce EEO complaint processing times,
eliminate redundancy, reinforce accountability, and provide for early
resolution. While not prescribing details of how the pilots were to be
designed or operated, the authorizing legislation provided that complaints
processed under the pilot program shall be subject to the procedural
requirements established for the pilot program and exempt from the
procedural requirements or other regulations, directives, or regulatory
restrictions prescribed by EEOC. The programs DOD authorized in August
2004 were in (1) the Defense Commissary Agency (DeCA), (2) Defense
Logistics Agency (DLA), and (3) components of the U.S. Air Force (USAF),
and they became operational in fiscal year 2005.
The 2001 authorization act requires GAO to submit two reports on the pilot
program-90 days after the first and last fiscal years of pilot program
operation. We provided briefings on the results of our initial review of
common features and preliminary case activity for the three programs to
interested congressional committees in December 2005 and January 2006.
This report provides additional information by (1) describing key features
and status of the programs and (2) assessing DOD's plan for evaluating the
effectiveness of the pilot program.
To describe the three programs and their key features, we reviewed
documents provided by DOD officials and interviewed those officials. In
addition, we reviewed information about the federal regulations governing
the federal EEO complaint process6 To assess DOD's plan 5 and our
reports.for evaluating the effectiveness of the pilot program, we reviewed
the plan and DOD's 6- and 9-month evaluation reports. We also interviewed
DOD officials about the evaluation reports. We consulted social science
and evaluation literature, along with our publicized guidance, to identify
key features of an evaluation plan. On the basis of criteria gathered from
these sources, we identified strengths and limitations of DOD's plan. As
the pilot program and the evaluation efforts were already under way when
we began our review, we focused on the plan's strengths and those specific
limitations where it would still be possible for DOD to implement
improvements.
Program officials provided data on case activity for the three programs.
The data are preliminary and because we do not use them to develop
findings, we did not conduct a data reliability assessment. Because we did
not assess the reliability of the data, the data are of undetermined
reliability.
We conducted our review in the Washington, D.C., metropolitan area from
August 2005 through March 2006 in accordance with generally accepted
government auditing standards.
Results in Brief
All three programs are under way and emphasize the use of ADR techniques
to resolve allegations before they reach the formal complaint stage-an
option already available under the current EEO regulations and encouraged
in DOD's memo soliciting pilot program proposals. While the legislation
provided DOD considerable latitude in designing its program, to a large
extent, two of the three pilot programs DOD authorized were designed to
operate within the parameters existing under law and EEOC regulations. In
the case of DLA, program officials indicated that the program's ADR
features are the same as those in the current EEO process. DeCA's program
emphasizes early resolution before the start of the informal precomplaint
stage of the current process through the use of a trained EEO facilitator
who attempts to negotiate resolution. In addition, according to a DeCA
official, DeCA's program seeks to reduce processing time frames in the
formal stage of the complaint process and replaces paper documents with
electronic files. Unlike the other two programs, USAF's focuses on the
formal stage, combining the two-step investigative and hearing phases of
the current EEO process into a single fact-finding process aimed at
reducing processing time and including voluntary participation in ADR. The
three programs also featured common implementation strategies-DeCA's and
USAF's programs conducted outreach to inform eligible staff about the
programs, and all three programs conducted staff training and used
electronic data collection. According to DOD and the pilot program
officials, as a result of the low case activity, program officials will
seek to extend their respective programs for an additional (third) year.
Our initial assessment of DOD's evaluation plan for the pilot program
found both strengths and limitations. One strength of the plan was the
inclusion of forms for collecting baseline data (before the programs
began) and program data, which provides a tool for the programs to measure
some aspects of their progress. Although DOD developed an evaluation plan
for the overall pilot program, the plan lacked some key features of a
sound evaluation plan, including measures that are directly linked to the
program objectives, criteria for determining individual pilot program
performance, and an appropriate data analysis plan for the evaluation
design. The lack of established key evaluation features in DOD's plan
increases the likelihood of insufficient data, further limiting confidence
in pilot program results. Without confidence in pilot program results, DOD
will be limited in its decision making regarding this pilot program, and
Congress will be limited in its decision making about the pilot program's
potential broader application.
We provided a draft of this report to the Secretary of Defense for his
review and comment. DOD generally concurrred with our recommendations.
Concerning our recommendation that DOD establish regular intra-agency
exchange of information, DOD stated that it would begin convening
quarterly meetings with DOD pilot program managers to discuss and exchange
relevant information regarding pilot implementation processes. As for the
recommendation to develop a sound evaluation plan, DOD partially concurred
and stated that it will consider and incorporate our recommended key
features into its pilot program evaluation plan as appropriate. DOD's
written comments are reprinted in appendix II.
Background
As of January 1, 2000, all federal agencies covered by EEOC regulations
were required to establish or make available an ADR program for both the
informal and formal complaint stages of the EEO process.
On March 9, 2000, at a joint hearing held by the Subcommittee on Civil
Service of the House Committee on Government Reform and the Subcommittee
on Military Readiness of the House Armed Services Committee, the Navy
discussed the results of its experiences under its 18-month pilot program
for resolving EEO complaints through the use of ADR, which resulted in
resolution on an average of 31 days.
The Floyd D. Spence National Defense Authorization Act, for fiscal year
2001, authorized the Secretary of Defense to carry out at least three
pilot programs-one at a military department and two at DOD agencies. The
programs were authorized to operate for 3 years. The act exempts the
programs from EEOC's procedural requirements or restrictions.7
In 2004, DOD authorized the following as pilot programs: (1) DLA, which
provides worldwide logistics support-munitions and supplies-for the
missions of military departments; (2) DeCA, which operates a worldwide
chain of commissaries providing groceries to military personnel, retirees,
and their families at a discount; and (3) 31 bases of the USAF, accounting
for about one-third of USAF bases with federal EEO programs. The pilot
programs were authorized by the Secretary for 2 years with an option for
an additional (third) year.
The legislative objectives for the programs are to:
o reduce processing time,
o eliminate redundancy,
o reinforce local management and chain of command accountability, and
o provide the parties involved with early opportunity for resolution.
The legislation also provides that pilot program participants voluntarily
participate in the pilot program, and that participants maintain their
right to appeal final agency decisions to EEOC and file suit in federal
district court as is the case in the federal EEO complaint process.
The Office of the Deputy Undersecretary of Defense for Civilian Personnel
Policy, the Office of the Deputy Undersecretary for Equal Opportunity, and
the Office of Complaint Investigations within the Civilian Personnel
Management Service have ongoing responsibility for oversight, monitoring,
and evaluation of the overall pilot program.
Under EEOC regulations, during the informal, or precomplaint counseling
stage, ADR techniques can be used. Counselors are to advise individuals
that, when the agency agrees to offer ADR in the particular case, they may
choose to participate in either counseling or in ADR. If the matter is not
resolved by counseling or if ADR is unsuccessful, the counselor is
required to inform the employee in writing of his or her right to file a
formal discrimination complaint with the agency. ADR can also be used
after an agency receives a formal complaint.
After a complainant files a formal discrimination complaint, the agency
must decide whether to accept or dismiss the complaint and notify the
complainant. If the agency dismisses the complaint, the complainant can
appeal the dismissal to EEOC.8 If the agency accepts the complaint, it has
180 days to investigate the accepted complaint and provide the complainant
with a copy of the investigative file.9 Within 30 days of receipt of the
copy of the investigative file, the complainant must choose between
requesting (1) a hearing and decision from an EEOC administrative judge
(AJ)10 or (2) a final decision from the agency. When a hearing is not
requested, the agency issues a final decision. A complainant may appeal an
agency's final decision to EEOC.
In cases where a hearing is requested, the AJ has 180 days to issue a
decision and send the decision to the complainant and the agency. If the
AJ issues a finding of discrimination, he or she is to order appropriate
relief. After the AJ decision is issued, the agency can issue a final
order notifying the complainant whether or not the agency will fully
implement the decision of the AJ, and the employee can file an appeal with
EEOC.11 If the agency issues an order notifying the complainant that the
agency will not fully implement the decision of the AJ, the agency also
must file an appeal with EEOC at the same time. See appendix I for more
details and associated time frames related to the EEO complaint process.
The Three Programs Emphasize ADR Techniques, Share Common Implementation
Strategies, and Report Low Case Activity
Although features of the three programs vary by agency and focus on
different stages of the complaint process, they all emphasize the use of
ADR techniques available under the current federal EEO process. They also
share common implementation strategies, including outreach to eligible
staff to inform them about the programs, staff training, and electronic
data collection. In its 9-month evaluation, DOD observed that pilot
program activity had been lower than anticipated; DOD did not provide a
baseline for its comparison or elaborate on the reason for this
occurrence. After 12 months, program officials continue to report low case
activity.
DOD's Pilot Program Emphasizes ADR Techniques
In developing the overall EEO pilot program, DOD allowed DLA, DeCA, and
USAF to determine their individual program design. However, in its memo
soliciting pilot program proposals, DOD encouraged potential participants
to work with the Office of Complaint Investigations to develop the format
and content of their proposals, offering the assistance of the Office's
experienced staff of certified complaint investigators and mediators with
success in using ADR techniques. Two of the programs-DLA and
DeCA-emphasize the use of ADR in the informal stage, consistent with
federal EEO regulations. Program officials said that their programs are
attempting to address the legislative objective of providing early
opportunity for resolution by focusing on ADR. The third program, in
selected bases of the USAF, changes the formal stage of the federal EEO
process by combining the investigative and hearing phases after a
complainant has filed a formal complaint. This program also emphasizes the
use of ADR techniques both during the informal stage as well as at the
time a complainant files a formal complaint.
DLA's Pilot Program
DLA's program, Pilot for Expedited Complaint Processing (PECP), began in
October 2004 at DLA headquarters in Fort Belvoir, Va.12 DLA considers
several types of cases, such as those that challenge government policy,
inappropriate for PECP and screens them out.13
The PECP process is similar to the informal stage of the current EEO
process. DLA officials said the PECP process has three steps. The first
step occurs when an employee who believes he or she has been discriminated
against makes initial contact with DLA's EEO office. An EEO Intake
Specialist collects specific information about the employee's concerns and
drafts an intake report, which includes a description and basis of the
claim. The EEO Intake Specialist advises the employee orally and in
writing about (1) PECP and how it compares to the federal counseling
process and (2) the employee's right to opt out of the pilot program at
any time before the filing of a formal complaint. The second step begins
when the employee chooses to participate in PECP. At this time, the EEO
Intake Specialist discusses and offers the employee ADR. The EEO Intake
Specialist also informs the employee that participating in ADR is optional
and can be used at any stage of the complaint process. The EEO Intake
Specialist considers two methods of ADR- mediation or facilitation.
Mediation is the primary method used by PECP. 14 According to DLA, the
method of ADR used is based on the employee's claim and the EEO Intake
Specialist's assessment of the method that would more likely encourage
communication between the employee and management and resulting
resolution.
Under the third step, ADR takes place. DLA pilot program officials
acknowledged that the pilot program's ADR features do not differ from
those offered under the current EEO process. DLA has an ADR program called
Reach Equitable Solutions Voluntarily and Easily (RESOLVE), which is used
when mediation is offered. RESOLVE is managed by DLA's General Counsel.
According to DLA officials, RESOLVE mediators cannot mediate precomplaints
or complaints involving organizations they may service in another
capacity, thus ensuring the neutrality of the mediator.15
DeCA's Pilot Program
DeCA's program, Early Resolution Opportunity (ERO), began in February 2005
and covers 23 stores16 in three zones (DeCA West Zone 16-San Diego,
Calif.; DeCA East Zone 28-Virginia Beach, Va; and DeCA East Zone 6-San
Antonio, Tex.). Using ADR techniques, ERO seeks to provide early
resolution opportunities, because according to DeCA, ineffective
communication between employees and supervisors or managers often results
in perceptions of discrimination. Moreover, DeCA believes that disputes
can be resolved before they enter the informal counseling stage if a
trained EEO facilitator17 can intervene to negotiate resolution. Cases
that involve alleged violent acts, theft, sexual harassment, termination,
or may be precedent setting, are ineligible for ERO.18
ERO is divided into two steps. In the first step, a trained DeCA
facilitator attempts to resolve a claim before the start of the informal
stage of the current process. Employees at stores participating in ERO can
call a toll-free number to discuss their concerns with a trained
facilitator. For example, an employee could call about perceived
discrimination over schedule changes, and the facilitator may discuss what
had occurred and rationale for schedule changes (e.g., to cover absences).
According to DeCA officials, some employees "self screen" during the
facilitation process, deciding not to pursue an EEO complaint or to pursue
another avenue, such as the negotiated grievance process.19
The second step of ERO, which follows if facilitation is unsuccessful in
resolving the employee's concerns, involves calling in a third-party
mediator. According to a DeCA official, DeCA uses mediators from DOD's
Office of Complaint Investigations, because they are trained, experienced
ADR professionals, and have a greater perception of neutrality as they do
not work for DeCA.
If mediation fails, an individual may choose to file a formal complaint.
According to a DeCA official, ERO seeks to reduce the processing time of
the formal stage. To help achieve this goal, DeCA reduces processing times
for two phases of the formal stage of the complaint process: (1) after a
complainant files a formal complaint, DeCA has set a goal in ERO of 14
days to accept, partially accept, or dismiss it; and (2) after the report
of investigation is completed, DeCA sends a notice informing the
complainant that he or she has 7 days to either request a hearing or a
final agency decision, reducing the time from 30 days under EEOC
regulations. In addition, to further reduce processing time for ERO cases,
paper documents are replaced with electronic files.20 Finally, according
to a DeCA official, officials from DeCA and the Office of Complaint
Investigations can download relevant case documents from a secure shared
drive for complaints filed under both ERO and under the current EEO
process.
USAF's Pilot Program
USAF's program, called Compressed Orderly Rapid Equitable (CORE), focuses
on the formal phase of the EEO complaint process. The program began
January 1, 2005, at 29 continental U.S. sites and 2 overseas offices that
we refer to as test bases.21 Although the 31 test bases account for less
than one-third of all USAF bases with EEO programs, they produce over 80
percent of all USAF EEO complaints. Cases that involve class and
mixed-case complaints22 or cases related to claims already accepted under
the current federal EEO complaint process are not eligible to participate
in CORE.23
CORE has a two-step process that begins at the time the complainant files
a formal complaint. Until a complaint is filed, USAF officials attempt
early resolution of allegations of discrimination in the informal stage
using the current federal EEO process.24 If resolution is not achieved
during this stage, the complainant must choose between CORE and the
current federal EEO process. The first step of CORE involves mediation. If
the complainant declines mediation or mediation is unsuccessful, step two
begins, and a CORE Fact-Finding Conference is conducted. USAF defines this
conference as a "non-adversarial, impartial fact-gathering procedure." The
conference is conducted by a CORE fact-finder, provided by the Office of
Complaint Investigations. During the conference, the fact-finder hears
testimony from witnesses and receives documentary evidence; also at this
time, a verbatim transcript is taken by a certified court reporter.
Following the conference, the fact-finder completes the record of the
complaint and recommends a decision25 in the case to the director of the
USAF Civilian Appellate Review Office. The director of the USAF Civilian
Appellate Review Office may accept, reject, or modify the fact-finder's
recommended decision. The director then prepares a final agency decision
for signature by the director of USAF Review Boards Agency.26 The director
of USAF Review Boards Agency issues the final agency decision. Any further
action on the complaint, including rights to appeal to EEOC and file a
lawsuit, are governed by current federal EEO complaint procedures.
According to USAF, by combining the investigative and hearing phases of
the current federal EEO complaint process, USAF aims to issue a final
agency decision within 127 days or less of filing the formal complaint;
the current process can take up to 360 days plus another 70 days to
provide the complainant and the agency their allotted time for decision
making. USAF officials also indicated that through the CORE Fact-Finding
Conference, each complainant gets their "day in court," whereas under the
current EEO process, complainants often wait months to request a hearing
and can have their complaint dismissed by an EEOC AJ without a hearing.
DOD's EEO Pilot Programs Share Common Implementation Strategies
The three programs share common implementation strategies but implement
them differently. In our review of the programs and subsequent discussions
with DOD and program officials, DeCA and USAF conducted some level of
outreach to program-eligible employees to inform them about the programs.
For example, DeCA officials went to participating stores and handed out
brochures describing ERO. According to USAF program officials, outreach on
CORE included sending a letter to all participating bases from the Chief
of Staff for Personnel as well as a notice to the unions. Additionally,
CORE was publicized in USAF news service and governmentwide media.
We also found that agencies varied in how they trained their EEO employees
about the programs. USAF officials used contractors to train some
employees in CORE over a 1-week period; in turn, those employees trained
others. DLA officials had informal in-house employee training. DeCA sent
EEO officials and an attorney from its headquarters trained in ERO to each
of its three zones to train EEO managers as well as managers and
supervisors at its 23 stores.
Finally, all three programs used electronic data collection for tracking
and monitoring, with each program developing its own electronic data
collection method. For example, USAF uses EEO-Net system and software to
collect program data.27 USAF also uses USAF-specific software, the Case
Management and Tracking System, to manage the EEO process, including CORE,
and an electronic case identifier to mark CORE cases to help in monitoring
those program cases that reach EEOC on appeal. DeCA currently uses an
Access database to track ERO activity, and DLA uses an Excel spreadsheet
to track PECP activity.
Officials from both the programs and DOD's EEO pilot program oversight
entities have indicated their willingness to share information. As we have
previously reported, by assessing their relative strengths and limitations
through collaboration,28 agencies can look for opportunities to address
resource needs by leveraging each others' resources and obtaining
additional benefits that would not be available if they were working
separately. While the focus of our earlier work was on coordination
between agencies from different departments, the findings would also be
applicable to agencies within a department that are engaged in similar
activities.
DOD's EEO Pilot Programs Report Low Case Activity
In its 9-month evaluation report, DOD stated that program activity for all
three programs had been lower than anticipated. At the end of the first
year, program officials reported continued low program activity. However,
in its report, DOD did not provide a baseline for its comparison or
elaborate on the reason for this occurrence. Instead, DOD's evaluation
plan states that data collected during the pilot program are to be
measured against fiscal year 2004 baseline data. Therefore, we are
including fiscal year 2004 data as reported to EEOC for each program for
comparison purposes. Since many cases are still going through the program
process, for comparison, we report only the number of initial contacts or
formal complaints.
According to DeCA officials, from January 1, 2005, through January 31,
2006, 42 employees contacted DeCA's EEO office; of those, 41 were offered
participation in ERO, and all opted for ERO. Of those who completed ERO,
16 did so with resolution; 9 did so without resolution, and 14 are still
in
process. Data are not available for DeCA test stores for fiscal year 2004,
the year before ERO was implemented.29
According to DLA officials, from January 1, 2005, through January 31,
2006, 15 employees contacted DLA's EEO office; of those, 13 were offered
participation in PECP, and 12 opted for it. Of those who completed PECP,
10 did so with resolution; 1 declined participation, and 1 withdrew the
precomplaint; 1 opted out of PECP. For fiscal year 2004, the year before
DLA implemented PECP, 26 employees contacted DLA's headquarters EEO
office.
According to USAF officials, from January 1, 2005, through January 31,
2006, a total of 634 formal complaints were filed USAF-wide. The CORE
process was available to 534 of the complainants. Of those complainants
offered CORE, 104 opted to process their complaint using CORE. Of these
104, 63 have been closed with resolution, and 28 CORE cases are still in
progress. Thirteen complainants opted out of CORE and chose to return to
the current EEO process. For fiscal year 2004, the year before USAF
implemented CORE, 667 formal complaints were filed USAF-wide; of these,
488 were filed at what are now CORE test sites.
DOD's 9-month report stated that case activity was lower than expected. As
a result of the low case activity, program officials have said they will
seek to extend their respective programs for an additional (third) year.
According to the authorizing memo from DOD implementing the pilot program,
in April 2006 program officials can request to extend the pilot program
for a third year. At the time of this report, DeCA and USAF had made
requests of DOD to extend the operation of their pilot programs for a
third year.
Although Containing Some Strengths, Limitations in Its Evaluation Plan
Will Hinder DOD's Ability to Assess Pilot Program Results
Our initial assessment of DOD's evaluation plan for the pilot program
found both strengths and limitations. One strength of the plan was the
inclusion of forms for collecting baseline data (before the programs
began) and pilot program data, which provides a tool for the pilot
programs to measure some aspects of their progress. Although DOD developed
an evaluation plan for the overall pilot program, the plan lacked some key
features of a sound evaluation plan, including measures that are directly
linked to the program objectives, criteria for determining pilot program
performance, and an appropriate data analysis plan for the evaluation
design. Without such features, DOD will be limited in its ability to
conduct an accurate and reliable assessment of the programs' results. In
addition, the lack of established key evaluation features in DOD's plan
increases the likelihood of insufficient or unreliable data, further
limiting confidence in pilot program results. Without confidence in pilot
program results, DOD will be limited in its decision making regarding this
pilot program, and Congress will be limited in its decision making about
the pilot program's potential broader application.
Officials from DOD's pilot program oversight entities have acknowledged
shortcomings and have indicated a willingness to modify the plan.
Strengths of DOD's Evaluation Plan
Considering the evaluation plan itself and interviews with DOD officials,
we found that DOD's plans for assessing the pilot programs had some
strengths, including:
o Forms in the evaluation plan for collecting baseline data (before the
pilot programs began) and pilot program data. According to the evaluation
plan, baseline data from fiscal year 2004 are recorded on a template
(i.e., a modified version of EEOC Form 462) appropriate to the part of the
complaint process the pilot program focuses on. Data collected during the
programs will be measured against the baseline data collected in the prior
year's EEOC Form 462.30 Pilot program and nonpilot program data are to be
collected by an Individual Data Report form, which is to collect
processing-time data, comparative information on early ADR, and early
management involvement in cases at each pilot program site. Comparing data
from the modified EEOC Form 462 to data from the program as well as to
nonprogram cases is expected to help DOD determine whether processing
times and redundancy were reduced concerning early resolution and
streamlining as a result of the pilot program.
o Detailed time frames, roles and responsibilities, and report planning in
the evaluation plan. The evaluation plan includes a schedule that details
tasks, roles and responsibilities, and milestones for completing set tasks
in evaluating the pilot program. This schedule provides a framework that
is organized and easy to follow.
o Inclusion of reasonable research design. The evaluation plan includes a
reasonable method for assessing pilot program results. Because the pilot
program legislation mandates voluntary participation in the program, DOD
was restricted from one form of design (i.e., randomly assigning employees
alleging or filing complaints of discrimination to participate in the
pilot program). As a result, DOD chose to compare prepilot and postpilot
program data as well as pilot and nonpilot program cases.
In addition, DOD officials said that the plan can be adjusted to the
extent feasible to ensure that the data collected are sufficient for
evaluating the pilot program.
Without Key Evaluation Plan Features, DOD Will Be Limited in Its Ability
to Assess Pilot Programs' Results
DOD's plan for evaluating the effectiveness of the pilot program lacks
some key features that are essential to assessing performance.
Well-developed evaluation plans, which include key evaluation features,
have a number of benefits, perhaps most importantly, increasing the
likelihood that evaluations will yield methodologically sound results,
thereby supporting effective program and policy decisions. The lack of
established key evaluation features in DOD's plan increases the likelihood
of insufficient or unreliable data, limiting confidence in pilot program
results. Without confidence in pilot program results, DOD will be limited
in its decision making regarding this pilot program, and Congress will be
limited in its decision making about the pilot program's potential broader
application.
Some key features of a sound evaluation plan include:31
o well-defined, clear, and measurable objectives;
o measures that are directly linked to the program objectives;
o criteria for determining pilot program performance;
o a way to isolate the effects of the pilot programs;
o a data analysis plan for the evaluation design; and
o a detailed plan to ensure that data collection, entry, and storage are
reliable and error-free.
DOD's evaluation plan contains the following limitations:
o The objectives in DOD's evaluation plan are not well defined or clear,
which makes measurement problematic. For example, the evaluation plan
identifies management accountability as an objective without defining it,
who it applies to, and how it will be measured. Without well-defined,
clear, and measurable objectives, the appropriate data may not be
collected, thus hindering the assessment of pilot program progress.
o DOD's data collection efforts are not linked to objectives in the
evaluation plan. For example, the evaluation plan contains a variety of
surveys that the individual pilot programs can use to measure customer
satisfaction, but customer satisfaction is not included in the evaluation
plan as an objective of the plan. Directly linking objectives and measures
is a key feature of an evaluation plan. Without such linkage, data
collection efforts may not directly inform stated objectives, and in turn,
may not inform the evaluation effort.
o DOD's evaluation plan does not establish standards for evaluating pilot
program performance. For example, DOD's plan does not state the amount or
type of change required to indicate that a pilot program has succeeded in
reducing processing time. Without targets or standards for determining
success, it will be difficult to determine if the pilot program was
effective.
o DOD's evaluation plan does not mention controlling for possible outcomes
that are attributable to factors other than the effects of the programs. A
preferred research method is to use random assignment of program
participants to provide greater confidence that results are attributable
to a program. As we mentioned, DOD was restricted from randomly assigning
employees alleging or filing complaints of discrimination to participate
in the pilot program. As a result, other factors, such as the type of
complaint, complainant, or the mediator may affect pilot program outcomes.
Establishing controls for such factors could help isolate the effects
attributable to the pilot programs. When an evaluation design involves,
for example, a comparison between prepilot and postpilot program
conditions, the research design should include controls to ensure that
results will be attributable to the pilot program and not to other
factors.
o DOD's evaluation plan does not explain how the data will be analyzed.
Although the evaluation plan has templates for collecting data, including
pilot program baseline data, individual data reports, and various surveys,
it does not state how the data collected will be analyzed. A data analysis
plan is a key feature of an evaluation plan as it sets out how data will
be analyzed to determine if program objectives have been met. Without a
data analysis plan, it is not clear how the data will be analyzed to
inform the objectives of the evaluation and assess the performance of the
programs.
o DOD's plan does not explain how the integrity of the data collected will
be ensured. A detailed plan to ensure that data collection, entry, and
storage are reliable and error-free is a key feature of an evaluation plan
that gives greater confidence to data quality and reliability and to any
findings made from these data. Without a detailed plan to ensure that data
collection, entry, and storage are reliable and error-free, confidence in
pilot program results will be limited.
Conclusions
All three programs share a common feature of emphasizing the use of ADR to
meet the legislative mandate to improve the efficiency of the EEO
complaint process. In addition, although authorized to operate outside of
current EEOC regulations, to a large extent, two of the three programs
have been designed by DOD to operate within the requirements of current
regulations. While sharing common strategies in such areas as electronic
data collection, the pilot programs implemented them differently. As the
challenges of the 21st century grow, it will become increasingly important
for DOD to consider how it can maximize performance and results through
the improved collaboration of its organizations. Officials from the
programs and DOD's EEO pilot program oversight entities have indicated
their willingness to share information and strategies.
To better ensure that it will provide useful results, DOD needs to make
changes to its evaluation plan. Although DOD's evaluation plan had some
strengths, the plan's shortcomings may impede DOD's ability to produce
sound results that can inform both program and policy decisions regarding
the overall pilot program. The lack of key evaluation features, such as
clear and measurable objectives, measures linked to these objectives, and
established criteria for determining pilot program performance may limit
confidence in pilot program results.
Recommendations for Executive Action
To improve the performance and results of the pilot program, we recommend
that the Secretary of Defense direct the Deputy Undersecretary of Defense
for Civilian Personnel Policy, the Deputy Undersecretary for Equal
Opportunity, and the Civilian Personnel Management Service to take the
following actions:
Establish regular intra-agency exchanges of information on outreach
strategies, training, and electronic data collection from which the pilot
programs could achieve potential benefits that would not be available if
working separately.
Develop a sound evaluation plan to accurately and reliably assess the
pilot programs' results, including such key features as
o well-defined, clear, and measurable objectives;
o measures that are directly linked to the program objectives;
o criteria for determining pilot program performance;
o a way to isolate the effects of the pilot programs;
o a data analysis plan for the evaluation design; and
o a detailed plan to ensure that data collection, entry, and storage are
reliable and error-free.
Agency Comments
We provided a draft of this report to the Secretary of Defense for his
review and comment. The Principal Deputy Undersecretary of Defense
provided written comments, which are included in appendix II.
DOD generally agreed with our recommendations. Regarding the establishment
of regular intra-agency exchanges of information among the pilot programs
to leverage potential benefits, DOD stated that it will hold quarterly
meetings with pilot program managers. Concerning the development of an
evaluation plan that accurately and reliably assesses the pilot programs'
results, DOD partially concurred with the recommendation and stated that
it would consider and incorporate the recommended key features into the
evaluation plan as appropriate. However, DOD also stated that the purpose
of the plan was to assist pilot program evaluators in their work by
specifying those procedures, tools, and objectives that would be unique to
the pilot programs. In its comments, DOD reasons that because all pilot
program officials agreed on a particular objective, which was common to
both the pilot and traditional EEO complaint procedures, that it was not
necessary to link data collection efforts to that objective or incorporate
either the objective or the data collection effort in the evaluation plan.
Because the plan is the long-term guide for the pilot program evaluation
process and because staff changes occur, it is important that DOD include
all objectives and methods they intend to use in the plan, allowing the
evaluation process to be more transparent and provide clearer guidance to
the pilot program officials on evaluation procedures.
In its response, DOD also commented on our observation that to a large
extent two of the three pilot programs were designed and are operating
within existing EEOC requirements. DOD noted that this was due in large
part to a presidential memorandum issued when the legislation was signed.
The memorandum, which addressed the implementation of the pilot program,
required that a complaining party be allowed to opt out of the pilot
program at any time. According to DOD, adhering to this requirement
necessitated using a similar design to the current EEO process so that
complaining parties who decided to opt out would not be penalized by
having to start at the very beginning of the current EEO complaint
process. It is not clear to us that ensuring the ability to opt out at any
point necessitates returning the complaining party to the very beginning
of the current EEO process in all cases. Rather, the complaining party
would be returned to the current EEO process at an appropriate point based
on what was achieved through the pilot program process. Overall, we see
nothing in the presidential memorandum that would limit DOD's legitimate
use of the procedural flexibility granted by Congress through the pilot
program authority.
We will send copies of this report to other interested congressional
parties, the Secretary of Defense, and the Chair of EEOC. We also will
make copies available to others upon request. In addition, the report is
available on GAO's home page at http://www.gao.gov .
If your staff have questions about this report, please contact me on (202)
512-9490. Key contributors to this report are listed in appendix III.
Sincerely yours,
George H. Stalcup Director, Strategic Issues
List of Congressional Addressees
The Honorable John Warner Chairman The Honorable Carl Levin Ranking Member
Committee on Armed Services United States Senate
The Honorable Duncan Hunter Chairman The Honorable Ike Skelton Ranking
Member Committee on Armed Services House of Representatives
The Honorable George V. Voinovich Chairman The Honorable Daniel K. Akaka
Ranking Member Subcommittee on Oversight of Government Management, the
Federal Workforce, and the District of Columbia Committee on Homeland
Security and Governmental Affairs United States Senate
The Honorable Jon C. Porter Chairman The Honorable Danny K. Davis Ranking
Member Subcommittee on the Federal Workforce and Agency Organization
Committee on Government Reform House of Representatives
EEO Laws and Regulations Applicable to Federal EmployeesAppendix I
Title VII of the Civil Rights Act of 1964, as amended, makes it illegal
for employers, including federal agencies, to discriminate against their
employees or job applicants on the basis of race, color, religion, sex, or
national origin.1 The Equal Pay Act of 1963 protects men and women who
perform substantially equal work in the same establishment from sex-based
wage discrimination.2 The Age Discrimination in Employment Act of 1967, as
amended, prohibits employment discrimination against individuals who are
40 years of age or older.3 Sections 501 and 505 of the Rehabilitation Act
of 1973, as amended, prohibit discrimination against qualified individuals
with disabilities who work or apply to work in the federal government.4
Federal agencies are required to provide reasonable accommodation to
qualified employees or applicants for employment with disabilities, except
when such accommodation would cause an undue hardship. In addition, a
person who files a complaint or participates in an investigation of an
equal employment opportunity (EEO) complaint or who opposes an employment
practice made illegal under any of the antidiscrimination statutes is
protected from retaliation. The Equal Employment Opportunity Commission
(EEOC) is responsible for enforcing all of these laws.
Federal employees or applicants for employment who believe that they have
been discriminated against by a federal agency may file a complaint with
that agency.5 The EEOC has established regulations providing for the
processing of federal sector employment discrimination complaints.6 This
complaint process consists of two stages, informal, or precomplaint
counseling, and formal. Before filing a complaint, the employee must
consult an EEO counselor at the agency in order to try to informally
resolve the matter. The employee must contact an EEO counselor within 45
days of the matter alleged to be discriminatory or, in the case of a
personnel action, within 45 days of the effective date of the action.
Counselors are to advise individuals that, when the agency agrees to offer
alternative dispute resolution (ADR) in the particular case,7 they may
choose to participate in either counseling or in ADR.
Counseling is to be completed within 30 days from the date the employee
contacted the EEO office for counseling unless the employee and agency
agree to an extension of up to an additional 60 days. If ADR is chosen,
the parties have 90 days in which to attempt resolution. If the matter is
not resolved within these time frames, the counselor is required to inform
the employee in writing of his or her right to file a formal
discrimination complaint with the agency. The written notice must inform
the employee of the (1) right to file a discrimination complaint within 15
days of receipt of the notice, (2) appropriate agency official with whom
to file a complaint, and (3) duty to ensure that the agency is informed
immediately if the complainant retains counsel or a representative.
After a complainant files a formal discrimination complaint, the agency
must decide whether to accept or dismiss the complaint and notify the
complainant. If the agency dismisses the complaint, the complainant has 30
days to appeal the dismissal to EEOC.8 If the agency accepts the
complaint, it has 180 days to investigate the accepted complaint and
provide the complainant with a copy of the investigative file.9 Within 30
days of receipt of the copy of the investigative file, the complainant
must choose between requesting (1) a hearing and decision from an EEOC
administrative judge (AJ)10 or (2) a final decision from the agency. When
a hearing is not requested, the agency must issue a final decision within
60 days. A complainant may appeal an agency's final decision to EEOC
within 30 days of receiving the final decision.
In cases where a hearing is requested, the AJ has 180 days to issue a
decision and send the decision to the complainant and the agency.11 If the
AJ issues a finding of discrimination, he or she is to order appropriate
relief. After the AJ decision is issued, the agency has 40 days to issue a
final order notifying the complainant whether or not the agency will fully
implement the decision of the AJ, and the employee has 30 days to file an
appeal with EEOC of the agency's final order.12 If the agency issues an
order notifying the complainant that the agency will not fully implement
the decision of the AJ, the agency also must file an appeal with EEOC at
the same time. Parties have 30 days in which to request reconsideration of
an EEOC decision. Figure I illustrates the EEO complaint process.
Figure 1: The EEO Administrative Complaint Process with Related Time
Frames
aWhere the agency agrees to offer ADR in the particular case, employees
may choose between participation in ADR and counseling activities. ADR
generally refers to any procedure agreed to by the parties in a dispute
that is used to resolve issues in controversy including, but not limited
to, mediation.
bWhere ADR is chosen, the parties have up to 90 days in which to attempt
resolution.
If a complaint is one that can be appealed to the Merit Systems Protection
Board (MSPB) such as a removal, reduction in grade or pay, or suspension
for more than 14 days,13 the complaint is a "mixed-case complaint." EEOC
regulations provide that an individual may raise claims of discrimination
in a mixed case, either as a mixed-case EEO complaint with the agency or a
direct appeal to MSPB, but not both.14
A complainant may file a civil action in federal district court at various
points during and after the administrative process.15 The filing of a
civil action will terminate the administrative processing of the
complaint. A complainant may file a civil action within 90 days of
receiving the agency's final decision or order, or EEOC's final decision.
A complainant may also file a civil action after 180 days from filing a
complaint with his or her agency, or filing an appeal with EEOC, if no
final action or decision has been made.
Comments from the Department of Defense Appendix II
GAO Contact and Staff AcknowledgmentsAppendix III
George H. Stalcup, (202) 512-9490 or [email protected]
In addition to the individual named above, Belva M. Martin, Assistant
Director; Karin K. Fangman; Cindy Gilbert; Emily Hampton-Manley; Anthony
Patterson; Rebecca Shea; Linda Sidwell (detailee); and Kiki Theodoropoulos
made key contributions to this report.
(450467)
www.gao.gov/cgi-bin/getrpt? GAO-06-538 .
To view the full product, including the scope
and methodology, click on the link above.
For more information, contact George H. Stalcup at (202) 512-9490 or
[email protected].
Highlights of GAO-06-538 , a report to congressional addressees
May 2006
EQUAL EMPLOYMENT OPPORTUNITY
DOD'S EEO Pilot Program Under Way, but Improvements Needed to DOD'S
Evaluation Plan
Delays in processing of equal employment opportunity (EEO) complaints have
been a long-standing concern. In 2000, as part of the Department of
Defense's (DOD) fiscal year 2001 authorization act, Congress authorized
DOD to carry out a 3-year pilot program for improving processes to resolve
complaints by civilian DOD employees by testing procedures that would
reduce EEO complaint processing times and eliminate redundancy, among
other things. The act requires two reports from GAO-90 days after the
first and last fiscal years of the pilot program's operation. In December
2005 and January 2006, we provided briefings on our initial review of the
pilot program. This report (1) describes key features and status of the
three programs and (2) assesses DOD's plan for evaluating the
effectiveness of the pilot program.
What GAO Recommends
GAO recommends that DOD
develop a sound evaluation plan that includes key evaluation features to
accurately and reliably assess the pilot programs' results.
DOD generally concurred with the recommendation and stated that it would
incorporate our recommended key features into the evaluation plan as
appropriate.
In August 2004, the Secretary of Defense authorized 2-year programs in (1)
Defense Logistics Agency (DLA), (2) the Defense Commissary Agency (DeCA),
and (3) components of the U.S. Air Force (USAF) which became operational
in fiscal year 2005. While the legislation stated that the pilot program
is exempt from procedural requirements of current Equal Employment
Opportunity Commission (EEOC) regulations, to a large extent two of the
three programs were designed and are operating within existing EEOC
requirements, with a specific emphasis on alternative dispute resolution
(ADR) as encouraged in DOD's memo soliciting pilot program proposals.
ADR techniques include, but are not limited to, conciliation,
facilitation, mediation, or arbitration and usually involve the
intervention or facilitation by a neutral third party. After the first
year, program officials reported low case activity and stated that they
plan to request approval from the Secretary to continue their respective
programs for a third year. To carry out the programs, officials used
similar strategies-outreach to inform eligible staff about the pilot
programs, staff training, and the use of electronic data collection-but
implemented them differently.
Our assessment of DOD's evaluation plan for the pilot program found both
strengths and limitations (see figure below). A sound evaluation plan
contains such features as criteria for determining program performance and
measures that are directly linked to program objectives. Such key features
increase the likelihood that the evaluation will yield sound results,
thereby supporting effective program and policy decisions. Lacking these
key features, DOD is limited in its ability to conduct an accurate and
reliable assessment of the program's results, and Congress is limited in
its ability to determine whether features of the overall program have
governmentwide applicability. Officials from DOD's pilot program oversight
entities have acknowledged shortcomings and have indicated a willingness
to modify the plan.
Strengths and Limitations of DOD's Evaluation Plan
*** End of document. ***