TITLE:  	Andrulis Corporation, B-281002.2, June 2, 1999
BNUMBER:  B-281002.2
DATE:  June 2, 1999
**********************************************************************
Andrulis Corporation, B-281002.2, June 2, 1999

Decision

Matter of: Andrulis Corporation

File: B-281002.2

Date: June 2, 1999

J. Patrick McMahon, Esq., McMahon, David & Brody, for the protester.

Peter F. Dineen, Westar Corporation, an intervenor.

Vera Meza, Esq., and Steven S. Phillips, Esq., Department of the Army, for
the agency.

Charles W. Morrow, Esq., and James A. Spangenberg, Esq., Office of the
General Counsel, GAO, participated in the preparation of the decision.

DIGEST

Protest that agency unreasonably evaluated the protester's proposal is
denied where the only evidence of an unreasonable evaluation is that the
protester's proposal received a lower technical score under a revised
evaluation methodology than how it was originally scored, and the record
otherwise supports the reasonableness of the agency's revised technical
evaluation and source selection decision.

DECISION

Andrulis Corporation protests the award of a contract to Westar Corporation
under request for proposals (RFP) No. DAAD07-98-R-0103, a total small
business set-aside, issued by the United States Army, Aberdeen Acquisition
Center, White Sands Missile Branch, New Mexico, for test planning and
reporting services. Andrulis contends that the Army improperly evaluated its
proposal.

We deny the protest.

The RFP, issued February 27, 1998, was to obtain a contractor to provide
non-personal test planning and report services for the West Desert Test
Center and the Joint Chemical and Biological Contact Point and Test Office,
at Dugway Proving

Ground, Utah. RFP sect. C.2. The RFP contemplated a fixed rate time and
materials contract for a 3-year base period with two 2-year option periods.
RFP

sect.sect. B.1, F.4. The services included planning, conducting, and reporting on
chemical and biological defense tests; reports on developmental tests for
the Commander in Chiefs; tests to assess the military value of chemical
defense systems and related operational concepts; and tests to evaluate
aerial dissemination systems, smoke munitions, and chemical\biological
detection\protection devices. RFP sect.sect. C-3, C-4; Contracting Officer's
Statement at 2. The RFP consolidates two existing service contracts.
Contracting Officer's Statement at 2.

The RFP contemplated a best-value award considering scored merit factors
(technical and management), non-scored merit factors (quality assurance
program and past performance), and probable costs. RFP sect.sect. M.1, M-3. The RFP
assigned equal weight to the technical and management scored merit factors,
which factors were significantly more important than the equally weighted
quality assurance program, past performance and cost factors. RFP sect. M.3.D.
Under the technical factor, the RFP identified "demonstrated approach to
meeting the requirements" worth 35 points and "utilization of technical
personnel and other resources" worth 15 points as subfactors. RFP sect. M.4.A.;
Contracting Officer's Statement at 3. The management factor specified
experience worth 30 points, organization worth 10 points, and administration
worth 10 points as subfactors. RFP sect. M.4.B; Contracting Officer's Statement
at 3. Under each subfactor, a list of discrete evaluation elements called
definers were listed. RFP sect. M.4.

A number of proposals were submitted, including Westar's and Andrulis'. On

August 31, the Army awarded a contract to Westar under this RFP. Westar's
evaluated price was $9,725,888 and its point score was 94 points. Andrulis'
slightly higher evaluated priced proposal received a point score of 87
points. Protest at 5; Contracting Officer's Statement at 1; Agency Report,
First Evaluation Documents,

at 1. On September 8, Andrulis protested the award to Westar, arguing among
other things that the proposals were misevaluated. In response, the Army
advised our Office that it would reevaluate proposals and make a new award
selection. Letter from Agency Counsel to GAO (Sept. 22, 1998). Since this
was the remedy that the protester requested, we dismissed the protest as
academic. Andrulis Corp., B-281002,

Sept. 23, 1998.

As a result of the protest, the Army concluded that the proposal evaluation
board (PEB) did not adhere to the evaluation methodology, as stated in the
RFP and the source selection plan, in evaluating and assigning the point
scores to the proposals. See Agency Report, Tab 31, Addendum to Source
Selection Decision, at 1. The agency reached the conclusion that it lacked a
proper basis to assign the point scores to the respective proposals because
there was no evidence that the PEB considered or evaluated the proposals
against the various definers listed under the evaluation subfactors. Agency
Response to Protest Comments, Apr. 27, 1999, at 2; see Agency Report, First
Evaluation Documents. Therefore, the Army revised the evaluation methodology
to measure the relative merits of each proposal specifically in relation to
the definers listed under each of the major subfactors. See Agency Report,
Tab 31, Addendum to Source Selection Decision, at 1. This was accomplished
by assigning discrete increments of points to each definer based on the
number of proposal advantages, disadvantages, and deficiencies. Contracting
Officer's Statement at 3-4.

Using the revised evaluation methodology, the agency reevaluated proposals.
Following discussions, Westar's final proposal at an estimated price of
$9,906,444.68 was awarded a merit-factor score of 84.0886 points, while
Andrulis' final proposal at an estimated price of $8,873,263.40 received a
merit-factor score of 67.2001 points. Agency Report, Tab 30, Amended Post
Negotiation Memorandum, at 3, 4, 6; Agency Report, Tab 31, Addendum to
Source Selection Decision, at 10. Under the non-scored factors, the Army
found both offerors' quality assurance plans to be acceptable, and that both
offerors had low performance risk based upon their past performance records.
Agency Report, Tab 30, Amended Post Negotiation Memorandum, at 3.

The source selection authority (SSA) again determined that Westar's
higher-ranked proposal represented the best value. In making a
cost/technical tradeoff decision, the SSA concluded and documented that the
Westar proposal had a clear technical advantage over Andrulis' proposal in
several technical areas, such as test planning and review of test plans,
witness/monitoring tests, data analysis, document editing, technical
studies, and treaty support. For example, the SSA noted that the Westar
proposal contained an innovative approach to integrating technical
requirements that allowed for continuity within test programs and more
effective use of work done in other task areas greatly decreasing the time
and cost required to complete tasks. On the other hand, the SSA found that
the Andrulis proposal offered the standard existing avenues for test
planning and reporting and that an "integrated technical approach, mutual
success strategy, and continuous communication with the customer are
glaringly absent from the Andrulis proposal." Further, the SSA found
Westar's higher price, which was lower than the government estimate, was
reasonable and that its technically superior proposal was worth the
additional cost due to the anticipated increased productivity associated
with its technical approach. See Agency Report, Tab 31, Addendum to Source
Selection Decision, at 11-12. Therefore, the Army awarded Westar the
contract on February 11, 1999. Contracting Officer's Statement at 12. This
protest followed.

Andrulis protests that the evaluation was unreasonable primarily because its
proposal received a significantly lower technical score under the revised
evaluation methodology than received initially, even though the same
evaluators scored the proposals, the same evaluation factors were
applicable, and the technical proposals were essentially the same. See
Protest at 8-9; Protester Comments at 4-5. Andrulis asserts that the new
evaluation is improper because the Army allegedly converted the prior
evaluation into one that consisted primarily of a mechanical assignment of
numerical scores based upon counting advantages, disadvantages, and
deficiencies, without allowing for qualitative distinctions weighing the
relative merits of the proposals' respective advantages and disadvantages.
Protester's Supplemental Comments, Apr. 19, 1999, at 2-4. Andrulis also
points out various alleged discrepancies in the individual evaluators'
worksheets related to the tabulation of the advantages, disadvantages,
deficiencies associated with some of the definers under the various
subfactors. Andrulis further notes that some of the individual evaluators'
scores are inconsistent with the consensus scores; that the consensus scores
did not properly compute the numerical scores attributed to the individual
evaluators; and the SSA relied upon the improperly recorded consensus scores
in making the source selection decision. Protester's Supplemental Comments,
Apr. 19, 1999, at 2-7.

The evaluation of proposals is primarily a matter within the agency's
discretion, since it is responsible for defining its needs and deciding on
the best method for accommodating them. Thus, we question the evaluation
only if the record demonstrates that it was unreasonable or inconsistent
with the RFP's evaluation criteria. International Consultants, Inc.;
International Trade Bridge, Inc., B-278165, B-278165.2, Jan. 5, 1998, 98-1
CPD para. 7 at 3.

We find the evaluation of Westar's and Andrulis' proposals reasonable and
consistent with the RFP evaluation factors. We first note that the award
selection by the SSA was not based simply on a comparison of the point
scores but on a detailed discussion of the relative advantages and
disadvantages of the proposals. The protester does not question the
reasonableness of the determination of the proposals' relative advantages
and disadvantages. Instead, Andrulis' protest focuses on individual
evaluators' worksheets and how they calculated the point scores.

From our review, there appears to be no material error in the calculation of
the revised point scores. For example, the record does not evidence that
individual evaluators mechanically compared advantages to disadvantages
under any given definer to arrive at their point scores, nor were they
required to do so by the instructions. [1] The Army explains that the
evaluators had the discretion to decide that a proposal's advantages
outweighed disadvantages without regard to the precise numbers of each that
may have been identified and nothing in the record indicates that this is
not the case.

Further, the record evidences that the consensus scores reflected the
overall ratings that the individual members as a group agreed should be
assigned to Andrulis' proposal, which did not precisely track the scores of
each individual member. [2] Supplemental Agency Report, Apr. 27, 1999, at
2-3. A consensus score need not be the same score as initially scored by the
individual evaluators; a score may reasonably be determined after
discussions among the evaluators. I.S. Grupe, Inc., B-278839, Mar. 20, 1998,
98-1 CPD para. 86 at 6. Our review of the consensus scores reveals no material
errors. [3]

In any case, the record shows that the different scores of the initial and
revised evaluations is simply a result of the different evaluation
methodology that the Army used to ensure that the evaluation was consistent
with the stated definers in the RFP. The initial evaluation did not consider
the same level of detail or the designated subfactor definers whereas the
revised method expressly accounted for the definers. It is generally
expected that there will be some difference in judgments any time proposals
are evaluated utilizing different methods because of the subjective nature
of such evaluations and there is no requirement that each of these
differences be rationalized. Because of this subjectivity, we recognize that
that while technical point ratings may be useful guides for informed
decision-making, these ratings should not be overemphasized. [4] It remains
the ultimate responsibility of the source selection official to determine
how much, if any, significance should be attached to technical

scores assigned by a technical evaluation panel. See Chemonics Int'l,
B-222793, Aug. 6, 1986, 86-2 CPD para. 161 at 5-6. As noted above, the protester
does not specifically challenge the detailed discussion of the proposals'
relative advantages and disadvantages or the determination that the
advantages of Westar's proposal were worth the requisite additional cost.
[5]

The protest is denied.

Comptroller General

of the United States

Notes

1. For each definer, the revised worksheet instructed that "[b]ased on the
criteria for scoring . . . after listing the advantages and disadvantages,
and deficiencies, circle the assigned points." Agency Report, Tab 5, Revised
Worksheet, at 3-26.

2. For instance, the protester asserts that the agency failed to properly
evaluate its proposal under the "Utilization of technical personnel and
other resources" subfactor because two individual evaluator's rated its
proposal above average under one of the definers and their narratives
contain favorable comments. Protesters Supplemental Comments, Apr. 19, 1999,
at 6. While the protester argues that this was a strength in its proposal
that the SSA considered to be a weakness based upon such an evaluation, the
record shows that other individual evaluators, whose narratives contain less
favorable comments, and the consensus evaluation disagreed with the
assessment of these two evaluators.

3. The examples of alleged discrepancies on the individual evaluator
worksheets involve in total very few points, so that even assuming these
allegations had merit, they would not have materially affected Westar's
significant technical advantage.

4. The protester complains that the reevaluation method was improper because
evaluators were limited to assigning points under each definer based upon a
scale restricted to precise percentages of points on a 100-point scale,
arguing that this method magnified slight technical differences. Protester
Comments at 6. Not only is such a scoring methodology within the proper
exercise of agency discretion, but it provides no basis to object to this
award selection, which was not based on the proposals' relative point
scores, but on Westar's proposal's unchallenged documented technical
superiority. Met-Pro Corp., B-250706.2, Mar. 24, 1993, 93-1 CPD para. 263 at
5-6.

5. Contrary to Andrulis' contentions, the agency credited Andrulis for its
experience as the incumbent contractor for these services but found Westar's
innovative approach was superior to Andrulis' proposal that was based on its
incumbent contract.