TITLE: Andrulis Corporation, B-281002.2, June 2, 1999 BNUMBER: B-281002.2 DATE: June 2, 1999 ********************************************************************** Andrulis Corporation, B-281002.2, June 2, 1999 Decision Matter of: Andrulis Corporation File: B-281002.2 Date: June 2, 1999 J. Patrick McMahon, Esq., McMahon, David & Brody, for the protester. Peter F. Dineen, Westar Corporation, an intervenor. Vera Meza, Esq., and Steven S. Phillips, Esq., Department of the Army, for the agency. Charles W. Morrow, Esq., and James A. Spangenberg, Esq., Office of the General Counsel, GAO, participated in the preparation of the decision. DIGEST Protest that agency unreasonably evaluated the protester's proposal is denied where the only evidence of an unreasonable evaluation is that the protester's proposal received a lower technical score under a revised evaluation methodology than how it was originally scored, and the record otherwise supports the reasonableness of the agency's revised technical evaluation and source selection decision. DECISION Andrulis Corporation protests the award of a contract to Westar Corporation under request for proposals (RFP) No. DAAD07-98-R-0103, a total small business set-aside, issued by the United States Army, Aberdeen Acquisition Center, White Sands Missile Branch, New Mexico, for test planning and reporting services. Andrulis contends that the Army improperly evaluated its proposal. We deny the protest. The RFP, issued February 27, 1998, was to obtain a contractor to provide non-personal test planning and report services for the West Desert Test Center and the Joint Chemical and Biological Contact Point and Test Office, at Dugway Proving Ground, Utah. RFP sect. C.2. The RFP contemplated a fixed rate time and materials contract for a 3-year base period with two 2-year option periods. RFP sect.sect. B.1, F.4. The services included planning, conducting, and reporting on chemical and biological defense tests; reports on developmental tests for the Commander in Chiefs; tests to assess the military value of chemical defense systems and related operational concepts; and tests to evaluate aerial dissemination systems, smoke munitions, and chemical\biological detection\protection devices. RFP sect.sect. C-3, C-4; Contracting Officer's Statement at 2. The RFP consolidates two existing service contracts. Contracting Officer's Statement at 2. The RFP contemplated a best-value award considering scored merit factors (technical and management), non-scored merit factors (quality assurance program and past performance), and probable costs. RFP sect.sect. M.1, M-3. The RFP assigned equal weight to the technical and management scored merit factors, which factors were significantly more important than the equally weighted quality assurance program, past performance and cost factors. RFP sect. M.3.D. Under the technical factor, the RFP identified "demonstrated approach to meeting the requirements" worth 35 points and "utilization of technical personnel and other resources" worth 15 points as subfactors. RFP sect. M.4.A.; Contracting Officer's Statement at 3. The management factor specified experience worth 30 points, organization worth 10 points, and administration worth 10 points as subfactors. RFP sect. M.4.B; Contracting Officer's Statement at 3. Under each subfactor, a list of discrete evaluation elements called definers were listed. RFP sect. M.4. A number of proposals were submitted, including Westar's and Andrulis'. On August 31, the Army awarded a contract to Westar under this RFP. Westar's evaluated price was $9,725,888 and its point score was 94 points. Andrulis' slightly higher evaluated priced proposal received a point score of 87 points. Protest at 5; Contracting Officer's Statement at 1; Agency Report, First Evaluation Documents, at 1. On September 8, Andrulis protested the award to Westar, arguing among other things that the proposals were misevaluated. In response, the Army advised our Office that it would reevaluate proposals and make a new award selection. Letter from Agency Counsel to GAO (Sept. 22, 1998). Since this was the remedy that the protester requested, we dismissed the protest as academic. Andrulis Corp., B-281002, Sept. 23, 1998. As a result of the protest, the Army concluded that the proposal evaluation board (PEB) did not adhere to the evaluation methodology, as stated in the RFP and the source selection plan, in evaluating and assigning the point scores to the proposals. See Agency Report, Tab 31, Addendum to Source Selection Decision, at 1. The agency reached the conclusion that it lacked a proper basis to assign the point scores to the respective proposals because there was no evidence that the PEB considered or evaluated the proposals against the various definers listed under the evaluation subfactors. Agency Response to Protest Comments, Apr. 27, 1999, at 2; see Agency Report, First Evaluation Documents. Therefore, the Army revised the evaluation methodology to measure the relative merits of each proposal specifically in relation to the definers listed under each of the major subfactors. See Agency Report, Tab 31, Addendum to Source Selection Decision, at 1. This was accomplished by assigning discrete increments of points to each definer based on the number of proposal advantages, disadvantages, and deficiencies. Contracting Officer's Statement at 3-4. Using the revised evaluation methodology, the agency reevaluated proposals. Following discussions, Westar's final proposal at an estimated price of $9,906,444.68 was awarded a merit-factor score of 84.0886 points, while Andrulis' final proposal at an estimated price of $8,873,263.40 received a merit-factor score of 67.2001 points. Agency Report, Tab 30, Amended Post Negotiation Memorandum, at 3, 4, 6; Agency Report, Tab 31, Addendum to Source Selection Decision, at 10. Under the non-scored factors, the Army found both offerors' quality assurance plans to be acceptable, and that both offerors had low performance risk based upon their past performance records. Agency Report, Tab 30, Amended Post Negotiation Memorandum, at 3. The source selection authority (SSA) again determined that Westar's higher-ranked proposal represented the best value. In making a cost/technical tradeoff decision, the SSA concluded and documented that the Westar proposal had a clear technical advantage over Andrulis' proposal in several technical areas, such as test planning and review of test plans, witness/monitoring tests, data analysis, document editing, technical studies, and treaty support. For example, the SSA noted that the Westar proposal contained an innovative approach to integrating technical requirements that allowed for continuity within test programs and more effective use of work done in other task areas greatly decreasing the time and cost required to complete tasks. On the other hand, the SSA found that the Andrulis proposal offered the standard existing avenues for test planning and reporting and that an "integrated technical approach, mutual success strategy, and continuous communication with the customer are glaringly absent from the Andrulis proposal." Further, the SSA found Westar's higher price, which was lower than the government estimate, was reasonable and that its technically superior proposal was worth the additional cost due to the anticipated increased productivity associated with its technical approach. See Agency Report, Tab 31, Addendum to Source Selection Decision, at 11-12. Therefore, the Army awarded Westar the contract on February 11, 1999. Contracting Officer's Statement at 12. This protest followed. Andrulis protests that the evaluation was unreasonable primarily because its proposal received a significantly lower technical score under the revised evaluation methodology than received initially, even though the same evaluators scored the proposals, the same evaluation factors were applicable, and the technical proposals were essentially the same. See Protest at 8-9; Protester Comments at 4-5. Andrulis asserts that the new evaluation is improper because the Army allegedly converted the prior evaluation into one that consisted primarily of a mechanical assignment of numerical scores based upon counting advantages, disadvantages, and deficiencies, without allowing for qualitative distinctions weighing the relative merits of the proposals' respective advantages and disadvantages. Protester's Supplemental Comments, Apr. 19, 1999, at 2-4. Andrulis also points out various alleged discrepancies in the individual evaluators' worksheets related to the tabulation of the advantages, disadvantages, deficiencies associated with some of the definers under the various subfactors. Andrulis further notes that some of the individual evaluators' scores are inconsistent with the consensus scores; that the consensus scores did not properly compute the numerical scores attributed to the individual evaluators; and the SSA relied upon the improperly recorded consensus scores in making the source selection decision. Protester's Supplemental Comments, Apr. 19, 1999, at 2-7. The evaluation of proposals is primarily a matter within the agency's discretion, since it is responsible for defining its needs and deciding on the best method for accommodating them. Thus, we question the evaluation only if the record demonstrates that it was unreasonable or inconsistent with the RFP's evaluation criteria. International Consultants, Inc.; International Trade Bridge, Inc., B-278165, B-278165.2, Jan. 5, 1998, 98-1 CPD para. 7 at 3. We find the evaluation of Westar's and Andrulis' proposals reasonable and consistent with the RFP evaluation factors. We first note that the award selection by the SSA was not based simply on a comparison of the point scores but on a detailed discussion of the relative advantages and disadvantages of the proposals. The protester does not question the reasonableness of the determination of the proposals' relative advantages and disadvantages. Instead, Andrulis' protest focuses on individual evaluators' worksheets and how they calculated the point scores. From our review, there appears to be no material error in the calculation of the revised point scores. For example, the record does not evidence that individual evaluators mechanically compared advantages to disadvantages under any given definer to arrive at their point scores, nor were they required to do so by the instructions. [1] The Army explains that the evaluators had the discretion to decide that a proposal's advantages outweighed disadvantages without regard to the precise numbers of each that may have been identified and nothing in the record indicates that this is not the case. Further, the record evidences that the consensus scores reflected the overall ratings that the individual members as a group agreed should be assigned to Andrulis' proposal, which did not precisely track the scores of each individual member. [2] Supplemental Agency Report, Apr. 27, 1999, at 2-3. A consensus score need not be the same score as initially scored by the individual evaluators; a score may reasonably be determined after discussions among the evaluators. I.S. Grupe, Inc., B-278839, Mar. 20, 1998, 98-1 CPD para. 86 at 6. Our review of the consensus scores reveals no material errors. [3] In any case, the record shows that the different scores of the initial and revised evaluations is simply a result of the different evaluation methodology that the Army used to ensure that the evaluation was consistent with the stated definers in the RFP. The initial evaluation did not consider the same level of detail or the designated subfactor definers whereas the revised method expressly accounted for the definers. It is generally expected that there will be some difference in judgments any time proposals are evaluated utilizing different methods because of the subjective nature of such evaluations and there is no requirement that each of these differences be rationalized. Because of this subjectivity, we recognize that that while technical point ratings may be useful guides for informed decision-making, these ratings should not be overemphasized. [4] It remains the ultimate responsibility of the source selection official to determine how much, if any, significance should be attached to technical scores assigned by a technical evaluation panel. See Chemonics Int'l, B-222793, Aug. 6, 1986, 86-2 CPD para. 161 at 5-6. As noted above, the protester does not specifically challenge the detailed discussion of the proposals' relative advantages and disadvantages or the determination that the advantages of Westar's proposal were worth the requisite additional cost. [5] The protest is denied. Comptroller General of the United States Notes 1. For each definer, the revised worksheet instructed that "[b]ased on the criteria for scoring . . . after listing the advantages and disadvantages, and deficiencies, circle the assigned points." Agency Report, Tab 5, Revised Worksheet, at 3-26. 2. For instance, the protester asserts that the agency failed to properly evaluate its proposal under the "Utilization of technical personnel and other resources" subfactor because two individual evaluator's rated its proposal above average under one of the definers and their narratives contain favorable comments. Protesters Supplemental Comments, Apr. 19, 1999, at 6. While the protester argues that this was a strength in its proposal that the SSA considered to be a weakness based upon such an evaluation, the record shows that other individual evaluators, whose narratives contain less favorable comments, and the consensus evaluation disagreed with the assessment of these two evaluators. 3. The examples of alleged discrepancies on the individual evaluator worksheets involve in total very few points, so that even assuming these allegations had merit, they would not have materially affected Westar's significant technical advantage. 4. The protester complains that the reevaluation method was improper because evaluators were limited to assigning points under each definer based upon a scale restricted to precise percentages of points on a 100-point scale, arguing that this method magnified slight technical differences. Protester Comments at 6. Not only is such a scoring methodology within the proper exercise of agency discretion, but it provides no basis to object to this award selection, which was not based on the proposals' relative point scores, but on Westar's proposal's unchallenged documented technical superiority. Met-Pro Corp., B-250706.2, Mar. 24, 1993, 93-1 CPD para. 263 at 5-6. 5. Contrary to Andrulis' contentions, the agency credited Andrulis for its experience as the incumbent contractor for these services but found Westar's innovative approach was superior to Andrulis' proposal that was based on its incumbent contract.