BNUMBER: B-274943.3 DATE: March 5, 1997 TITLE: Matter of:Resource Applications, Inc. ********************************************************************** DOCUMENT FOR PUBLIC RELEASE A protected decision was issued on the date below and was subject to a GAO Protective Order. This version has been redacted or approved by the parties involved for public release. Matter of:Resource Applications, Inc. File: B-274943.3 Date:March 5, 1997 Jacob B. Pompan, Esq., Gerald H. Werfel, Esq., and Neil H. Ruttenberg, Esq., Pompan, Ruffner & Werfel, for the protester. Avital G. Zemel, Esq., Environmental Protection Agency, for the agency. Robert A. Weissman, Esq., for TechLaw, Inc., an intervenor. Adam Vodraska, Esq., and James A. Spangenberg, Esq., Office of the General Counsel, GAO, participated in the preparation of the decision. DIGEST Allegation that protester's and awardee's consensus evaluation scores did not reflect the individual evaluators' scores is denied where the record indicates that the consensus reasonably reflects the collective view of the evaluators and the characteristics of the proposals, and there is no credible evidence that the consensus evaluation was unreasonable. DECISION Resource Applications, Inc. (RAI) protests the award of a Regional Oversight Contract (ROC) to TechLaw, Inc. by the Environmental Protection Agency (EPA) under request for proposals (RFP) No. W500823G3, for technical services. We deny the protest. The RFP, a total small business set-aside, contemplated the award of a cost-plus- fixed-fee contract for technical services to support EPA's Federal Facilities Revitalization and Reutilization Office in its mission of oversight and enforcement of the Comprehensive Environmental Response, Compensation, and Liability Act (CERCLA), and the Resource Conservation and Recovery Act (RCRA) at federal facilities in EPA Zone 3.[1] The RFP contained a detailed statement of work and set forth EPA's best estimate of the level-of-effort hours that would be needed. Work to be performed under the ROC will be ordered through written work assignments issued by the contracting officer. The ROC is for a base period of 2 years with 3 option years as well as options to increase the level of effort for each period of performance. The RFP stated that technical quality was more important than cost, but as proposals became more technically equal, evaluated cost would become more important. The RFP established three technical evaluation factors worth a maximum score of 570 points: three sample regional response scenarios (one for each region within EPA Zone 3, including Region V) worth 75 points a piece, for a total of 225 points; a labor mix matrix worth a maximum of 105 points; and past performance worth a maximum of 240 points. The past performance factor was broken down into 14 subfactors grouped in three categories: RCRA compliance, CERCLA assessment, and remedial activities support. The RCRA compliance category included a technical review subfactor and the remedial activities support category included a removal activities and a remedial action activities subfactor. EPA charged a technical evaluation panel (TEP) with evaluating each offeror's proposal on a scale of 0 (deficient) to 5 (superior in most features) for each factor and subfactor. These scores were then to be weighted according to the RFP's evaluation scheme. In addition, the score for each past performance subfactor was to be multiplied by a level of confidence assessment rating (LOCAR) to determine the final score for the past performance factor. The LOCARs reflected the government's degree of confidence that the offeror would keep the promises made in its proposal and were to be derived based on past performance information obtained by the contracting officer from references listed in past performance project summary sheets submitted by each offeror. The RFP required offerors to submit written technical and cost proposals, and also to present oral technical proposals, which would be videotaped by EPA for evaluation by the TEP.[2] During the oral presentations each offeror was to address the three sample regional response scenarios specified in the RFP; the offerors' proposed labor mix; and project summaries for past performance. Offerors were required to submit, for each project to be referred to in the oral presentation, a written past performance project summary sheet identifying the client, a description of the project, and a point of contact. TechLaw and RAI submitted proposals in response to the RFP. Besides their written technical and cost proposals, both offerors submitted past performance project summary sheets and later gave oral presentations to EPA. Soon after the videotaping, EPA misplaced the videotape of TechLaw's oral presentation, and invited TechLaw to repeat its oral technical presentation. TechLaw did so, and its repeat presentation was videotaped by EPA for evaluation by the TEP.[3] Each member of the TEP then individually reviewed the written proposals and the videotapes of the offerors' oral presentations, and evaluated and scored the proposals in accordance with the RFP's evaluation scheme. In preparation for consensus discussions among the TEP members, each member sent a computer file to the TEP chairperson, listing, for each evaluation factor and subfactor, that evaluator's score for each proposal and what that evaluator considered the proposal's strengths and weaknesses. The TEP chairperson then compiled these files in order to summarize the individually assessed scores, strengths, weaknesses for consensus discussions. The members of the TEP later met for consensus discussions on the initial proposals and agreed to consensus comments regarding the strengths and weaknesses of each proposal for each factor and subfactor, and assigned numerical consensus scores. To obtain the information to develop the LOCARs to apply to the TEP's past performance scores, the contracting officer and contract specialist contacted, by telephone, the references provided by the offerors in their past performance project summary sheets. At the conclusion of TEP's evaluation of technical proposals and the contracting officer's application of the LOCARs, TechLaw had the highest overall technical score with 417.9 points, including 201.9 points for past performance. RAI's overall technical score was 310.2 points, including 139.2 points for past performance. The contracting officer included both proposals in the competitive range. EPA conducted discussions with both offerors, including questions about aspects of their past performance and requested and obtained best and final offers (BAFO). Both offerors also made BAFO oral presentations, which were videotaped by EPA. The record shows that TechLaw responded to EPA's questions about certain areas of its past performance in its BAFO oral presentation, but did not include additional past performance project summary sheets with its BAFO. Following the submission of TechLaw's and RAI's BAFOs, the TEP reconvened, and each TEP member again individually evaluated the offerors' BAFO written technical proposals and videotaped BAFO oral presentations. The TEP then held consensus discussions as a group, and arrived at a consensus evaluation and score for each factor and subfactor addressed in the offerors' BAFOs. As a result of improvements to the proposals made by the offerors by their respective BAFOs, EPA increased TechLaw's overall technical score to 469.2 points, including 211.2 points for past performance, and raised RAI's overall technical score to 394.5 points, including 181.5 points for past performance. Overall, the TEP considered TechLaw's proposal to be technically superior to RAI's, as indicated by the point spread. In performing the required cost realism analysis, the contracting officer adjusted upward TechLaw's proposed cost-plus-fixed-fee of $13,415,879 to $13,789,572 and RAI's proposed cost-plus-fixed-fee of $11,024,804 to $12,763,298, in order to more accurately reflect the quality of the labor the offerors had proposed. The contracting officer determined that TechLaw's superior technical proposal outweighed RAI's lower evaluated cost, and awarded the ROC to TechLaw as the offeror whose proposal was most advantageous to the government. RAI first protests that the TEP's consensus scores for certain subfactors were arbitrary and not rationally related to the merits of TechLaw's and RAI's proposals, because the consensus scores do not reflect or account for the individual evaluators' scores for those subfactors. For example, with regard to the TEP's evaluation of the Region V scenario subfactor under the regional response scenarios factor, RAI contends that TechLaw's consensus score of 5 (superior in most features) is inconsistent with the individual evaluator scores, which ranged from a low of 3 (adequate) to a high of 4 (good with some superior features). Similarly, RAI points out that TechLaw's proposal received a maximum consensus score of 5 for the removal activities subfactor of the remedial activities support category of the past performance factor, even though no individual evaluator gave TechLaw a score greater than 4 for that subfactor. RAI alleges that the reverse problem--a lower consensus score than the individual scores--existed in the TEP's initial consensus scoring of RAI's proposal. Specifically, for the technical review subfactor under the RCRA compliance category of the past performance factor, RAI received an initial consensus score of 0 (totally deficient), but no evaluator gave RAI an individual score less than 1 (contains significant deficiencies) and two evaluators gave RAI a score of 4 for the subfactor. Likewise, for the remedial action activities subfactor under the remedial activities support category of the past performance factor, RAI received an initial consensus score of 1, yet no evaluator gave RAI a score less than 2 (clarification required before final scoring), and one evaluator gave RAI a top score of 5. RAI further contends that the TEP's consensus evaluation does not offer a rationale or explanation for the difference between the consensus scores and the individual evaluator scores. In this regard, the protester points out that the strengths and weaknesses listed on the consensus evaluation report were lifted almost verbatim from the TEP chairman's compilation of the individual evaluators' score sheets for the initial evaluation. The protester questions why the same strengths and weaknesses listed by the evaluators produced higher scores for TechLaw and lower scores for RAI for the consensus evaluation than when these same strengths and weaknesses served as the basis for the individual scores. There is nothing inherently objectionable in an agency's decision to develop a consensus rating. Appalachian Council, Inc., B-256179, May 20, 1994, 94-1 CPD para. 319. The fact that the evaluators individually rated TechLaw's proposal for the Region V scenario and removal activities subfactors less favorably than they did on a consensus basis for those subfactors, and individually rated RAI's proposal for the technical review and remedial action activities subfactors more favorably than they did on a consensus basis for those subfactors does not, by itself, warrant questioning the final evaluation results. See Syscon Servs., Inc., 68 Comp. Gen. 698 (1989), 89-2 CPD para. 258; Dragon Servs., Inc., B-255354, Feb. 25, 1994, 94-1 CPD para. 151. Agency evaluators may discuss the relative strengths and weaknesses of proposals in order to reach a consensus rating, which often differs from the ratings given by individual evaluators, since such discussions generally operate to correct mistakes or misperceptions that may have occurred in the initial evaluation. Schweizer Aircraft Corp., B-248640.2; B-248640.3, Sept. 14, 1992, 92-2 CPD para. 200; The Cadmus Group, Inc., B-241372.3, Sept. 25, 1991, 91-2 CPD para. 271. Thus, a consensus score need not be the score of the majority the evaluators initially awarded--the score may properly be determined after discussions among the evaluators. GZA Remediation, Inc., B-272386, Oct. 3, 1996, 96-2 CPD para. 155 (note 3). In short, the overriding concern in the evaluation process is that the final score assigned accurately reflect the actual merits of the proposals, not that it be mechanically traceable back to the scores initially given by the individual evaluators. Id.; Dragon Servs., Inc., supra. Here, the record shows that the TEP consensus report reasonably reconciles the differences of opinion among the evaluators and accurately reflects the relative qualities of TechLaw's and RAI's proposals. Specifically, while the protester correctly asserts that the strengths and weaknesses listed on the consensus evaluation sheets for TechLaw's proposal for the Region V scenario and the removal activities subfactors were derived from the strengths and weaknesses listed on individual evaluation sheets as compiled by the TEP chairperson, the protester is incorrect in asserting that the TEP then merely assigned a higher numerical score for these same listed strengths and weaknesses. Our review indicates that the TEP's consensus evaluation sheets articulate, separately from the listed strengths and weaknesses derived from the individual score sheets, the TEP's rationale for its consensus scores by listing the strengths the TEP found most pertinent in TechLaw's proposal for these subfactors. For example, as stated on the TEP's consensus evaluation sheet for the Region V scenario, "[t]he technical panel found" that TechLaw's "presentation of the scenario was comprehensive, thorough and addressed all the important elements required and therefore . . . superior in most features." For the removal activities subfactor, the TEP's consensus evaluation sheet stated the reasons why "the panel" determined that TechLaw's proposal element was rated superior in most features for this subfactor. Since the protester does not explain why the TechLaw proposal for the Region V scenario and the removal activities subfactors was not "superior in most features," as documented by the TEP, but simply disagrees with the consensus rating reached by agency's evaluators, we have no basis upon which to find that the agency's evaluation was unreasonable. See Dragon Servs., Inc., supra. With regard to the evaluation of RAI's proposal for the technical review and remedial action activities subfactors, we think TEP also sufficiently articulated the reasons why it reached its initial consensus scores for these subfactors. The TEP determined that for the technical review subfactor, RAI "did not address any of the criteria listed under the technical review task for RCRA compliance." For the remedial action activities subfactor, the TEP stated that "[b]ecause only one of the five project summaries provided [by RAI] was relevant to remedial action activities, the panel determined that the past performance did not demonstrate the ability to perform this task and therefore, this element is deficient." In any event, the differences between the individual evaluators' scores and the initial consensus evaluation of RAI's technical proposal for the technical review and remedial action activities subfactors is not relevant here because RAI was afforded, and took advantage of, the opportunity to revise the weaknesses identified during the initial evaluation in its BAFO, and the evaluators, with general unanimity, found RAI's BAFO responses to be adequate for both subfactors. RAI does not challenge the BAFO consensus scores for these subfactors. RAI also protests that because TechLaw failed to provide past performance project summary sheets for all of the projects TechLaw discussed in its BAFO oral presentation, EPA improperly increased TechLaw's past performance score and failed to adjust TechLaw's LOCAR scores. However, the RFP only required the submission of past performance project summary sheets as part of the offerors' initial submission, which TechLaw provided. Moreover, the record shows that TechLaw had submitted past performance summary sheets with its initial proposal for the bulk of the projects it referred to during its BAFO oral presentation. There is no evidence that the TEP required such sheets for the few additional projects referred to in TechLaw's BAFO oral presentation to reasonably evaluate that offeror's past performance. We also note that TechLaw's LOCAR scores were not adjusted as a result of TechLaw's BAFO because TechLaw had already received good or excellent LOCAR scores for each subfactor, i.e., most of the references contacted had given TechLaw a rating of good with superior features for its performance, or a significant majority of the references contacted had given TechLaw a superior rating for its performance; the record evidences that the few additional projects referred to in TechLaw's BAFO oral presentation would not have affected this rating. Under the circumstances, we find unobjectionable EPA's evaluation of TechLaw's BAFO oral presentation. The protest is denied. Comptroller General of the United States 1. EPA Zone 3 encompasses EPA Region V (Illinois, Indiana, Michigan, Minnesota, Ohio, Wisconsin), Region VI (Arkansas, Louisiana, New Mexico, Oklahoma, Texas), and Region VII (Iowa, Kansas, Missouri, Nebraska). 2. EPA states that this procurement was EPA's first use of oral presentations in lieu of substantial written technical proposals as a means of streamlining the acquisition process. Offerors were given a time limit in which to make their presentations and the videotaping took place at an EPA facility. EPA reports that the only individuals present during the videotaping, besides the offeror's team, were the contracting officer, the contract specialist, and the cameraman. No member of the TEP was present and none of the EPA officials asked questions during the presentation. The contracting officer informed the TEP that the offeror's presentations were not to be evaluated on the basis of delivery style, but strictly on the basis of technical content. According to the contracting officer, the offeror's representatives essentially read from a prepared text in making their presentations. 3. RAI protests that TechLaw had an unfair competitive advantage because EPA provided TechLaw with this second opportunity to present--and thus improve--its initial oral presentation. However, according to EPA, the protester was aware of the basis for this allegation at least as early as October 29, 1996, when the protester mentioned the repeat videotaping of TechLaw's presentation to EPA. RAI has not rebutted EPA's assertion. Since RAI did not raise this protest ground until November 25, more than 10 calendar days after the basis of its protest was apparently known, it is untimely raised and will not be considered. Bid Protest Regulations, sec. 21.2(a)(2), 61 Fed. Reg. 39039, 39043 (1996) (to be codified at 4 C.F.R. sec. 21.2(a)(2)).