TITLE: Apex Marine Ship Management Company, LLC; American V-Ships, B-278276.25; B-278276.28, September 25, 2000 BNUMBER: B-278276.25; B-278276.28 DATE: September 25, 2000 ********************************************************************** Apex Marine Ship Management Company, LLC; American V-Ships, B-278276.25; B-278276.28, September 25, 2000 Decision Matter of: Apex Marine Ship Management Company, LLC; American V-Ships Marine, Ltd. File: B-278276.25; B-278276.28 Date: September 25, 2000 Marc J. Fink, Esq., Anne E. Mickey, Esq., and Heather M. Spring, Esq., Sher & Blackwell, for Apex Marine Ship Management Co., LLC; and Eric J. Marcotte, Esq., Robert A. Mangrum, Esq., Scott A. Schipma, Esq., Paul S. Ebert, Esq., and Adam B. Walker, Esq., Winston & Strawn, for American V-Ships Marine, Ltd., the protesters. Kevin M. Kordziel, Esq., Laura K. Kennedy, Esq., and David A. Leib, Esq., Jenner & Block, for American Overseas Marine Corporation; Robert G. Fryling, Esq., and Edward J. Hoffman, Esq., Blank, Rome, Comisky & McCauley, for Keystone Shipping Services, Inc.; Bradley Gilman, Esq., and Carl Winner, Esq., Robertson, Monagle & Eastaugh, for Ocean Duchess, Inc.; Timothy F. Noelker, Esq., Michael G. Roberts, Esq., Linda L. Shapiro, Esq., and Steven E. Kellogg, Esq., Thompson Coburn, for Crowley Liner Services, Inc., intervenors. Janis P. Rodriguez, Esq., Michael Rose, Esq., and J. Patrick Wiese, Esq., Maritime Administration, Department of Transportation, for the agency. Paul E. Jordan, Esq., Glenn G. Wolcott, Esq., and Paul Lieberman, Esq., Office of the General Counsel, GAO, participated in the preparation of the decision. DIGEST 1. Agency's comprehensive technical evaluation report, which summarizes individual evaluators' comments and assessments and explains the evaluated strengths of each proposal on a factor-by-factor basis, provides an adequate basis for GAO to review the agency's evaluation and determine that it was not arbitrary. 2. Where proposals are considered acceptable and within the competitive range, the agency is not obligated to discuss every aspect of a proposal that receives less than the maximum score; here, agency satisfied its obligation to provide meaningful discussions. 3. Price/technical tradeoffs in best value determination, based on combined technical/risk scores, evaluators' recommendations, limits to numbers of awards, and economy of scale pricing are reasonably supported by the record as a whole, which includes comments of evaluators and reasoning of selection board, as verified by consideration of a price-per-technical-point calculation. DECISION Apex Marine Ship Management Company, LLC, and American V-Ships Marine Ltd., protest the award of contracts to other offerors under request for proposals (RFP) No. DTMA91-97-R-00002, issued by the Maritime Administration (MARAD), Department of Transportation, for the operation and maintenance of ships in the Ready Reserve Force (RRF). [1] The protesters assert that the agency improperly evaluated technical/management proposals, failed to conduct meaningful discussions, and failed to make a proper best value determination. We deny the protests. BACKGROUND This procurement has a long and tortured history. The solicitation was first issued in August 1997, and has now been amended 32 times; as amended, it seeks proposals for the operation and management of 74 RRF vessels. [2] The RFP contemplated award of multiple fixed-price contracts, with certain cost-reimbursable line items, at an estimated total cost in excess of $1 billion during a 5-year performance period. Each of the ships is to be maintained within a designated readiness status, as specified by the Department of Defense (DOD), which ranges from 4 to 30 days. Twenty-three offerors submitted initial proposals by the original October 1997 closing date; eighteen proposals were included in the competitive range. [3] In June 1998, the agency announced contract awards to 10 offerors. After various protests were filed, the agency determined that errors had been made in the source selection process. As a result of this determination, the agency rescinded the contract awards and reopened discussions with the competitive range offerors. [4] As part of its corrective action, the agency issued RFP amendment No. 0012, which reflected various revisions and clarifications regarding the proposal evaluation process. Among other things, section M of the RFP, as amended, advised offerors as follows: Technical/Management proposals will be evaluated quantitatively under Technical/Management Evaluation Factors for understanding and compliance with the requirements. Proposals will also be evaluated to assess the risks associated with the offeror's proposed approach and the degree to which the offeror's past performance on other relevant contracts provides the Government with confidence that the offeror will accomplish the requirements of the current solicitation for the proposed ship groups. . . . . . The evaluation factors and subfactors are listed in descending order of importance, in other words, the most important factor is listed first, and the least important factor is listed last. Factor A [past performance and technical experience] is twice as important as Factors B [purchasing, cost controls, and accounting] or C [management organization], which are of relatively equal importance.[ [5]] RFP sect.sect. M.3.1, M.3.2. The technical/management evaluation factors and subfactors correspond to RFP sect. L.10.4, which identifies various types of information offerors were required to provide, and represent a total of 37 evaluation areas under which proposals were evaluated. Offerors were required to submit electronic price proposals with fully burdened per diem prices for all ship groups on which they were proposing, and were permitted to propose economies of scale (EOS) expressed in terms of per diem discounts for each additional ship awarded in excess of three vessels. Offerors were advised that prices would be evaluated on the basis of the "notional schedule," [6] and would also be evaluated for reasonableness and realism. RFP sect. M.4.1. The RFP also advised offerors of certain limitations on how contracts would be awarded. For example, the RFP stated that no offeror would be awarded contracts for more than 12 ships, and that within the 12-ship limitation, the agency would make a determination regarding the maximum number of ships for which each offeror would be considered, based on each offeror's combined technical/management and risk scores. RFP sect.sect. M.6.1, M.6.2. Offerors were advised that non-price factors were more important than price, and that multiple awards would be made on the basis of the agency's determination regarding which proposals represented the best overall value to the government. Following various other legal challenges, [7] the agency reopened discussions with all offerors in the competitive range on November 23, 1999. Revised technical and price proposals were submitted in January 2000 and, thereafter, the agency conducted another round of discussions with offerors during February and March 2000. Final revised proposals were submitted on March 24, 2000. Throughout this procurement, the offerors' technical/management proposals, and revisions, were evaluated by a technical evaluation team (TET) comprised of six voting members, one of whom functioned as the chair. Each TET member individually evaluated each proposal, assigning separate technical and risk scores in each of the 37 evaluation areas. [8] Evaluators were required to provide narrative explanations regarding "extreme" scores--that is, technical/management scores of 0, 1, or 5 and risk scores of 0 or 5--and permitted, but not required, to provide supporting comments for scores that were other than "extreme." Revised [Source] Selection Plan (RSP) sect. 5.7.9. Accordingly, the individual evaluators' worksheets contained individual technical scores of 2, 3, or 4 and risk scores of 2 or 4 that were not specifically supported by narrative explanations other than the definitions contained in the source selection plan. The evaluators' scores were aggregated and averaged in each evaluation area and, following the final evaluation, the offerors' total scores reflecting the technical/management and risk assessments were as follows: Offeror Tech/Mgt. Risk Combined [9] Score Score Score KEY 447 94 353 IUM 438 114 324 MTL 418 133 285 MOR 391 126 265 CLS 416 156 260 AOM 415 165 250 Off. 7 [deleted] [deleted] 240 Apex 388 148 240 PCS 404 168 236 PGM 395 160 235 V-Ships 380 153 227 Off. 12 [deleted] [deleted] 171 Off. 13 [deleted] [deleted] 169 Off. 14 [deleted] [deleted] 133 ODI 348 223 125 Off. 16 [deleted] [deleted] 80 Off. 17 [deleted] [deleted] 51 Off. 18 [deleted] [deleted] 4 Contracting Officer's Statement at 7. The evaluators' scores and supporting comments were provided to the TET Chair, who incorporated them into a comprehensive TET report in which the evaluation of each offeror's proposal was comprehensively summarized. Specifically, the TET report, which was over 200 pages in length, contained separate sections discussing each offeror's proposal with regard to each evaluation factor and subfactor. Upon completion of the final TET report, each evaluator reviewed and approved the report. After completing the final technical/management evaluation, the agency developed the following graphical depiction of the offerors' scores and grouped the proposals to determine the number of ships for which each offeror would be considered. As shown below, Apex and V-Ships were placed in the group of offerors considered for possible award of 6 to 8 ships. [Image] Contracting Officer's Statement, Encl. (ii). The TET report and proposal groupings were then sent to the source evaluation board (SEB), which made source selection decisions by first considering the proposals with the highest technical/management ratings and determining whether the prices proposed for those highest ranked offerors represented the best value to the government. In making these best value determinations, the agency calculated a "price/technical value ratio" (PTVR) by dividing each offeror's proposed prices for each ship group by its combined technical score, essentially calculating a "price per technical point" for each proposal. Apex's and V-Ships' proposed prices were generally higher than the prices proposed by offerors with comparable technical/management scores, and neither protester's proposal offered the lowest price per technical point for any ship group on which it had proposed. [10] Based on the agency's subjective evaluations, and after considering the PTVR for each offeror, the agency made source selection decisions resulting in the following awards: Keystone and IUM, 11 ships each; MTL and MOR, 9 ships each; CLS, AOM, and PGM, 8 ships each; PCS, 6 ships; and ODI, 3 ships. These protests followed. DISCUSSION As a preliminary matter, we recognize the complexity of this procurement and the attendant complications encountered by the agency in conducting it. Beginning in 1997 with 23 initial proposals, the agency conducted multiple evaluations and discussions with 19 competitive range offerors before making the original 39 awards (then encompassing 89 ships) to 10 offerors, which necessitated making a substantial number of best value determinations. [11] In taking corrective action to resolve other issues, the agency attempted to simplify the proposal and evaluation process to make the procurement more manageable while still being fair to all offerors. Even as "simplified," the evaluation involved 18 offerors, a significant volume of electronic price proposals and 25 awards encompassing 33 ship groups (74 individual ships), necessitating dozens of alternative best value determinations. In view of the complexities involved in this undertaking, in reviewing these protests we have taken into account the agency's good faith efforts to conduct the procurement in a fair and efficient manner while ensuring that the government obtains the best overall value. Technical/Management Evaluation Apex and V-Ships first challenge the adequacy of the agency's evaluation of technical/management proposals, complaining that the evaluation record does not adequately explain the basis for rating their proposals lower than the highest rated proposals. The record does not support the protesters' contentions. In reviewing protests against allegedly improper evaluations, our Office will examine the record to determine whether the agency's determination was reasonable and consistent with the evaluation criteria listed in the solicitation, Hattal & Assocs., B-243357, B-243357.2, July 25, 1991, 91-2 CPD para.90 at 7, and an agency must document its judgments in sufficient detail to show that they are not arbitrary. U.S. Defense Sys., Inc., B-245563, Jan. 17, 1992, 92-1 CPD para. 89 at 3. The amount and detail of documentation necessary to demonstrate that an agency's judgments were reasoned and rational will vary from procurement to procurement, and there is no absolute requirement that evaluation records must include narrative explanations for every score assigned. See Delta Dental Plan of Cal. v. Perry, 1996 WL 83881 (N.D. Cal. 1996) at *15, citing Champion-Alliance, Inc., B-249504, Dec. 1, 1992, 92-2 CPD para. 386 at 6-7 (contemporaneous summary evaluation narrative and post-protest amplification sufficient documentation). Based on our review of the agency record, we see no basis to question the adequacy of the documentation supporting the agency's evaluation of technical/management proposals. As discussed above, the agency generated a comprehensive evaluation record supporting the agency's evaluation of each competitive range offeror's proposal. Specifically, the lengthy final TET Report contains separate detailed discussions regarding each offeror's proposal. Each evaluation summary contains an initial section discussing the general strengths and weaknesses of the proposal overall, followed by a narrative discussion of the offeror's proposal with regard to each factor and subfactor established in section M of the RFP. For example, the section in the TET report discussing Apex's proposal began by stating that, overall: Apex Ship Management is an above average proposal. Apex addressed all factors. All personnel qualified both in education and experience with [deleted] in major departments (engineering, operations and procurement). Apex has had experience on MARAD ships with good results. TET Report at APX 2. This summary was followed by nine pages of text in which the agency discusses Apex's proposal under each individual evaluation factor and subfactor. By way of example, in discussing Apex's proposal under evaluation factor B, purchasing, cost controls and accounting, the TET report refers to various aspects of Apex's proposal, noting "[deleted] in the contracting staff" and referencing Apex's [deleted] of procedural compliance and sole source procurements; describes Apex's accounting system; and discusses other specific aspects of Apex's proposal in some detail, as follows: [Deleted] TET Report at APX 7. Similarly, regarding the evaluation of Apex's proposal under evaluation factor C, management organization, the TET report discusses the port engineers Apex proposed; notes that Apex [deleted]; discusses the fact that "Apex is sometimes late in submittal of some reports, however, the reports are always of the highest quality"; discusses the organizational supervisory responsibilities and chain of command for various positions, addresses the qualifications of various personnel; and discusses Apex's "contingency plan," its quality assurance system, the manner in which accidents and hazardous situations are handled, and the involvement of Apex's [deleted] in day-to-day activities. [12] TET Report at APX 8-11. The TET Report similarly addresses the various evaluated aspects of V-Ships' proposal, stating at the outset that "V[-S]hips [M]arine has been a ship manager and general agent for MARAD for over 12 years thereby demonstrating ship operation and maintenance experience. Proposal is well written and addresses all factors. PPI supports claims of above average performance." TET Report at AMV 3. As to the factor-by-factor evaluation, under evaluation factor A, past performance and experience, the TET report acknowledges various aspects of V-Ships' past performance, referring to a particular incident involving vessel propeller damage, stating "nice operations solution, not much on maintenance," and describes V-Ships' proposal with regard to the corporate systems subfactor in part as "restatement of RFP." Under evaluation factor B, purchasing, cost controls and accounting, the TET report noted that V-Ships' proposal "[p]rovided policies on small business, good controls . . . good detail"; discussed V-Ships' purchasing procedures; referred to V-Ships' proposal of modern preventative maintenance programs and computerized status reports; and noted that V-Ships had "[taken] [deleted] steps[,] most of which were recommended by DCAA to improve [its] accounting system," that V-Ships "[h]as added [deleted]," and finally that "[d]uring proposal[,] V-Ships had trouble securing [deleted] due to complaint by facility of previous [deleted] problems." TET Report at AMV 4-5. Following the factor-by-factor discussion of each offeror's proposal, the TET Report contained the individual evaluators' narrative supporting comments for the "extreme" scores. Because Apex's proposal was rated higher than a majority of the competitive range proposals, and V-Ships' proposal was rated only slightly lower, the narrative support for both proposals' "extreme" scores include multiple descriptions of evaluated strengths. Nonetheless, the TET Report also documents multiple evaluated strengths for those proposals that were rated higher than Apex's and V-Ships'. [13] Overall, the agency concluded that the strengths of the proposals rated higher than Apex's and V-Ships' were more significant and/or offered greater value than the evaluated strengths in Apex's and V-Ships' proposals. In sum, based on our review of the evaluation record as a whole, we find that it adequately supports the differing technical evaluation of the various proposals. See KMS Fusion, Inc., B-242529, May 8, 1991, 91-1 CPD para. 447 at 7. Apex's and V-Ships' mere disagreements with the agency's multiple judgments do not render the evaluation unreasonable. BFI Waste Sys. of Neb., Inc., B-278223, Jan. 8, 1998, 98-1 CPD para. 8 at 2. In addition to the general complaint that the evaluation was flawed for lack of adequate documentation, Apex identifies a specific portion of its technical/ management proposal which it maintains the agency misevaluated. In an amendment to its proposal submitted on January 5, 1998, Apex stated: [W]e are also most pleased to note that [Apex] has recently been awarded the management of [deleted] . . . . The addition of these vessels to our commercial fleet represents a major amendment to our proposal. As required by Section L.10.4.1.2 of the solicitation, we have set forth below the pertinent details for these vessels: [deleted] The addition of these [deleted] vessels will necessitate the [deleted] . . . and will provide the Company with expanded "surge capability" with regard to the operation of the MARAD fleet. Agency Report, Tab D.2, at 8-9. In fact, Apex had not been awarded contracts for these vessels and, subsequently, the contracts were awarded to another company. In pursuing this protest, Apex's president acknowledges that its January 1998 submission was factually inaccurate, stating: Early in this solicitation . . . Apex was in line for award of management contracts for several [deleted] vessels. We thought this would be a benefit to MARAD as the agency had in the past complained about Apex's lack of commercial ships as limiting our surge. Ultimately, however, Apex was not awarded the [deleted] ships. They were instead awarded to [another company] in 1998. This was a major issue at MARAD and was common knowledge in the industry at the time. . . . . . [W]e did not consider it necessary to submit written notice of the change. Affidavit of Apex President, July 19, 2000, para.para. 2, 4. Not only did Apex "not consider it necessary to submit written notice of the change," Apex's final proposal revision, submitted in January 2000, specifically referenced RFP section L.10.4.1.2 (the RFP section that was amended by the "award" of the [deleted] vessels) and affirmatively advised the agency that "[t]his section remains as submitted in our amended technical proposal." Apex Conformed Technical Proposal, Jan. 12, 2000, at 4. Contrary to Apex's expectation that its factual misrepresentation regarding "award" of the [deleted] vessels would enhance the agency's evaluation of its proposal, the TET report states, at APX 3: While the addition of [deleted] vessels on the commercial side gives Government greater availability of off crews for surge, it also means Government will compete for Apex's attention with commercial (profit making) vessels. This is considered moderate but acceptable risk. Apex now complains that the agency's risk assessment was improper because it reflected the agency's consideration of "incorrect, improper and outdated information." [14] Apex Comments on Agency Report, June 30, 2000, at 7. Apex asserts that the agency's reliance on Apex's misrepresentation regarding award of contracts for the [deleted] vessels--an inaccuracy that Apex "did not consider necessary" to correct--constitutes a material flaw in the procurement process which should provide a basis for sustaining Apex's protest. Apex knew the information to be inaccurate at the time it was submitted, but now complains it should not have been considered because it was "incorrect." We find frivolous Apex's assertion that the procurement was materially flawed because of the agency's consideration of the information Apex itself provided in its proposal. Grouping of Proposals Apex next protests that the agency's determination regarding the number of ships for which Apex would be considered was arbitrary and irrational, asserting that it was unreasonable for the agency to have included Apex in the group of offerors considered for 6 to 8 ships rather than in the group of offerors considered for 8 to 10 ships. We disagree. The record shows that the offerors were specifically advised that the agency would make a determination regarding the number of ships for which each offeror would be considered, and that this determination would be based on the relative technical ranking of their proposals, along with the agency's desire to distribute ships among a reasonably large number of ship managers. In this regard, RFP amend. 0013 stated: Question: What factors will MARAD consider in determining the number of ships it feels an offeror can handle? Answer: The determination in [RFP sect.] M.6.2 [regarding the maximum number of ships an offeror may receive] is not based on the number of ships MARAD feels an offeror can handle. . . . . . [I]n order to meet the mission needs and distribute the ships among a reasonably large number of ship managers possessing a wide range of expertise, MARAD may award fewer than 12 ships to many offerors, and may award as few as 2 ships to a successful offeror. MARAD will determine the number of ships an offeror may receive based on the offeror's combined technical/management and risk assessment scores. . . . On the basis of the combined score, each offeror will be ranked in comparison to all other offerors. MARAD will graphically compare the offerors' rankings and look for natural breaks in the scores. . . . Depending on the scores, there may be several groupings of offerors. The top grouping will be targeted to receive the most ships, and the subsequent groups will be targeted to receive fewer ships. The exact score needed for each group and number of ships considered for each group cannot be predetermined as the number of offerors in each group and the range of scores will affect the determination. RFP amend. 0013 at 3-4. As shown in the graph above, consistent with the quoted RFP provision concerning the application of "natural breaks," the agency identified clear groupings of certain proposals; specifically, the proposals of KEY and IUM (scores of 353 and 324) were within 11 points of each other; the proposals of Offeror 12 and Offeror 13 (scores of 171 and 169) were within 2 points of each other; the proposals of Offeror 14 and ODI (scores of 133 and 125) were within 8 points of each other; the proposals of Offeror 16 and Offeror 17 (scores of 80 and 51) were within 29 points of each other; and Offeror 18's proposal (score of 4) was in a group by itself. The agency states that grouping the proposals of MTL, MOR, CLS, AOM, Offeror 7, APX, PCS, PGM and V-Ships (with scores ranging from 285 to 227) was more problematic. Ultimately, the agency divided these proposals into two groups, the first one comprised of the four higher rated proposals (with scores ranging from 285 to 250), since they were within 35 points of each other, and the second, lower rated group comprised of Apex's, V-Ships' and three other offerors' proposals (with scores ranging from 240 to 227), since they were within 13 points of each other. Apex essentially argues that the agency was required to include the Apex proposal in the group of higher rated offerors. We disagree. Had the agency included Apex in the higher group, it would have created a group with a 45 point range between the highest and lowest rated offerors--a larger range than existed in any other group. Further, because there was a second proposal (Offeror 7's) with the same score as Apex's, the agency would have been required to include that proposal in the group as well, creating a group comprised of six, rather than four proposals, all of which would have been considered for award of 8 to 10 ships. In light of the agency's stated desire to distribute the contracts among a reasonably large number of offerors, as well as its statements that the grouping would be based on the relative ranking of proposals and the number of proposals in each group, we do not find arbitrary or unreasonable the agency's inclusion of Apex's proposal with the group of lower rated proposals--all of which were within 13 points of each other. Weighting of Evaluation Factors Apex and V-Ships next complain that the agency's evaluation of technical/ management proposals failed to comply with the RFP provisions regarding the relative importance of the evaluation factors. Specifically, the protesters refer to the following three provisions in RFP section M, wherein offerors were advised that: "[t]he evaluation factors and subfactors are listed in descending order of importance," RFP sect. M.3.2; "[f]actor A is twice as important as [f]actors B or C," RFP sect. M.3.2; and "[a]ll subfactors [under factor C] are approximately equal." RFP sect. M.3.2.3. After reviewing the agency report, the protesters point out that the agency accorded 49 points to factor A, 26 points to factor B, and 31 points to factor C, arguing that this is inconsistent with the RFP provision that factor A would be twice as important as factor C. Further, the protesters note that the agency accorded from 15 to 0.5 points to the eight subfactors established under factor C, arguing that this is inconsistent with the RFP provision that these subfactors would be "approximately equal." [15] The agency acknowledges that "the weighting used was not in exact conformance with the statement of the weights in [RFP] Section M." Supplemental Agency Report at 3. Nonetheless, the agency maintains that neither protester was prejudiced by the agency's failure to follow the arguably conflicting RFP provisions. We agree. After reviewing the agency report, Apex submitted an alternative weighting system which is generally consistent with the RFP provisions that factor A be approximately twice as important as factor C and that the factor C subfactors be approximately equal, yet in descending order of importance. [16] Apex maintains that, if the evaluation factors had been properly weighted under this alternative system, its score would have increased by "approximately 10 to 11 points," which would have led to inclusion of Apex's proposal in the group considered for 8 to 10 ships and, therefore, its earlier consideration in the source selection process. [17] Apex's Post-Hearing Comments at 8. The agency has shown that, even if the proposals had been evaluated under an alternative weighting system which is clearly consistent with the RFP provisions, [18] neither of the protesters' proposals would have offered the lowest price per technical point for any ship group and, therefore, that the source selection decisions would not have been affected. We have reviewed the record in this regard and find the agency's conclusions reasonable. Accordingly, we decline to sustain the protests on the basis of the protesters' assertions that the evaluation factor weights applied by the agency were not precisely as stated in the RFP. Meaningful Discussions Apex and V-Ships next assert that the agency failed to conduct meaningful discussions regarding the agency's evaluation of technical/management factors. We disagree. Agencies are required to conduct meaningful discussions with all competitive range offerors. Stone & Webster Eng'g. Corp., B-255286.2, Apr. 12, 1994, 94-1 CPD para. 306 at 10-11. In order for discussions to be meaningful, contracting officials must advise offerors of deficiencies in their proposals and afford offerors an opportunity to revise their proposals to satisfy the government's requirements. Id.; FAR sect.sect. 15.610(c)(2), (5). [19] This does not mean that offerors are entitled to all-encompassing discussions. Agencies are only required to lead offerors into areas of their proposals that require amplification. Caldwell Consulting Assocs., B-242767, B-242767.2, June 5, 1991, 91-1 CPD para. 530 at 10. Apex and V-Ships first maintain that the agency was required to conduct discussions regarding any of the 37 evaluation areas in which any individual evaluator gave their technical/management proposals a numerical score of 2. [20] We disagree. Here, the individual evaluators' scores did not represent the final scores under any of the evaluation criteria. Rather, the final score in each evaluation area was obtained by aggregating and averaging the six individual evaluators' separate assessments. Neither of the protesters' proposals received aggregated/averaged scores of 2 in any evaluation area and, in fact, their final ratings for each evaluation area exceeded a score of 3 in all instances. Accordingly, we find no merit in this aspect of the protests. The protesters next maintain that the agency should have conducted discussions in any area in which a proposal received an aggregated/averaged score of 3. [21] In short, the protesters maintain that they should have been offered an opportunity to improve their proposals in any area in which their proposals, despite being rated "good" and as having "address[ed] a factor completely," were also perceived as reflecting "unevenness." [22] Where, as here, a proposal is considered acceptable and within the competitive range, the agency is not obligated to discuss every aspect of the proposal that receives less than the maximum score. Caldwell Consulting Assocs., supra. Based on our review of the record here, we do not consider scores of 3, that is, ratings of "good," to represent material weaknesses in an offeror's proposal. Thus, the agency was not required to conduct discussions in those areas. TRW, Inc., B-243450.2, Aug. 16, 1991, 91-2 CPD para. 160 at 9. Our conclusion is not changed by the TET chair's testimony that it was "correct" that if an offeror "cleaned up these significant areas of unevenness, [it] could materially improve [its] proposal." Tr. at 160. We agree with this common sense notion that if every matter which keeps a "good" proposal from receiving an "excellent" score is identified for an offeror, it could improve its score. However, the TET chair's testimony in this regard does not alter the standard our Office will apply to this matter. As V-Ships recognizes in its post-hearing comments, an agency is not required to "spoon-feed" an offeror as to each and every item that could be raised to improve its proposal. Du & Assocs., Inc., B-280283.3, Dec. 22, 1998, 98-2 CPD para. 156 at 7-8. V-Ships next contends, more specifically, that the agency failed to provide complete discussions with regard to its evaluation under the "vessel management" subfactor of evaluation factor C. With regard to V-Ships' conformed proposal, the agency provided V-Ships with the following comments during discussions: Conformed document was hard to follow. New te[x]t was difficult to distinguish from original. When new text was provided it was often general in nature, for example, what does "[V-Ships] enhanced its control of the technical aspects of its Management philosophy by subscribing to modern preventative Maintenance programs and [deleted] classification status Reports." mean? Agency Report, AVS Tab D.10, at 2. After explaining the meaning of the questioned aspect, the agency advised V-Ships to address it in its next proposal submission. Id. The protester observes that it addressed the one example identified by the agency in its final revised proposal and that it received credit for its response. Tr. at 45-46, 121. Nonetheless, V-Ships complains that the agency should have advised it of other specific areas that were considered to be "too general," so that V-Ships could have improved its score still more. While it is true that the protester could have attempted to improve its proposal with additional explanation, we believe the agency's discussions were meaningful as provided. The agency identified the one area as an example. By providing the protester with its view that there were overall problems with the conformed proposal, the agency led it into the area which V-Ships needed to improve its proposal. See Caldwell Consulting Assocs., supra. V-Ships was certainly aware of what aspects of its conformed proposal represented "new text" and could review for itself those aspects which were "general in nature." As such, it should have been capable of addressing those matters in hopes of increasing its score without additional specific rewriting guidance from the agency. Apex also protests that the agency's discussions were less than meaningful with regard to Apex's proposed prices. Apex complains that, although the agency advised Apex during discussions in February 1999 that its prices were so high as to be potentially non-competitive, the agency failed to repeat this advice during final discussions in March 2000. Although an agency must advise a competitive range offeror of weaknesses, excesses, or deficiencies in its proposal, correction of which would be necessary for the offeror to have a reasonable chance of being selected for award, an agency has no obligation to tell an offeror that its price is high, relative to other offerors, unless the government believes the price is unreasonable. State Management Servs., Inc.; Madison Servs., Inc., B-255528.6 et al., Jan. 18, 1995, 95-1 CPD para. 25 at 5-6; Marwais Steel Co., B-254242.2, B-254242.3, May 3, 1994, 94-1 CPD para. 291 at 6. Here, as Apex acknowledges, the agency advised Apex during discussions in February 1999 that its prices were potentially non-competitive and that, in response, Apex lowered its prices somewhat. During final discussions in March 2000, the contracting officer states that she did not view Apex's proposed prices as being unreasonably high, [23] and therefore did not, again, specifically state that Apex's prices were so high as to be potentially non-competitive. Nonetheless, the record does establish that the contracting officer reminded Apex that other offerors might further lower their prices with their final proposal submissions, which would render Apex's prices relatively higher--thereby effectively advising Apex that it might wish to consider further price reductions. In their final proposal submissions following the March 2000 discussions, several offerors whose prices had been higher than Apex's did, in fact, lower those prices--in effect making Apex's proposed prices comparatively higher. Based on our review of the record here, it is clear that, at the time final discussions were conducted, the agency did not view Apex's proposed prices as unreasonably high. Further, by reminding Apex that other offerors might lower their prices with their final proposal submissions, the contracting officer effectively suggested that Apex consider further price reductions of its own. Finally, after final proposals were submitted, the agency still did not consider Apex's prices to be unreasonably high; rather, it simply concluded, upon consideration of Apex's and the other offerors' technical/management ratings, that Apex's prices did not represent the best value to the agency. [24] In sum, we find without merit Apex's assertion that the agency's discussions were inadequate. Best Value Determination Finally, V-Ships contends that the agency's best value determinations were flawed and not properly supported by the evaluation record. Specifically, V-Ships argues that the SEB failed to identify which perceived technical advantages justified payment of price premiums. [25] In this regard, V-Ships notes that it offered lower prices on some ship groups that were ultimately awarded to other offerors. [26] In a negotiated procurement, the government is not required to make award to the lowest-priced, technically acceptable offeror unless the RFP specifies that price will be the determinative factor for award. General Servs. Eng'g, Inc., B-245458, Jan. 9, 1992, 92-1 CPD para. 44 at 9. Agency officials have broad discretion in determining the manner and extent to which they will make use of technical and price evaluation results. Price/technical tradeoffs may be made; the extent to which one may be sacrificed for the other is governed by the test of rationality and consistency with the established evaluation factors. Id. Based on our review of the record, including the TET report, the SEB's report to the SSO, the contracting officer's statement, and testimony obtained at the GAO hearing, we conclude that the record contains adequate support for the selection decisions. [27] While the agency's source selection tradeoffs relied more heavily on the proposals' numerical ratings and the agency's calculation of a PTVR than V-Ships asserts is appropriate, we find the agency's actions reasonable. Agencies are discouraged from relying on point scores alone; however, when point scores are reasonable and supported by the record, an agency may use the results of the scoring as indicative of whether evaluated technical superiority is worth the associated cost premium. PCL/Am. Bridge, B-254511.2, Feb. 24, 1994, 94-1 CPD para. 142 at 5; see also General Offshore Corp.-Riedel Co., a Joint Venture, B-271144.2, B-271144.3, July 2, 1996, 96-2 CPD para. 42 at 8. Where, as here, an agency is faced with evaluating a significant number of proposals from offerors with wide and varied experience, coupled with the need to make a substantial number of awards to multiple contractors, an agency's reasonable attempts to simplify the source selection process is not objectionable. See QualMed, Inc., B-257184.2, Jan. 27, 1995, 95-1 CPD para. 94 at 7. In making its award recommendations, the SEB began with the highest scored/ranked offerors and worked downward, relying on the technical/management ratings and prices proposed to determine the best mix of vessels. Tr. at 202. At the GAO hearing, the SEB chair testified at length regarding the bases for the SEB's determinations. By way of background, the SEB chair pointed out that the ships being awarded are at the "highest level of readiness that the [DOD] assigns any sealift asset," that the majority of the RRF fleet "are in the highest level of readiness," and that they represent a "critical aspect of [DOD's] ability to move cargo" for overseas operations. Tr. at 179, 181-82. Accordingly, consistent with the RFP provisions that technical/management considerations were more important than price, the SEB gave first consideration to the higher ranked proposals. Tr. at 227. With regard to limiting the number of ships to be awarded any single offeror, the SEB considered the ability of offerors to activate multiple ships on short notice. Tr. at 184-85. In making the selections, the SEB also considered the TET report and the TET evaluators' expertise in reviewing offerors' technical/ management proposals and risk, vessel-type recommendations, and the strengths and weaknesses the TET report identified. Tr. at 190. Ultimately, the SEB considered multiple alternative award scenarios in arriving at what it considered the overall best value determinations. Tr. at 203, 205. To verify and validate its selections, the SEB considered each offeror's PTVR or "price per technical point." V-Ships proposal did not offer the lowest PTVR for any ship group on which it proposed. On this record, we see no basis to conclude that the source selection decisions are unreasonable. In a related argument, V-Ships contends that the agency should have conducted a ship-by-ship best value determination, essentially comparing every offeror against every other offeror that proposed on each ship group. This aspect of the protest is untimely. The agency's intention to make award selections as it did was part of a clearly disclosed evaluation scheme. In this regard, RFP amendment No. 0012 advised offerors that the technical/management and risk assessment score would result in a combined score for each offeror. RFP sect. M.6.2. Further, during the August 7, 1998 conference with the offerors, the contracting officer answered the following question: [Question:] Please confirm that a best value determination will be made which reflects the respective expertise and capabilities and experience offerors have in managing ships of a particular class or type, included in each offeror's bid and not just a generic technical ranking. [Answer:] The specific technical experience to vessel type . . . is part of the best value decision where we talked to the best mix of vessels and vessel types commensurate with the offeror's technical and management proposal. We understand that that expertise is important and that certainly will be taken into consideration in the best value decision as stated here. RFP amend. 0013, attach. A, at 65. An agency representative then added, "But if this thing is indicating are we going to have a separate best value for each class of ships or types of ships, we're not." Id. An offeror then asked, "Will there be a different technical rating or score?" Id. at 66. The contracting officer responded, "No, because we are not requiring a separate technical proposal for each class of ships. There will be one technical ranking." Id. Clearly, this exchange between the agency and the offerors notified offerors that each proposal would only receive a single, combined technical/management and risk score and that, while offerors' experience would be considered, there would be no separate best value determinations or technical scores based on an offeror's particular experience relative to each ship grouping. V-Ships' failure to raise this issue until after award renders untimely its objection in this regard. 4 C.F.R. sect. 21.2(a)(1) (2000). The protests are denied. [28] Anthony H. Gamboa Acting General Counsel Notes 1. The RRF is part of the National Defense Reserve Fleet, which supports the rapid deployment of U.S. military forces during national emergencies. The RRF is specifically structured to transport military unit equipment during initial surge for forces deploying anywhere in the world. RFP sect. C.1.1.1. 2. The solicitation divides the 74 ships into 33 ship groups based on various factors including ship type. The ships for which services are sought include what are referred to as tankers, barge carriers, break bulks and roll-on roll-offs. 3. The proposal of another offeror was also initially included in the competitive range, but that offeror subsequently withdrew its proposal. 4. Based on the agency's corrective action, we dismissed the protests as academic. Thereafter, several awardees protested the agency's rescission action. We denied those protests because we concluded that the agency's corrective action was reasonable. Patriot Contract Servs., LLC et al., B-278276.11 et al., Sept. 22, 1998, 98-2 CPD para. 77. 5. Section M of the RFP also identified various subfactors under the three primary factors. 6. The "notional schedule" included specified numbers of days in a year that a ship could be in a particular readiness status. It was designed to allow the agency to normalize per diem rates for all phases over a 365-day period. RFP sect. M.4.2. 7. In March 1999, an offeror filed another protest with this Office which was subsequently withdrawn and refiled at the United States Court of Federal Claims. That court determined that the protest was a maritime matter over which it did not have jurisdiction, and the case was transferred to the United States District Court for the District of Columbia. Subsequently, the matter was withdrawn. In April 1999, a maritime union filed suit against MARAD challenging a solicitation provision relating to the Service Contract Act, which affected the wages and fringe benefits that would be provided under the contracts. On October 14, 1999, the court rendered a decision in favor of the union, enjoining MARAD from awarding any contracts under the solicitation. On November 23, 1999, MARAD issued RFP amendment No. 0027, which incorporated provisions implementing the court order, and reopened discussions with all competitive range offerors. Contracting Officer's Statement at 4-5. 8. Each TET evaluator assigned technical scores from 0 to 5, which corresponded to adjectival ratings that were defined in the agency's source selection plan. For example, a technical/management rating of "5" corresponded to "outstanding," and was defined as reflecting a proposal that: "Addresses a factor completely, and with more than enough original detail as to be convincing, leaving no doubt as to the capability and assurance of consistent high performance. Very comprehensive, in-depth, clear response." RSP sect. 5.7.6. Additionally, evaluators assigned risk scores of 0, 2, 4, or 5, which were also defined in the source selection plan. For example, a risk rating of "0" corresponded with "exceptional," which was defined as: "Little risk. Risk is considered minimal due to but not limited to organizational structure, education and/or experience of personnel, redundancy of personnel and/or the availability of resources. This includes offerors whose PPI [past performance information] surpasses contractual performance requirements in an exceptionally fine, reliable manner." RSP sect. 5.7.8. 9. Legend for offeror abbreviations: Keystone (KEY); Interocean Ugland Management Corp. (IUM); Marine Transport Lines, Inc. (MTL); Mormac Marine Enterprises, Inc. (MOR); Crowley Liner Services (CLS); American Overseas Marine Corp. (AOM); Patriot Contract Services, LLC (PCS); Pacific Gulf Marine, Inc. (PGM); Ocean Duchess, Inc. (ODI). Offerors receiving no awards who are not parties to this protest are designated as "Off." 10. Apex proposed on only [deleted] of the 33 ship groups. For [deleted] groups, the ultimate awardee's proposal had both a higher technical/management rating and a lower price. For the remaining [deleted] groups, contracts were awarded on the basis of proposals with slightly lower technical/management ratings and considerably lower prices, that is, proposals that offered lower prices per technical point. Agency Report (Apex) at 9-10. Similarly, V-Ships proposed on only [deleted] of the 33 ship groups, and its proposal was both higher priced and lower rated in [deleted] of those [deleted] groups. For the remaining ship groups, V-Ships' proposal did not offer the lowest PTVR. Agency Report (V-Ships) at 13. 11. In this regard, though the evaluation employed computer review of electronic price proposals, the agency estimated that the original scheme was so complex that it would allow each offeror to submit up to 7 million generic EOS proposals. 12. The record also reflects that Apex was specifically advised during discussions that its proposal was poorly organized. Specifically the agency's written discussion questions to Apex stated: "Throughout this proposal, Apex chose to merge factors instead of specifically addressing factors. This left it to the Government to decipher and give[ ] credit where applicable. Each of the factors is touched somewhere, but not as a coherent response. Makes evaluation a challenge." Agency Report, APX Tab D.1, at 1. This point was reiterated by one of the evaluators that testified at the hearing GAO conducted in connection with this protest. Hearing Transcript (Tr.) at 316. In this regard, section M of the RFP specifically advised offerors that "[e]valuation of an offeror's proposal shall be based on the information presented in the proposal . . . . The quality of organization and writing reflected in the proposal will be considered to be an indication of the quality of organization and writing which would be prevalent in the proposed deliverable data." RFP sect. M.7.6. 13. For example, under the "general corporate experience" subfactor (RFP sect. L.10.4.1.1), the evaluation record supports Keystone's significantly better technical and risk evaluations in comparison with V-Ships' proposal evaluations. In this regard, all six evaluators found Keystone's proposal to be exceptional with regard to risk and noted that its proposal "described clear lines of authority between parent, affiliates, and offeror [and] described a corporate history of maintenance and operation of a large number of ocean going vessels of varied ship types and types of plants. Corporate experience is lengthy with repeated service contracts. Outstanding tech with many years in managing many ships contributing to low risk." TET Report at KEY 11. 14. Apex asserts that it discussed its failure to receive the contract for the [deleted] ships with various agency personnel. Affidavit of Apex President para.para. 2, 3. At the GAO hearing, the identified personnel testified that such discussions had not occurred, Tr. at 87-88, 355, and one of the witnesses elaborated, stating, "had I heard of it [the loss of the contract for the [deleted] ships], I would have had lots of questions." Tr. at 88. 15. The agency argues that this aspect of the protest is untimely since the RFP was patently ambiguous regarding how the evaluation factors would be applied. Specifically, the agency asserts that the RFP provision that "evaluation factors and subfactors are listed in descending order of importance" conflicts with the provision that "[factor C] subfactors are approximately equal," and, therefore, that this patent ambiguity should have been protested before the proposals were submitted. We need not resolve this matter in light of our conclusion that, even accepting the assertion that the proposals should have been evaluated in an alternative manner under evaluation factor C, the source selection decisions would not have been affected. 16. The alternative evaluation system established a total weight for factor C that was approximately half the value the agency applied to factor A, and established weights for the factor C subfactors that are very close, yet in descending order. 17. In its post-hearing comments, V-Ships also proposed an alternative weighting scheme under which it argued that its proposal's combined score would be higher than two awardees' proposals. 18. The agency response properly rejects any changes to the evaluation system other than those necessary to meet the RFP's stated provisions that the primary evaluation factors be listed in descending order of importance, that factor A would be twice as important as factor C, and that all factor C subfactors be approximately equal but in descending order. 19. The RFP in this procurement was issued prior to the effective date of the comprehensive rewrite of FAR Part 15 (FAC 97-02). Accordingly, the FAR provisions in effect at that time are applicable to this procurement. 20. The agency's source selection plan provided that a numerical score of 2 corresponded with an adjectival rating of "Fair," further defining this rating as: "Does not fully address factor, or does so with little original detail so as to raise questions as to the ability of the offeror to fully meet the requirements of the factor. Weaknesses remain, but are confined to areas with minor impact on performance and are acceptable." RSP sect. 5.7.6. 21. The source selection plan established that a numerical rating of 3 corresponded with an adjectival rating of "good," further defining this rating as: "Addresses a factor completely but with significant areas of unevenness, which may impact performance." RSP sect. 5.7.6. 22. V-Ships argues that the language regarding "unevenness" effectively makes this rating a negative score. We disagree. When all evaluation definitions are read together, it is plain that the differences were simply intended to address varying levels of offeror competence and the "unevenness" reference, in context, was not intended to indicate a material weakness. 23. Less than a third of the prices proposed by Apex were in the upper third of the prices proposed. 24. Apex complains that "[the agency] knew that Apex's price competitiveness would be substantially affected by its low ranking score. Nevertheless it did not advise Apex that this might be a problem." Supplemental Protest at 11. Based on this assertion, Apex appears to be suggesting that the agency had a differing obligation regarding price discussions with each offeror, depending on each offeror's relative technical/ management rating. We do not believe the agency had any such differing obligation. 25. The source selection official (SSO) states that he relied on the recommendations of the SEB in making the source selection decisions. Declaration of SSO, July 19, 2000. Because the SSO relied on the expertise of the evaluators, SEAIR Transport Servs., Inc., B-274162, Nov. 25, 1996, 96-2 CPD para. 198 at 2, our review of the best value determinations is based on adequacy of the documentation and reasonableness of the SEB recommendations. 26. As noted above, V-Ships proposed on only [deleted] of the 33 ship groups, and its proposal was both higher priced and lower technically rated for [deleted] of these [deleted] groups, eliminating any need for cost/technical tradeoffs regarding those groups. With regard to the remaining groups, the record supports the reasonableness of the agency's tradeoff determinations. 27. V-Ships argues that we should ascribe little weight to the contracting officer's statement because it was prepared in response to the protest. We disagree. The SEB chair testified that the discussion of the SEB's selections in the contracting officer's statement accurately reflected the agency's decisions. Tr. at 209. In this regard, the contracting officer was a voting member of the SEB and attended all of its sessions. Tr. at 173, 224-25. Since the contracting officer had personal knowledge of the information regarding selection decisions and recounted decisions made at the time of the selections, her statement has probative value in our review of the adequacy of those decisions. 28. Both Apex's and V-Ships' protests refer to various additional issues, or present alternative arguments, which this decision does not directly address. We have considered all of the issues and arguments raised and conclude that none of them provides a basis to sustain either protest.