TITLE: B-296861, EDO Corporation, September 28, 2005 BNUMBER: B-296861 DATE: September 28, 2005 ********************************************* B-296861, EDO Corporation, September 28, 2005 DOCUMENT FOR PUBLIC RELEASE The decision issued on the date below was subject to a GAO Protective Order. This redacted version has been approved for public release. Decision Matter of: EDO Corporation File: B-296861 Date: September 28, 2005 Jeffrey P. Bialos, Esq., Christer L. Mossberg, Esq., John M. Wingate, Esq., and Bradley E. Prendergast, Esq., Sutherland, Asbill & Brennan LLP, for the protester. Maj. Lawrence Anderson, Maj. Christina M. Slicker, and Bridget E. Lyons, Esq. Department of the Air Force, for the agency. Mark D. Colley, Esq., and Kara L. Daniels, Esq., Holland & Knight LLP, for Raytheon Company, an intervenor. Paul E. Jordan, Esq., and John M. Melody, Esq., Office of the General Counsel, GAO, participated in the preparation of the decision. DIGEST 1. Cost realism evaluation of protester's proposal was unobjectionable where record shows that upward cost adjustments were based on evaluators' reasonable assessments that proposal significantly understated labor hours needed to perform. 2. Agency's identification of risk associated with protester's very low fixed production price is not relevant to award decision where selection authority did not cite the identified risk in making award decision. 3. Agency reasonably evaluated protester's proposal as high risk, despite protester's experience, due to proposal's failure to include sufficient support for low proposed labor hours. DECISION EDO Corporation's EDO MTech, Inc. (EDO) protests the award of a contract to Raytheon Company under request for proposals (RFP) No. FA8626-04-R-2067, issued by the Department of the Air Force for development of the Smart Triple Ejector Rack (TER) bomb rack unit. EDO challenges the agency's cost realism evaluation of EDO's proposal and the resulting award to Raytheon. We deny the protest. The RFP contemplated the award of a cost-plus-fixed-fee contract to develop modifications to the TER-9A, with two fixed-price options to produce up to 60 units each. This is a follow-on effort to modify a portion of the existing TER-9A, a bomb carriage and release mechanism to carry precision guided (smart) weapons as well as conventional weapons on F-16 aircraft. The basic contract effort was expected to be completed in 20 months. The RFP included a statement of objectives outlining the agency's expectations. Offerors were required to submit detailed technical proposals for development and production of the units, and detailed cost proposals including work breakdown structures (WBS) and basis of estimate (BOE) data as support. Proposals were to be evaluated on a "best value" basis considering four factors--mission capability, proposal risk, past performance (all of equal importance), and cost/price. The mission capability factor was divided into five subfactors, the first four of which--system performance; system engineering, system support, production readiness, and participation of small and small disadvantaged businesses, historically black colleges and universities, and minority institutions--were considered of equal importance. The technical factors combined were significantly more important than cost/price. Each mission capability subfactor was to be rated on a color/adjectival basis with individual risk ratings.[1] Proposed costs were to be evaluated for realism, including calculation of a most probable cost (MPC), and proposed prices were to be evaluated for reasonableness. Five proposals, including EDO's and Raytheon's, were received and were evaluated by the source selection evaluation team. Three proposals were included in the initial competitive range and the agency conducted written discussions with the three offerors. Based on evaluation of the offerors' responses, only EDO and Raytheon were included in the final competitive range. The agency conducted discussions with and obtained final proposal revisions (FPR) from both. The source selection authority (SSA) subsequently reopened negotiations and, following a third round of discussions, both offerors submitted revised FPRs. The agency concluded from its cost realism evaluation that EDO had failed to adequately support the adequacy of its proposed labor hours in various categories, and therefore added hours in those categories. EDO's MPC was adjusted upward based on the burdened cost of these additional hours and its risk rating under the system engineering subfactor was raised to high. The results of the final evaluation were as follows: +------------------------------------------------------------------------------+ | | EDO | Raytheon | |---------------------------+-------------------------+------------------------| |Mission Capability | | | |---------------------------+-------------------------+------------------------| |system performance | blue/low | blue/low | |---------------------------+-------------------------+------------------------| |system engineering | green/high | green/low | |---------------------------+-------------------------+------------------------| |system support | green/low | green/low | |---------------------------+-------------------------+------------------------| |production readiness | green/low | green/low | |---------------------------+-------------------------+------------------------| |small business | green/low | green/low | |---------------------------+-------------------------+------------------------| |Past Performance | significant/confidence | significant/confidence | |---------------------------+-------------------------+------------------------| |Most Probable Cost | $2,739,118 | $3,103,972 | +------------------------------------------------------------------------------+ Based on an integrated assessment of the evaluation factors, the SSA concluded that Raytheon's proposal represented the best value. While he found that both proposals met the RFP requirements, he concluded that Raytheon's proposed tasks under the systems engineering subfactor demonstrated a more complete plan to accomplish the work and provided greater confidence that the firm would deliver a final product on time, within cost, and would provide the required performance. Although EDO's proposal offered a lower price, the SSA found that its systems engineering approach was not adequately supported by sufficient labor hours, and therefore warranted a high risk rating. The SSA determined that Raytheon's lower risk outweighed any advantage attributable to EDO's lower cost, and therefore made award to Raytheon. After receiving notice of the award and a debriefing, EDO filed this protest. EDO challenges the award on numerous bases. We have considered all of EDO's arguments and find that they have no merit or did not prejudice the protester. We address EDO's most significant arguments below. MPC EVALUATION The RFP provided that an offeror's proposed costs would be evaluated to determine if they were realistic for the work to be performed, reflected a clear understanding of the requirements, and were consistent with the unique method of performance and materials described in the technical proposal. As part of this evaluation, the RFP called for calculation of an MPC that would include the basic requirements and all options, and would be calculated as the sum of contractor proposed cost of the system development and demonstration (SDD); other government costs (OGC) "includ[ing] Government test and integration/interface into the F-16 and aircraft platform"; proposed use of government furnished property (GFP) to include that identified by the offeror; and risk. RFP sect. M.1.6.3. EDO challenges the agency's calculation of the MPC in all but the risk area. EDO's Proposed Labor Hours EDO asserts that the agency's cost realism evaluation arbitrarily added the cost of additional labor hours to its MPC. Specifically, EDO complains that the additions were based on the evaluators' opinions instead of proven cost analysis techniques-- such as comparison with historical data or parametric cost estimating relationships--as suggested in Federal Acquisition Regulation (FAR) sections 15.404-1(b) and (c). When an agency evaluates proposals for the award of a cost-reimbursement contract, an offeror's proposed estimated costs are not considered controlling because, regardless of the costs proposed, the government is bound to pay the contractor its actual allowable costs. FAR sections 15.305(a)(1), 15.404-1(d). Consequently, a cost realism analysis must be performed by the agency to determine the extent to which an offeror's proposed costs represent what the contract should cost, assuming reasonable economy and efficiency. FAR sect. 15.404-1(d)(2); Hanford Envtl. Health Found., B-292858.2, B-292858.5, Apr. 7, 2004, 2004 CPD para. 164 at 8-9. However, an agency is not required to conduct an in-depth cost analysis, see FAR sect. 15.404-1(c), or to verify each and every item in assessing cost realism; rather, the evaluation requires the exercise of informed judgment by the contracting agency. Cascade Gen., Inc., B-283872, Jan. 18, 2000, 2000 CPD para. 14 at 8. Our review of an agency's cost realism analysis is limited to determining whether the analysis was reasonable. Systems Integration & Research, Inc., et al., B-279759.2 et al., Feb. 16, 1999, 99-1 CPD para. 54 at 7-8. The cost realism analysis of EDO's labor hours was reasonable. Each offeror's cost proposal was to be based on the number of labor hours, material, and indirect costs necessary to accomplish the statement of work as proposed to be performed by the offeror. As part of the cost proposal, offerors were required to submit WBSs and BOE information, defined as detailed data supporting the estimates in the system development and demonstration (SDD) chapter, and to "completely describe the philosophy and methodology used in developing the estimates together with appropriate references to any historical supporting cost data." RFP sect. L.5.3.2. EDO's initial cost proposal did not include detailed [deleted] data, but after two rounds of discussions, which included specific requests by the agency for the BOEs, EDO furnished detailed WBSs and BOEs in its revised FPR. When the evaluators--an armament integration engineer and an armament integration technical expert--reviewed the FPR, they found that the proposed hours were not sufficient. Evaluators' Declaration at 1, Aug. 23, 2005. Based on the evaluators' experience, they adjusted EDO's proposed labor hours in various categories, using EDO's proposal as guidance. The evaluators only added hours when EDO did not provide a sufficient explanation for why the proposed hours were adequate for the tasks proposed. Id. at 2. Where EDO's proposed hours were supported with historical references--such as program titles or contract numbers, as required by RFP sect. L.5.3.2(c)--no hours were added. In making these adjustments, the evaluators also gave consideration to EDO's experience, as documented in its proposal. Contracting Officer's Statement at 12. In all, the evaluators added more than 7,000 hours to the [deleted] hours proposed by EDO. These additional hours were multiplied by the appropriate labor rates and the resulting cost was added to EDO's proposed cost in calculating the MPC. EDO asserts that no additional labor hours were required and maintains that the agency has not provided a sufficient rationale for the adjustments. EDO's assertions are without merit. The agency's contemporaneous cost documentation includes a matrix identifying the bases for all but two of the labor hour increases, and two spreadsheets, one of which includes embedded comments identifying the remaining bases for additional hours. Agency Report (AR) Tab 9. In addition, the agency has provided several declarations detailing the evaluators' steps and rationales in making the labor hour and cost adjustments.[2] Example--PM Labor Hours The evaluators noted that EDO only proposed [deleted] hours for the proposed project manager (PM) over the 20-month performance period, or less than [deleted] per month. This was inconsistent with the evaluators' view of the PM position. Based on the complexity of the deliverables and development effort, the evaluators viewed the PM as essentially responsible for the success of the entire program; the PM would have to manage every aspect of the program, ensuring that technical tasks were completed and that funds were expended on time, and also would have to update the estimated time of completion to determine whether the program was ahead or behind schedule. Consistent with the evaluators' view, EDO's WBSs and BOEs identified seven tasks that the PM would perform, including [deleted]. Evaluators' Declaration at 3. EDO's proposal described the PM as [deleted]. EDO Proposal, Vol. I, at 38. Based on their view of the PM position and the specific tasks proposed by EDO, the evaluators determined that a full-time PM was required; they therefore added 1,723 labor hours for the PM over the 20-month performance period. Evaluators' Declaration at 4. They also added another 108 hours to cover the PM's attendance at three interface control working group meetings and the technical interchange meetings; since the evaluators' experience was that these meetings were attended by the government, contractor (EDO), aircraft contractors, and weapons contractors, they concluded that the PM, as the contract point person responsible for the overall success of the contract, should attend the meetings. Evaluators' Declaration at 5-7. Subsequently, the evaluators met and determined that, given the duties and responsibilities of the PM as described in EDO's proposal and the complexity of the program, there remained a high level of risk in EDO's ability to complete the tasks even with the additional hours; they therefore determined that an additional 1,300 hours should be added in order to make the PM full time for the entire 20-month program. Evaluators' Supplemental Declaration, Sept. 6, 2005, at 1-2. This justification appears reasonable, and EDO has not shown otherwise. EDO asserts that adding any hours for its PM is improper because the RFP did not require any specific threshold of PM participation and did not require the PM to perform all program management tasks.[3] EDO notes that its past performance proposal showed that, in several projects performed in overlapping performance periods, it [deleted], thus supporting its proposal of a part-time PM here. EDO Comments at 30. This argument is without merit. The RFP clearly required each offeror to completely describe the philosophy and methodology used in developing the BOEs, together with appropriate references to any historical supporting cost data, in its cost proposal. RFP sections L.5.1.1, L.5.3.2. Further, the RFP specifically advised that the burden of proof for cost credibility rested with the offerors and cautioned offerors to include fully responsive cost information only in the cost/price section of the proposal. RFP sect. L.5.1.1. EDO did not support its proposal of a part-time PM in its cost proposal, and there was no duty on the evaluators' part to search the technical proposal volumes to see if they could find some credible support for the proposed hours. In view of the importance of the PM position, and given the responsibilities assigned the PM in EDO's own proposal, we find no basis for taking exception to the labor hours added to EDO's proposal for this position. EDO's disagreement with the evaluators' judgment does not render the evaluation unreasonable. See Command Mgmt. Servs., Inc., B-292893.2, June 30, 2004, 2004 CPD para. 168 at 3. Example--Software Development Labor Hours EDO asserts that the addition of labor hours associated with software development and related tasks was based only on the evaluators' "intuition," and was not reasonably supported. More specifically, EDO maintains that the evaluators ignored its past experience with the [deleted] system, from which it proposed to extend and reuse more than 75 percent of the source lines of code required for the Smart TER-9A. EDO Comments at 30-31. This argument is without merit. Contrary to EDO's perception, the evaluators did credit EDO in the software development area for its past experience. For example, with regard to developing and writing code for the [deleted] emulation mode and Smart and Dumb TER modes, the evaluators gave EDO the "benefit of the doubt based on demonstrated experience," and only added hours in conjunction with development of new code. Evaluators' Declaration at 21. The evaluators added these hours after noting that EDO had proposed only [deleted] hours to write [deleted] lines of new code. The evaluators determined that there was "an absolute minimum estimate of one hour per each new line of code (and it usually takes longer)," and therefore reasoned that "the time to produce [deleted] lines of new code would be [deleted] hours." Since only [deleted] of EDO's [deleted] proposed hours were designated for code writing, the evaluators added 200 hours, which also were intended to cover the absence from EDO's proposal of any hours for coding updates. Id. Apart from asserting that these additions are arbitrary and unsupported, EDO does not explain how the evaluators' reasoning was flawed. In our view, this and the other software-related additions were reasonable; EDO has not shown otherwise. Evaluation of EDO's Production Price Noting that the RFP called for inclusion of the cost of options in the MPC, EDO asserts that the agency erred by failing to calculate an MPC for its production price. EDO also asserts that the evaluation of its production price was flawed because it included a comparison to a higher-priced, older system--the [deleted]--and improperly concluded that its lower proposed price was risky. The evaluation of EDO's production price was unobjectionable. While the RFP called for the MPC to include options (RFP sect. M.1.6.3), it did not specify how this aspect of the MPC was to be calculated. The agency compared EDO's proposed production price to its own estimate and pricing information for other bomb release units, including the older [deleted] system. Noting that the [deleted] units cost more than $100,000 and that the estimate for the Smart TER-9A exceeded $54,000, the evaluators found EDO's proposed price of [deleted] unrealistic. Price Competition Memorandum at 2, 6, and 8. We see nothing unreasonable in the agency's methodology or in its analysis of EDO's production price. As for the evaluators' risk assessment, there is no indication that this assessment adversely impacted the source selection. In this regard, the evaluators accepted EDO's price without adjustment, as they were required to do given that this aspect of the procurement was fixed-price and any cost risk was EDO's, not the agency's. Supplemental Contracting Officer's Statement at 9. Further, the SSA was aware that EDO's basic effort MPC and fixed-price options were lower than Raytheon's and that EDO's blue/low risk rating under the system performance subfactor was based in part on its having surpassed the affordability objective for the production units. While the SSA was also aware that the evaluators considered the production price to be unrealistic and risky, there is no evidence that these assessments were considered in, or had any impact on, his award decision. Rather, the record shows that the SSA's consideration of risk was associated only with the basic cost--and not the fixed-price option--aspect of the procurement. Specifically, the SSA noted that "[w]hile EDO's proposal offered a lower price, its . . . approach, . . . was not adequately supported by sufficient proposal hours, and was determined to be high risk in its ability to deliver a product to the government without overruns to their proposed cost and an extension in schedule. It was determined that it is not in the best interest of the government to accept the higher risk proposal, although proposed at a lower cost." AR, Tab 17 at 2. Under these circumstances, we find nothing unreasonable in this aspect of the evaluation. Calculation of OGC and GFP EDO asserts that the MPC calculation is flawed because it includes no cost for OGC and GFP. In EDO's view, calculation of these costs could have given it an advantage over Raytheon, since a more experienced offeror such as itself might require less GFP and its possession of pre-certified testing equipment could lessen the OGC factor. EDO Comments at 26. This argument is without merit. The agency did not cost GFP for either offeror, since GFP cost was to be based on offerors' "proposed use" (RFP sect. M.1.6.3(a)(3)), and neither offeror proposed any GFP beyond that listed in the RFP. The agency did not cost OGC because government test and integration costs were to be handled under a separate contract and were otherwise included within the 5-percent risk calculation for each offeror. Supplemental Contracting Officer's Statement at 8. While EDO disagrees with the agency, it has not established that any different amount should have been included. We conclude that the evaluation in this area was reasonable. PROPOSAL RISK ASSESSMENT EDO asserts that the agency failed to take its experience into account when rating it as high risk under the systems engineering subfactor. In reviewing a protest of an agency's proposal evaluation, our review is confined to a determination of whether the agency acted reasonably and consistent with the terms of the solicitation and applicable statutes and regulations. United Def. LP, B-286925.3 et al., Apr. 9, 2001, 2001 CPD para. 75 at 10-11. The risk assessment here was unobjectionable. The RFP provided for the evaluation of proposal risk associated with the offerors' approach covering the various mission capability subfactors to determine the likelihood and consequence of performance shortfalls, schedule slips, increased costs, the need for increased government oversight, and the likelihood of unsuccessful contract performance. RFP sect. M.1.4. A high risk rating was defined as "[l]ikely to cause significant disruption of schedule, increased cost or degradation of performance." Id. EDO received evaluation credit for its past experience, as reflected in its superior technical and past performance scores. However, EDO received a high proposal risk rating due to its significant understatement of labor hours necessary to accomplish its proposed tasks. With respect to proposal risk assessments, the appropriate point of departure is the proposal, not the agency's experience with the offeror, no matter how good its performance may have been. Modern Techs. Corp. et al., B-278695 et al., Mar. 4, 1998, 98-1 CPD para. 81 at 7. While EDO's proposal referred to its experience and success in relevant technology, it simply failed to provide sufficient support for its proposed effort in its cost proposal as required by RFP sect. L.5.1.1. Consequently, the evaluators added labor hours and determined that EDO's failure to propose them initially indicated a high risk of unsuccessful performance, notwithstanding its experience. We find nothing unreasonable in this determination. MEANINGFUL DISCUSSIONS In its comments on the agency's report, EDO asserts that the agency improperly failed to hold discussions with it concerning its understated labor hours, risk associated with its production prices, and alleged negative past performance information. EDO asserts that, had the agency brought up these matters in discussions, it could have satisfactorily defended its proposal and past performance. EDO Comments at 12-24. Protest grounds such as this must be raised within 10 days after the basis for protest is known or should have been known. Bid Protest Regulations, 4 C.F.R. sect. 21.2(a)(2) (2005). Here, with regard to the labor hour and production pricing issues, the agency provided the evaluation documents on which these arguments are based during the debriefing held on June 30, 2005 and in the agency report filed on August 11. Information concerning EDO's past performance was provided to the protester in a supplemental document disclosure on August 18. However, EDO raised these issues for the first time in its comments, which were filed on August 30, more than 10 days after it received these documents. Accordingly, these protest grounds are untimely and will not be considered.[4] The protest is denied. Anthony H. Gamboa General Counsel ------------------------ [1] The color ratings included blue/exceptional, green/acceptable, yellow/marginal, and red/unacceptable. Risk ratings included low, moderate, and high. Past performance ratings included exceptional/high confidence, very good/significant confidence, satisfactory/confidence, marginal/little confidence, and unsatisfactory/no confidence. [2] EDO asserts that our Office should not consider these documents created in response to the protest because they were not contemporaneous evaluation documents. See Boeing Sikorsky Aircraft Support, B-277263.2, B-277263.3, Sept. 29, 1997, 97-2 CPD para. 91 at 15. While we accord greater weight to contemporaneous source selection materials, we will nonetheless consider the entire record, including statements and arguments made in response to a protest, in considering whether an agency's source selection decision is supportable. Id. Post-protest explanations that provide a detailed rationale for contemporaneous conclusions may, as is the case here, simply fill in previously unrecorded details, and will generally be considered in our review of the rationality of the selection decision as long as those explanations are credible and consistent with the contemporaneous record. Jason Assocs. Corp., B-278689 et al., Mar. 2, 1998, 98-1 CPD para. 67 at 6. [3] EDO also asserts that the additions are improper because adding 3,131 hours to its [deleted] proposed hours results in a total of [deleted] hours, which is more than a full-time equivalent over a 20-month period. In this regard, EDO notes that the agency based its calculation on 92 weeks, an average of 41 hours per week, when 20 months actually encompasses 87 weeks and a standard work year is 2,000 hours, less than the 41 hours per week used by the evaluators. EDO Supplemental Comments at 13, n.12. EDO has not provided any calculation of the impact of the cost of the alleged excess hours. Our own calculations indicate that adjusting the hours as suggested by EDO would result in an MPC of $2,589,139 instead of the $2,739,118 used by the agency, a difference of $149,979. Since the SSA specifically determined that EDO's lower cost proposal did not outweigh the risk associated with it, and therefore awarded the contract to Raytheon at its higher MPC of $3,103,972, there is no reason to believe that this somewhat reduced MPC would have had any effect on the tradeoff and award decision. [4]In any event, none of these grounds has merit. Despite the RFP's requirements for BOEs, EDO did not provide them until the third round of discussions. It was at this time that the agency was able to identify EDO's detailed basis for the hours it had proposed. Where as here, an offeror introduces defects into its proposal in its FPR, it runs the risk that the agency will exercise its discretion not to reopen discussions. Metcalf Constr. Co., Inc., B-289100, Jan. 14, 2002, 2002 CPD para. 31 at 5. As to the production price, as discussed above, the agency accepted it, simply noting the apparent risk. Similarly, with regard to past performance, the record reflects that the agency did not downgrade EDO's proposal for any past performance weakness and, in fact, rated it as significant confidence, the second highest rating. Since these two matters had no adverse impact on EDO's evaluation, there was no need for the agency to raise them in discussions.