BNUMBER: B-277029.3 DATE: February 18, 1998 TITLE: [Letter], B-277029.3, February 18, 1998 ********************************************************************** DOCUMENT FOR PUBLIC RELEASE The decision issued on the date below was subject to a GAO Protective Order. This redacted version has been approved for public release. Matter of:McAllister & Associates, Inc. File: B-277029.3 Date:February 18, 1998 Paulette Cross-Castle, Esq., for the protester. Jeffrey C. Morhardt, Esq., Department of Education, for the agency. Marie Penny Ahearn, Esq., and John M. Melody, Esq., Office of the General Counsel, GAO, participated in the preparation of the decision. DIGEST Exclusion of protester's proposal from competitive range was reasonable where agency reasonably concluded that proposal contained informational deficiencies so pervasive that it did not demonstrate protester's understanding of solicitation requirements and thus had no reasonable chance of being selected for award. DECISION McAllister & Associates, Inc. protests the Department of Education's (DOE) elimination of its proposal from the competitive range, and the award of multiple contracts to other offerors, under request for proposals (RFP) No. 96-011, for debt collection services. We deny the protest. The RFP contemplated the award of one or more contracts for collection and administrative resolution activities on debts resulting from nonpayment of student loans. Award was to be based on a best value evaluation, with technical merit and past performance combined significantly more important than cost or price. Under the technical factor, the RFP included the following subfactors (with possible ratings out of 450 available points): (1) executive summary (25 points); (2) strategic collections approach (75 points); (3) administrative resolutions (30 points); (4) litigation/administrative wage garnishment (AWG) preparation (45 points); (5) quality control (QC) plan (75 points); (6) training plan (40 points); (7) computer system resources (60 points); and (8) project personnel experience (100 points). Under the price factor, the RFP requested commissions/fees for certain work categories and provided target maximum rates above which a proposal could be considered unacceptable for award. The RFP specifically provided that "[a] detailed work plan must be submitted indicating how each aspect of the [SOW] is to be accomplished" and that "[t]he technical proposal should reflect a clear understanding of the nature of the work being undertaken." Sixty-five proposals were received, 21 of which ultimately were included in the revised competitive range. McAllister's proposal, which was rated deficient in all technical areas, received a final composite technical score (the average of the scores from the five evaluators) of 131.6, or 29.2 percent of the 450 possible points,[1] and was excluded from the competitive range.[2] In its rejection letter to the protester, the agency stated that its proposal "d[id] not provide sufficient information to demonstrate [the] ability to fulfill the general requirements of the RFP," and specifically failed to (1) "demonstrate that [its] system can handle the volume of accounts [DOE] expects to transfer within a reasonable time frame," (2) "demonstrate an understanding of the procedures [for] identifying and preparing accounts for litigation and AWG," (3) "discuss [QC] measures," (4) "include [DOE] specific training," and (5) "contain an adequate staffing plan." The agency concluded that these informational deficiencies were so severe that the proposal did not demonstrate an understanding of DOE's requirements and would require "extensive and exhaustive discussions to correct its information deficiencies." The agency proceeded with awards to 18 other offerors. McAllister maintains that the information contained in its proposal was sufficiently detailed to show compliance with the RFP requirements and to warrant retaining it in the competitive range. The protester asserts that any deficiencies were of a type that could have been corrected during discussions. An offeror has the burden of submitting an adequately written proposal for the agency to evaluate, Premier Cleaning Sys., Inc., B-255815, Apr. 6, 1994, 94-1 CPD para. 241 at 5, and agencies may exclude proposals with significant informational deficiencies from further consideration. HITCO, 68 Comp. Gen. 10, 12 (1988), 88-2 CPD para. 337 at 3. This is true whether the deficiencies are attributable to either omitted or merely inadequate information addressing fundamental factors. Id. In reviewing an agency's determination to exclude a proposal from the competitive range, we apply the standard used in reviewing all aspects of an agency's technical evaluation of proposals--we review the record to determine whether the agency's judgment, including the judgment that a particular proposal did not have a reasonable chance of award, was reasonable, supported by the record, and consistent with the applicable evaluation criteria. Pedus Bldg. Servs., Inc., B-257271.3 et al., Mar. 8, 1995, 95-1 CPD para. 135 at 2-3. Based on our review of the record, including the narrative and point score sheets completed by each of the five evaluators and the summary narrative prepared by the evaluation panel chair, we conclude that the agency reasonably determined that McAllister's proposal had significant informational deficiencies such that the firm lacked a reasonable chance for award. We discuss some key areas of the evaluation below. PROJECT PERSONNEL EXPERIENCE Project personnel experience (or staffing plan, as the agency describes it), was the most heavily weighted subfactor, worth 100 points.[3] McAllister's proposal received an average score of 30.2 points in this area. The summary evaluation narrative lists 10 evaluated weaknesses, including: (1) "[o]fferor does not discuss the use of an [a]dministrative [s]upport staff," (2) "[s]ystems [m]anager is offeror's president[; i]t would be necessary to have some idea of his other workload, number of clients, and such to determine adequacy," (3) "[q]ualifications of key personnel are fair[; p]ast experience with [DOE] and [DOE] contractors is not recent," (4) "[t]here appears to be conflicts concerning the duties and the time commitments indicated[; f]or example, it appears that the QC manager will be doing ALL the reviews, however the time commitment is only 50 [percent]???," and (5) "[t]here is no information on the offeror's commitment to high quality service[; t]hey only list four (4) clients and give the placement per year but there is no indication of length of the contracts or how long they have been doing business with offeror[; t]here are no letters of commendation, etc." McAllister does not rebut any of the specific evaluated deficiencies in this area and does not dispute that they are material. Instead, the protester contends that, based on the past successful experience of its chief executive officer (CEO) and other management staff with similar contracts, including a DOE contract, the agency could not have reasonably concluded that the firm's staffing capacity, qualifications, and understanding of the requirements in this area were deficient. McAllister's reliance on the past experience of its staff without regard to the specific information required by the RFP is misplaced. A procuring agency's technical evaluation is dependent upon the information furnished in the offeror's proposal. Computerized Project Management Plus, B-247063, Apr. 28, 1992, 92-1 CPD para. 401 at 3. An agency is not required to overlook a flawed proposal on the basis of the offeror's prior performance; on the contrary, all offerors are expected to demonstrate their capabilities in their proposals. Pedus Bldg. Servs., Inc., supra, at 4. Consequently, the agency's reliance on McAllister's proposal in determining the adequacy and qualifications of the firm's staffing was proper, and since McAllister has not shown that the evaluated deficiencies are unfounded based on information contained in its proposal, we have no basis to object to this aspect of the evaluation. See Premier Cleaning Sys., Inc., supra, at 6. QUALITY CONTROL Under QC, one of two subfactors weighted second highest, worth 75 points, the RFP advised that "[t]he offeror must provide its proposed [QC] plan outlining its corporate commitment to effective, economical [QC] measures and specific descriptions of [QC] functions to be included in the [QC] plan deliverable."[4] The evaluation of this subfactor encompassed the following: (1) "[o]fferor's corporate commitment to effective, economical [QC] measures"--30 points; (2) "[s]pecific descriptions of QC functions to be included in QC plan deliverable"--25 points; and (3) "[n]ature of reporting mechanisms, both internal to the offeror and to [DOE], that will be developed to report problems identified in QC reviews"--20 points. McAllister's proposal received an average of 19.2 points, or 25.6 percent, for QC. The summary evaluation narrative identifies 10 weakesses in McAllister's proposal under QC, including the following: (1) "[t]he offeror briefly mentions that they are committed to QC measures, but they don't state what they plan on measuring," (2) "[t]his proposal does not tell . . . how problems will be prevented and does not indicate the number (frequency or amount) of samplings for the few proposed audits," (3) "[t]he offeror does not talk about QC functions to be included in the QC plan deliverable," (4) "[n]o reporting mechanisms are discussed other than the Management and Fiscal Report, Offeror's bi-annual audits, and statements of awareness of [DOE]'s reviews of contractors," and (5) "[p]ages 43 through page 52 [i.e., last 10 pages of the 20 pages devoted to QC] of this section are a reiteration of the SOW." McAllister disputes that its proposal was informationally deficient in the QC area. It maintains that the 20-page QC section of its proposal was a "lengthy discussion" of "the actions that take place in the process of monitoring, fostering and facilitating [QC] . . . by [the protester]" with "the first 12 pages [i.e., pages 32 through 43] . . . procedurally specific" to the protester in terms of "software facilitation and telephonic system" and was "in no way a restatement of the SOW." The protester asserts that this section of its proposal also "includes a detailed account of the procedures for handling correspondence, the handling/processing of payments, the receipt/posting of daily fund transactions, [and] the policies and maintenance of reverse entries for payments." The evaluation in this area was reasonable. The proposal areas the protester cites are essentially procedural discussions of various contract activities, and do not appear to address the evaluated QC deficiencies. For example, while the cited proposal discussion does lay out procedures for handling correspondence, it does not address QC measurements, functions, or reporting mechanisms, as required by the RFP and evaluated by the agency as lacking. Similarly, although the proposal discusses a quality control report, it simply repeats verbatim the SOW, stating that it will be "[b]roken down by category, state size of sample, state numbers and types of problems found . . . ," and also merely repeats the SOW-required categories for the QC report. Our review confirms the agency's determination that the protester failed to describe what will be measured (within a category), the number of samplings, or how problems will be prevented.[5] We conclude that, notwithstanding McAllister's position that it provided substantial QC information in its proposal, substantial information still was omitted. As the protester has not alleged or shown that the omitted information was not material, there is no basis to object to the evaluation in this area.[6] Training Plan In this area, worth 40 points, the RFP notified offerors that proposals would be evaluated for corresponding SOW requirements as follows: (1) completeness of offeror's approach to train new staff in the treatment of student loan accounts placed for collection--15 points, (2) details of procedures for identifying ongoing training needs and accomplishing the required training--10 points, and (3) specifics on anticipated course content, length of training, and training coordinator's qualifications--15 points. McAllister's proposal received an average of 7.4 points, or 18.5 percent, in this area. The summary evaluation narrative lists four evaluated weaknesses as follows: (1) "[t]here was no discussion of the offeror's approach to train new staff in the treatment of student loan accounts placed for collection," (2) "[n]o discussion of procedures for identifying ongoing training needs," (3) "[t]raining outline does not . . . appear to relate specifically to this contract," and (4) "[d]id not address training coordinator(s), nor the qualifications." McAllister neither rebuts the specific evaluated deficiencies, nor disputes that they are material. Instead, McAllister relies on its past training experience and contends that the agency could not have reasonably concluded that the firm lacked an adequate training plan. In particular, the protester contends that its CEO, who is identified in the training plan section of the proposal as the "Assistant Training Manager," "has over five years experience in managing the [DOE] contract and training hundreds of individuals in the collections of defaulted student loans," and "has been responsible for the placement of the executive staff at [another offeror]," who received an award under the RFP here. This argument is without merit. As discussed above, an agency's technical evaluation generally is dependent upon the information furnished within the four corners of the proposal; the agency properly may rely on this information in reaching its evaluation conclusions. Our review confirms the agency's finding that the proposal omitted required information. For example, in the proposal's training plan section, there is no discussion of a training coordinator's qualifications, as called for by the RFP. The submitted resume for the firm's CEO lists his current position as "CEO/training & project director," but does not specify his training responsibilities and activities in his current position, or his training experience at three other listed firms. In a separate section of the proposal (project personnel experience), the protester's vice-president is identified as the "[t]raining [m]anager." (There is no mention of this individual in the training plan section of the proposal.) The submitted resume for this individual reveals that he is a medical doctor with a specialty in psychiatry and addictions, but there is no indication of his training qualifications relevant to the requirements here. (In a separate "Executive Summary Flow Chart" this individual is listed as having 10 years of "collection agency experience on [DOE] contracts," but no specific experience is listed.) While the proposal states that this individual "implements a 'pre-employment' psychological testing program," which "provides a measurement of an applicant's character, degree of discipline, credibility and reliability," experience with this type of testing clearly does not equate to training qualifications specific to the collections work to be performed under the contract, as was sought here. Based on this confusion in the proposal as to who would be the training coordinator (for the training specific to the RFP) and the lack of discussion of the relevant qualifications of a training coordinator, we have no basis to question the agency's determination that the proposal was informationally deficient in this area. We conclude that the agency reasonably found that McAllister's proposal omitted substantial material information, that the protester thus failed to demonstrate its capabilities and understanding of the requirements, and that correction of these failures would necessitate a major revision of the proposal. See Defense Group Inc., B-253795, Oct. 25, 1993, 94-1 CPD para. 196 at 6. Under these circumstances, the agency properly eliminated the proposal from the competitive range.[7] The protest is denied. Comptroller General of the United States 1. The average evaluated scores of McAllister's proposal by subfactor were as follows: (1) executive summary--4.4 out of 25 points or 17.6 percent, (2) strategic collection approach--23.4 out of 75 points or 31.2 percent, (3) administrative resolutions--7.6 out of 30 points or 25.3 percent, (4) litigation/AWG preparation--19 out of 45 points or 42.2 percent, (5) QC--19.2 out of 75 points or 25.6 percent, (6) training plan--7.4 out of 40 points or 18.5 percent, (7) computer system resources--20.4 out of 60 points or 34 percent, and (8) project personnel experience--30.2 out of 100 points or 30.2 percent. 2. The average total score of the nearest offeror included in the revised competitive range was 215.2 points, or 47.8 percent of the possible points. 3. The RFP provided that proposals would be evaluated under this subfactor as follows: (1) "[c]larity and completeness of the overall management approach to staff contract with capable personnel"--10 points; (2) "[d]iscussion of offeror's approach to utilize administrative support staff necessary to interface with [DOE] and support the resolution of accounts under this contract"--15 points, (3) "[q]ualifications of individuals proposed under key positions of contract, specifically: Contract Administrator, Collections Manager, Administrative/Support Manager and Systems Manager"--35 points, (4) "[c]ompleteness of resumes for proposed staff"--5 points, (6) "[d]iscussion of offeror's approach to maintain personnel in an environment where retention and loyalty is generally very low" --10 points, and (7) "[e]vidence of the offeror's commitment to high quality service and performance, as demonstrated by prior experience"--25 points. In this area, the RFP specifically required an "outline [of] all current contractual obligations in which key personnel are currently assigned, and the time currently devoted to those projects" and a matrix "that specifies the amount of time each staff member will be committed during the contract period." 4. The SOW further required a monthly QC report "[b]roken down by category, state size of sample, state numbers and type of problems found, corrective action taken and preventive action so that the same problem(s) do not happen in the future." The RFP also listed 14 required categories for the QC report, such as "[m]onitoring of collector/customer service calls" and "[c]orrespondence for proper handling/response." 5. In this regard, we note that the RFP included a sample QC report format in Appendix VIII, which was filled in with specific examples of "QC sample [i.e., the required QC categories]," "sample size," "errors found," "types of errors," "corrective action taken," and "preventive measures," which should have provided ample guidance to the protester as to the type of information the agency sought. 6. We also note that, of the 10 pages (i.e., pages 43 through 52) cited by the agency as being a reiteration of the RFP, the protester disputes the agency's position only as to 1 of those pages (page 43); it does not rebut the agency's position as to the other 9 pages (i.e., pages 44 through 52). Our review indicates that page 43 in fact contains a substantial reiteration of the SOW. For example, paragraph E.8, "Complaint Resolutions and [DOE] Monitoring," of the protester's proposal (which continues onto page 44) repeats almost verbatim a SOW paragraph under section 2.4, "Resolution of Complaints." The main difference is that the protester's proposal combines two sentences into one by deleting portions of both sentences. 7. The protester alleges unequal competition because another offeror with alleged significant proposal deficiencies was provided an opportunity to correct its deficiencies and subsequently was awarded a contract. However, our review of the two proposals and the evaluation documentation does not indicate that the two offerors were treated disparately in either of the two areas cited by the protester--hiring programs and interface with DOE under the second and fourth evaluation subfactors. Moreover, the other offeror's proposal clearly was situated differently than McAllister's, having received a composite technical score before discussions of 326.4 points, significantly higher than the protester's 131.6 points.