TITLE: OSI Collection Services, Inc., B-286597; B-286597.2, January 17, 2001
BNUMBER: B-286597; B-286597.2
DATE: January 17, 2001
**********************************************************************
OSI Collection Services, Inc., B-286597; B-286597.2, January 17, 2001
Decision
Matter of: OSI Collection Services, Inc.
File: B-286597; B-286597.2
Date: January 17, 2001
Joseph J. Petrillo, Esq., and Karen D. Powell, Esq., Petrillo & Powell, for
the protester.
Jeffrey C. Morhardt, Esq., and Jose Otero, Esq., Department of Education,
for the agency.
Tania Calhoun, Esq., and Christine S. Melody, Esq., Office of the General
Counsel, GAO, participated in the preparation of the decision.
DIGEST
Protests against award of federal supply schedule task order contracts for
private collection agency services are sustained where the record shows that
the contracting agency's evaluation of offerors' past performance, which
largely relied upon a mechanical comparison of past performance scores for
incumbent contractors, was unsupported and unreasonable.
DECISION
OSI Collection Services, Inc. protests the decision by the Department of
Education to award federal supply schedule (FSS) task order contracts to
numerous other firms under a request for task order proposal (RFTOP) for
private collection agency (PCA) services. OSI contends that the agency's
evaluation of offerors' past performance was inconsistent with the
solicitation's stated evaluation terms, unsupported, and unreasonable.
We sustain the protests.
BACKGROUND
The agency's Office of Student Financial Assistance performs collection and
administrative resolution activities on debts resulting from nonpayment of
student loans made under numerous federal programs, and on debts resulting
from a student's failure to fulfill grant requirements under other federal
programs. At the time this solicitation was issued, 17 PCA contractors,
including OSI, were performing these services for the agency under a
contract that commenced in 1997 and runs through September 2001.
The RFTOP was issued in July 2000 to obtain the services of PCAs with
contracts under the General Services Administration's Financial Asset
Management Services Schedule, Special Item Number (SIN) 621-4, "Loan and
Other Asset Services/Management." Task order contracts were to be issued to
10-12 FSS contractors, with at least two awarded under a small business
set-aside. The agency announced that it had about two million accounts,
worth about $9 billion, to place under this award. It planned to conduct an
initial transfer of 20,000 accounts to each successful contractor, each of
whom was to locate and contact the borrowers to demand payment of their
debts or to otherwise resolve the account through such measures as wage
garnishment, litigation, or other administrative resolutions. Additional
account transfers were to occur throughout the life of the contract.
Firms could compete for the task order contracts only if they were
specifically invited to do so. Incumbent contractors, such as OSI, were to
be invited to compete only if they had performed "consistently well" for the
agency based upon its Competitive Performance and Continuous Surveillance
(CPCS) evaluation. The CPCS evaluation, performed every 4 months, measures
the relative performance of each contractor on all accounts transferred
under various performance indicators and is used to determine bonus payments
and the transfer of new accounts. Under the CPCS methodology, the contractor
ranked the highest under a particular performance indicator receives the
maximum number of points available for that indicator, and the remaining
contractors receive points in proportion to their standing relative to the
leading contractor. Each contractor's overall CPCS score for each 4-month
period is the sum of its scores for all of the performance indicators for
that period.
Offerors were required to propose a commission or fee for each type of
service to be performed under the contract. Since the RFTOP established
target rates for these commissions or fees, "quality factors" and the
commitment to small business were to be more important than price in making
the award selection decision. Section M.1.b. of the RFTOP stated that
evaluation factors were to be considered in the following order of
importance: past performance, including the past performance of key
personnel; technical evaluation; commitment to small business; and price.
The past performance evaluation is the critical issue in these protests.
Section M.2. of the RFTOP provided that the following past performance
information was to be obtained and considered: a Dun & Bradstreet (D&B) past
performance evaluation for all offerors; information obtained when checking
references for all offerors; and, "[f]or those companies with a current
contract, the Department will use performance data that we have on hand such
as the CPCS scores." Recent and relevant information was to receive greater
consideration than less recent and less relevant information. The agency
considered competitive ranking information to be "extremely relevant." RFTOP
M.2.
Twenty-six FSS contractors submitted proposals in response to the RFTOP.
Thirteen contractors were large businesses currently performing PCA services
for the agency, including OSI. Of the other 13 FSS contractors, 7 were
non-incumbents and 6 were small businesses invited to compete for award
under the small business set-aside portion of the solicitation.
A source evaluation board (SEB) met to consider offerors' proposals. The SEB
considered three elements in evaluating past performance. First, for each
offeror, the SEB reviewed the results of a D&B past performance evaluation.
These results were in the form of numerical scores on a scale of 1-5, with 1
being "outstanding" and 5 "unsatisfactory". Second, for each offeror, the
SEB reviewed the results of interviews conducted with contractor-provided
references. These results included subjective comments along with numerical
scores on a scale of 1-4, with 1 being "extremely satisfied" and 4 "never
satisfied." Third, for incumbent contractors, the SEB considered the CPCS
score attained over the life of the contract--the arithmetic total of the
seven periodic CPCS scores available at that time. Based upon that total
score, the SEB placed the firms in a first or a second CPCS "tier." The
second tier was defined as containing firms with an overall performance of
average or below. At the conclusion of the evaluation, the SEB assigned each
PCA an overall rating of a low, moderate, or high probability of success
based on the past performance, technical, and small business commitment
evaluations. [1]
OSI's overall D&B rating was [DELETED], placing its performance between very
good and outstanding. The firm's overall reference rating was [DELETED],
with two of its three references responding to the agency's inquiries. [2]
The evaluators noted that OSI's clients were extremely satisfied with the
firm's performance. With respect to the CPCS evaluation, the SEB concluded
that OSI was in the second tier of the 17 PCAs. In making its overall
assessment, the SEB stated that OSI demonstrated a thorough knowledge of
student loan collections--its technical proposal was evaluated as having a
high probability of success. [3] The SEB concluded, however, that the most
relevant indicator of success was the CPCS evaluation. According to the SEB,
while "net back recovery percentages" were important, contractors had to
perform the full range of requirements. [4] The SEB determined that OSI's
performance on the CPCS placed it in the second tier--the lowest five
rankings of current PCAs invited to compete. The SEB stated that the
information obtained from OSI's references indicated that its performance on
other contracts was very good to outstanding, but that the firm's
performance on the current contract could not be discounted. The SEB
determined that OSI's proposal merited an overall rating of a moderate
probability of success.
The SEB submitted its award recommendation memorandum to the contracting
officer, who served as the source selection authority (SSA). The memorandum
divided the offerors into six groups. First, six incumbent offerors were
recommended for award: Pioneer Credit Recovery, Inc., Van Ru Credit
Corporation, National Asset Management Enterprises, Inc. (NAM), Diversified
Collection Services, Inc. (DCS), Financial Asset Management Systems, Inc.
(FAM), and Aman Collection Services, Inc. The SEB stated that these firms
were the top six performers based on CPCS rankings which, when combined with
their D&B and reference ratings, gave them high past performance ratings;
they also received high technical ratings. Second, three non-incumbent
offerors were recommended for award: USA Education Group, Recovery Bureau of
America (RBA), and Maximus. The SEB stated that while no CPCS rankings were
available for these firms, they had very good to outstanding past
performance based upon their D&B and reference ratings and received high
technical ratings. Third, two incumbent offerors were recommended for award:
NCO Financial Systems, Inc., and Nationwide Credit, Inc. (NCI). The SEB
stated that, while these firms were clearly ranked below the nine
recommended firms, their overall good past performance rating and high
technical rating made both reasonable, although marginal, candidates for
selection; their significant commitments to small business made it worth
recommending both for award. Fourth, two small business firms were
recommended for award under the small business set-aside. Fifth, five
incumbent offerors were not recommended for award, including OSI. The SEB
stated that these firms had lower CPCS scores than the incumbents
recommended for award and that these lower CPCS scores, when considered with
the D&B and reference scores, combined to produce significantly lower past
performance rankings for these firms than for the incumbents recommended for
award. [5] The memorandum also shows that the SEB considered the most
recent--eighth--set of CPCS scores for some of the incumbent offerors to
ascertain upward or downward trends, but determined that inclusion of this
set of scores would not have affected its award recommendations.
The SSA concurred with the SEB's recommendations. The final relevant
evaluation results for the full and open competition, with the proposed
awardees listed in italics, were as follows:
Past Performance Technical Overall
D&B References CPCS
Pioneer [DELETED] [DELETED] 493 High High
VanRu [DELETED] [DELETED] 481 High High
NAM [DELETED] [DELETED] 507 High High
DCS [DELETED] [DELETED] 458 High High
FAM [DELETED] [DELETED] 450 High High
Aman [DELETED] [DELETED] 451 High High
USA-ED [DELETED] [DELETED] N/A High High
RBA [DELETED] [DELETED] N/A High High
Maximus [DELETED] [DELETED] N/A High High
NCO [DELETED] [DELETED] 433 High High
NCI [DELETED] [DELETED] 431 High High
Firm A [DELETED] [DELETED] 396 Moderate Moderate
Firm B [DELETED] [DELETED] 388 Moderate Low
Firm C [DELETED] [DELETED] 403 Moderate Moderate
Firm D [DELETED] [DELETED] 391 Moderate Moderate
OSI [DELETED] [DELETED] 384 High Moderate
The SSA independently examined the eighth set of CPCS scores to see whether
factoring them in would have an effect on the award decisions and concluded
that there would be no such effect. [6] He also used a formula and a
weighted average to combine the CPCS total scores, the D&B scores, and the
reference scores into a single past performance score for each offeror. He
concluded that the results of this combined past performance score supported
the SEB's findings.
OSI filed its initial protest in our Office after its debriefing. The agency
subsequently determined that performance of the task order contracts was in
the best interest of the United States and that "circumstances exist[ed]
which would not permit waiting" for our decision, and executed an override
of the statutory stay of performance of these contracts. See 31 U.S.C. sect.
3553(d)(3)(C)(i) (1994). OSI supplemented its protest after receiving the
agency report. OSI alleges that the agency improperly limited its
consideration of performance data under the current contract to the CPCS
scores, in contravention of the solicitation's terms, and that the agency's
reliance on the CPCS scores alone was also unreasonable. OSI also alleges
that the agency's selection of the non-incumbent firms' proposals in
preference to its own lacked a reasonably supported basis.
DISCUSSION
Under the Federal Supply Schedule (FSS) program, agencies are not required
to conduct a competition before using their business judgment in determining
whether ordering supplies or services from an FSS vendor represents the best
value and meets the agency's needs at the lowest overall cost. Federal
Acquisition Regulation sect. 8.404(a); Amdahl Corp., B-281255, Dec. 28, 1998,
98-2 CPD para. 161 at 3. However, when the agency decides to conduct a formal
competition for award of a task order contract, we will review the agency's
actions to ensure that the evaluation was reasonable and consistent with the
terms of the solicitation. Computer Prods., Inc., B-284702, May 24, 2000,
2000 CPD para. 95 at 4-5; COMARK Fed. Sys., B-278343;
B-278343.2, Jan. 20, 1998, 98-1 CPD para. 34 at 4-5. Our review of the record
here shows that the agency's past performance evaluation was both
inconsistent with the solicitation's terms and unreasonable.
Initial Protest
The principal issue raised by OSI's initial protest concerns the agency's
near-exclusive reliance on the summary CPCS scores. The agency correctly
asserts that it is vested with the discretion to determine the type of past
performance data from among the various data on hand which was most relevant
and reliable in conducting the past performance evaluation. Evaluation of an
offeror's past performance is a matter within the discretion of the
contracting agency, and we will not substitute our judgment for reasonably
based past performance ratings. However, we will question such conclusions
where they are not reasonably based or are undocumented. Green Valley
Transp., Inc., B-285283, Aug. 9, 2000, 2000 CPD para. 133 at 4; see also
Future-Tec Management Sys., Inc.; Computer & Hi-Tech Management, Inc.,
B-283793.5, B-283793.6, Mar. 20, 2000, 2000 CPD para. 59 at 7. The critical
question here is whether the evaluation was conducted fairly, reasonably,
and in accordance with the stated evaluation terms, and whether it is based
upon relevant information sufficient to make a reasonable determination of
the offeror's overall past performance rating, including relevant
information close at hand or known by the contracting personnel awarding the
contract. See U.S. Tech. Corp., B-278584, Feb. 17, 1998, 98-1 CPD para. 78 at 6;
International Bus. Sys., Inc., B-275554, Mar. 3, 1997, 97-1 CPD para. 114 at 5.
Since the CPCS scores are central to this protest, we set out their basis in
further detail here. As discussed above, the CPCS evaluation, performed
every 4 months, measures the relative performance of each PCA contractor on
all accounts transferred. The current contract provides that the CPCS
evaluation will gauge the performance of each contractor under four
performance indicators. [7] The most important indicator, net back recovery,
is the net amount of money recovered by the government when defaulted loans
are placed in repayment. The indicator is measured as a ratio of the net
dollars collected over the average current inventory balance--the average
dollar value of the accounts--held by the contractor. The remaining three
indicators are, respectively, the number of accounts placed for
administrative wage garnishments, the number of litigation packages
prepared, and the number of accounts prepared and returned for non-cash,
administrative resolution. Each of these indicators is measured as a ratio
of the number of accounts resolved (by administrative wage garnishment,
litigation, or administrative resolution, respectively) over the average
current inventory of accounts--the average number of accounts--held by the
contractor. For each 4-month period, the contractor rated highest under each
indicator receives the maximum number of points assigned to that indicator
and the remaining contractors receive points in proportion to their standing
relative to the leading contractor. [8] Each contractor's periodic CPCS
score is the sum of its scores for all four performance indicators for that
period. To obtain the overall ("cumulative") score, the agency simply added
together each contractor's scores for each period.
The record shows that these cumulative scores were virtually the only
performance data the agency used in evaluating the performance under the
current contracts by the offerors who are incumbent contractors. The
individual evaluation documents for each firm contain virtually identical
narrative regarding the CPCS portion of the evaluation, and little of that
narrative concerns the actual quality of the individual offeror's past
performance. Instead, the narrative assigns each contractor to a first or
second tier based upon its CPCS scores; characterizes the 8 "first tier"
offerors as "consistent top performer[s];" and characterizes the 5 "second
tier" offerors as having overall performance of average or below. The SEB's
award recommendation memorandum repeats these general statements, merely
adding notations regarding whether some firms' most recent CPCS rankings
trended upward or downward and whether this had an impact on their overall
CPCS rankings. The SSA's decision document adopts the SEB's recommendations
without further substantive analysis. [9]
In his post-protest affidavit, the SEB chairperson states that the SEB "knew
how the CPCS rankings were derived and . . . the meaning ‘behind the
numbers,'" considered the data generated by the CPCS system, and considered
the CPCS data for each of eight ranking periods. SEB Chairperson's Affidavit
para.para. 3, 6. The contemporaneous evaluation documentation does not support these
statements. Rather, the contemporaneous evaluation documentation shows that
the CPCS aspect of the agency's past performance evaluation contained
virtually no analysis of individual offerors' past performance, and that the
agency limited its consideration of the performance data on hand to ranking
the incumbents based upon the arithmetic total of each firm's seven periodic
CPCS scores and reviewing whether some firms' eighth periodic CPCS score
reflected an upward or downward trend in performance. To the extent the
agency performed any qualitative analysis, it is not documented. For our
Office to perform a meaningful review of an agency's selection
determination, however, the agency must have adequate documentation to
support its evaluation of proposals and its selection decision. Green Valley
Transp., Inc., supra, at 8; Future-Tec Management Sys., Inc.; Computer &
Hi-Tech Management, Inc., supra.
Here, we are faced with a lack of contemporaneous evidence that the agency
considered performance data relevant to the offerors' performance under the
current contracts other than the cumulative CPCS numbers, while the
solicitation committed to offerors that, "[f]or those companies with a
current contract, the Department will use performance data that we have on
hand such as the CPCS scores." It is true that the solicitation stated that
competitive ranking information (such as the CPCS scores) was extremely
relevant, but the agency had other relevant performance data it could have
considered. Given the relevance of that data and the limitations of the
cumulative CPCS scores, we agree with the protester that the agency's
near-exclusive reliance on the cumulative CPCS scores was unreasonable.
Our principal concern stems from the agency's overly mechanical application
of the cumulative CPCS scores to evaluate the past performance of incumbent
contractors. We recognize that the agency constructed the CPCS system in a
way intended to capture various aspects of contractors' performance through
the CPCS scores. Point scores can, however, only be aids in decision-making,
and they must be used in a defensible way. In this case, the nature of the
underlying data meant that the cumulative CPCS scores could not be relied on
mechanically to assess offerors' past performance.
The CPCS methodology set forth in the current contracts envisioned not a
cumulative evaluation over the life of the contract, but a series of
discrete periodic evaluations of performance. In defending the use of
cumulative scores, the SSA states that the agency needs PCAs that will
perform well consistently over the life of the contract, not just during a
4-month period. SSA's Affidavit para. 6.
We agree that considering one CPCS period in isolation would not give an
actual picture of the overall performance of the contractors overtime. In
fact, review of the CPCS scores for the individual periods demonstrates
another flaw in the agency's reliance on cumulative scores. The data shows
that the performance of most PCA contractors, including those in the SEB's
first tier, varied, often widely, from period to period. For example, the
rankings of Pioneer over the eight CPCS periods were 9th, 7th, 1st, 1st,
10th, 11th, 2nd and 1st; the rankings of FAM were 3rd, 11th, 10th, 13th,
11th, 2nd, 4th and 10th; and the rankings of Aman were 5th, 6th, 14th, 15th,
5th, 1st, 6th and 14th. [10] These disparities demonstrate the
unreasonableness of the SEB's characterization of these first tier
contractors as "consistent top performers," and reinforce the protester's
point that an examination of each of the CPCS periodic scores, and the
underlying data, should have been a component of the past performance
evaluation.
That point is given further support when the impact of shifting account
inventories on CPCS scores is considered. Contractors' periodic CPCS scores
were based upon variable numbers of accounts, and those numbers also varied,
often dramatically, from period to period. It was not unusual for the
average number of accounts held by contractors to vary by more than 100,000
within a CPCS period, or for the average number of accounts held by one
contractor to fluctuate by as many as 50,000 accounts from period to period.
In this regard, contractors began the first CPCS period on relatively equal
footing: each contractor was transferred 20,000 accounts with roughly the
same average balances. However, one purpose of the CPCS evaluation was to
determine the number of accounts contractors might receive in the next CPCS
period; the higher a contractor's CPCS score in one period, the more
accounts might be subsequently transferred to that contractor. The agency
acknowledged during a bidder's conference that this mechanism meant that it
could be difficult for top performers to remain on top because they might
have large transfers of accounts during the next period. First Bidder's
Conference Transcript at 62-63. Conversely, accounts that remained
unresolved after 9 months were required to be returned to the agency for
reassignment to other contractors, and contractors could return unresolved
accounts earlier than that provided they had engaged in a minimal level of
effort to resolve the accounts. This minimal level of effort was defined
differently in each PCA contractor's contract, which adds a further
difficulty in comparing contractors' scores. As the agency acknowledged,
contractors could "manipulate" the size of their inventory to move up in the
rankings. Id. at 62.
The record contains additional evidence of a relationship between the
average number of accounts held by a contractor and its CPCS scores. Our
review of the data shows a frequent correlation between a decreased volume
of accounts and an increased percentage of account resolutions, and vice
versa. For example, between the 5th and 6th CPCS periods, Pioneer's average
number of accounts increased by approximately 37,000 and all of its
resolution rates decreased. Between the 6th and 7th CPCS periods, the firm's
average number of accounts decreased by approximately 76,000 and all of its
resolution rates increased substantially. Again, the CPCS methodology
provides that higher account resolution rates generally result in higher
CPCS scores, and lower account resolution rates generally result in lower
CPCS scores.
The protester does not contend, as the agency states, that the CPCS rankings
are unfair because there are differences in the number of accounts
transferred to contractors during a period or that the SEB should have
adjusted or disregarded CPCS scores as a result. Rather, the protester
asserts, and we agree, that the differing workloads of contractors ought to
have been considered when assessing their actual past performance. As an
example of what might be considered, the protester points out that its
having received a large number of accounts in the second CPCS period as a
result of its success in the first CPCS period presented particular
challenges associated with the need to provide staffing early in performance
to handle those newly assigned accounts.
While the indicators used in calculating CPCS scores for a specific period
factored in the differences in inventory size for that period, we conclude
that it was unreasonable for the agency to rely as exclusively as it did on
the cumulative CPSC scores. Given the significantly different account
inventories held by the contractors both within one CPCS period and from
period to period, we view it as irrational to focus only on the CPCS scores
without examining the circumstances and the performance data in the agency's
possession to reach a considered judgment regarding the quality of an
offeror's performance and to make comparisons between offerors' performance.
[11] Green Valley Transp., Inc., supra, at 6-7. We recognize that reducing
the past performance of contractors to a single score might result in a more
streamlined evaluation, but the use of such a technique is no substitute for
the reasoned judgment of evaluators in examining and comparing the actual
past performance of offerors. [12]
Supplemental Protest
In its supplemental protest, OSI contends that the agency awarded at least
one task order contract to a non-incumbent offeror whose evaluation was, in
every comparable category, inferior to OSI's. Specifically, the protester
correctly asserts that the proposals of both OSI and Maximus received high
technical ratings; that OSI had far more years of experience than Maximus;
and that OSI was rated higher than Maximus under the key personnel component
of the technical evaluation factor, the D&B ratings, and the reference
ratings. The only factor the firms did not have in common was incumbent
performance. OSI was an incumbent contractor with CPCS scores and Maximus
had no experience with PCA contracts. OSI contends that its incumbent
performance should have been a positive factor in its favor but that,
notwithstanding its proposal's status as equal or superior to that of
Maximus, the record contains no indication why Maximus was selected over
OSI. [13]
The agency's only response to this allegation is to repeat the information
contained in the individual evaluations of these offerors and to assert that
OSI is merely disagreeing with those individual evaluations. However, OSI is
not challenging these individual evaluations (save its own CPCS evaluation)
but, rather, is correctly asserting that the record is devoid of any
comparison between OSI's proposal and the proposals of the non-incumbent
awardees, notwithstanding the fact that the evaluation documentation shows
that its proposal is equal or superior to at least one of these proposals.
Again, an agency that fails to adequately document its source selection
decision in sufficient detail to show that it is not arbitrary bears the
risk that our Office will not conclude that the agency had a reasonable
basis for its determinations. Future-Tec Management Sys., Inc.; Computer &
Hi-Tech Management, Inc., supra, at 7. The record here leads us to conclude
that the source selection in this regard was unsupported and unreasonable.
Prejudice and Recommendation
Our Office will not sustain a protest unless the protester demonstrates a
reasonable possibility that it was prejudiced by the agency's actions, that
is, unless the protester demonstrates that, but for the agency's actions, it
would have had a substantial chance of receiving the award.
McDonald-Bradley, B-270126, Feb. 8, 1996, 96-1 CPD para. 54 at 3; see
Statistica, Inc. v. Christopher, 102 F.3d 1577, 1581 (Fed. Cir. 1996). We
believe that OSI has met this standard.
It is undisputed that more weight was to be accorded more recent competitive
ranking data and that OSI's most recent CPCS ranking trended downward, and
it is undisputed that OSI's performance was erratic over the course of the
contract based upon its CPCS rankings. However, the agency's reliance on
these rankings to argue that OSI has not been prejudiced perpetuates the
central flaw in this past performance evaluation: the agency's failure to
review relevant performance data in its possession and to make reasoned
judgments as to its value to arrive at an accurate assessment of each
offeror's past performance. Since the agency's past performance evaluation
did not comply with the stated evaluation terms; the CPCS scores alone are
an inadequate indicator of overall past performance without consideration of
the circumstances and underlying performance data; the agency has not
adequately supported its past performance evaluation of offerors, both on an
individual and a relative basis; and OSI has otherwise very high scores and
a positive evaluation, we conclude that the source selection cannot be
viewed as reasonable and that the protester was prejudiced by the agency's
actions.
We recommend that the agency reevaluate the proposals with respect to past
performance, giving appropriate consideration and weight to the performance
data in its possession. If the reevaluation shows that the previously made
award decisions should be revised, the agency should terminate the
appropriate task order contracts and make award to OSI or other appropriate
offerors. We also recommend that the protester be reimbursed the reasonable
costs of filing and pursuing the
protests, including attorneys' fees. 4 C.F.R. sect. 21.8(d)(1) (2000). The
protester should
submit its certified claim for such costs, detailing the time expended and
the costs incurred, directly to the contracting agency within 60 days after
receipt of this decision.
The protests are sustained.
Anthony H. Gamboa
Acting General Counsel
Notes
1. The agency evaluated each technical proposal as representing a low,
moderate, or high probability of success, and evaluated small business
participation proposals based upon how close offerors came to the RFTOP's
specified targets.
2. OSI argues that the agency improperly failed to secure a response from
its third reference, but there is generally no legal requirement that all
references listed in a proposal be checked. Advanced Data Concepts, Inc.,
B-277801.4, June 1, 1998, 98-1 CPD para. 145 at 10.
3. OSI's proposal also met the RFTOP's small business participation and
pricing targets.
4. "Net back recovery percentage," the most important CPCS performance
indicator under the current contract, measures the net proportion of dollars
collected from the contractor's inventory of accounts.
5. The sixth group was comprised of eight non-incumbent offerors not
recommended for award.
6. The SSA also experimented with the CPCS data using weighted average
formulas to give the most recent past performance more weight, in accordance
with the RFTOP instructions. He concluded there would be an effect on the
award recommendations only if he used a highly skewed weighting system,
which he deemed unreasonable.
7. The current contract initially included two additional performance
indicators that were eliminated by contract amendments in 1998.
8. The effect of this methodology was that a contractor's score reflected
not simply its own performance, but also how well other contractors
performed during the same period. Thus, a contractor could receive different
scores for the same level of performance in two different periods.
9. The SSA states that the single past performance score he put together
from the D&B scores, reference scores, and CPCS scores was used only as part
of his post-recommendation analysis to confirm that the SEB's
recommendations were appropriate. Regardless of its purpose, for the reasons
discussed below in the context of the CPCS scores, the SSA's use of this
technique was overly mechanical and the formula he used to arrive at his
single past performance score suffered from several errors pointed out by
the protester.
10. These ratings were generally based on a universe of 17 PCA contractors.
11. Since the CPCS performance indicators contemplated by this RFTOP differ
from the indicators previously used, OSI contends that the agency should
have considered whether certain aspects of performance should be weighted
more than others in order to more accurately predict future good
performance. Our review of the two sets of indicators and their methodology
shows that they are not materially different, but the agency may wish to
consider whether it deems certain aspects of past performance more valuable
than others in reevaluating proposals.
12. The RFTOP also required the agency to evaluate "key personnel" for all
offerors under both the past performance and technical factors. The record
shows the SEB considered the experience of key personnel as part of the
technical evaluation of all proposals, but did not consider this experience
in the past performance evaluation where CPCS data was available. The agency
states that if the key personnel proposed were also involved in the past
performance of a PCA contract for that offeror, their relative success or
failure in the performance of that contract would be reflected in the CPCS
scores. To the extent the agency failed to evaluate key personnel in the
past performance evaluation of all offerors, that was inconsistent with the
RFTOP's terms. To the extent the agency factored key personnel into the past
performance evaluations of incumbent offerors, it is not apparent from the
evaluation documentation and should be addressed in the reevaluation.
13. OSI makes these same arguments with respect to the two other
non-incumbent awardees. In addition, in its supplemental protest, OSI argues
that the rating given to one of these firms for its references was suspect
because two of the three references appeared to be from related firms. The
agency's supplemental report fails to address this allegation, which appears
to be well-founded, and the agency should review the matter as part of its
reevaluation of past performance.