TITLE: B-298833.4; B-298833.5, IBM Global Business Services, March 1, 2007
BNUMBER: B-298833.4; B-298833.5
DATE: March 1, 2007
*******************************************************************
B-298833.4; B-298833.5, IBM Global Business Services, March 1, 2007

   DOCUMENT FOR PUBLIC RELEASE
   The decision issued on the date below was subject to a GAO Protective
   Order. This redacted version has been approved for public release.

   Decision

   Matter of: IBM Global Business Services

   File: B-298833.4; B-298833.5

   Date: March 1, 2007

   Thomas Humphrey, Esq., John E. McCarthy, Jr., Esq., David Z. Bodenhiemer,
   Esq., Rebecca K. Lee, Esq., Steven H. Talkovsky, Esq., and Amy Laderberg
   O'Sullivan, Esq., Crowell & Moring; Todd Hutchen, Esq., IBM Global
   Business Services, for the protester.

   Carl J. Peckinpaugh, Esq., and Helaine G. Elderkin, Esq., Computer
   Sciences Corporation, for the intervenor.

   Brent Curtis, Esq., Richard C. Bean, Esq., Maj. Angela L. Penny, and Maj.
   Erik A. Troff, Department of the Air Force, for the agency.

   David A. Ashen, Esq., and John M. Melody, Esq., Office of the General
   Counsel, GAO, participated in the preparation of the decision.

   DIGEST

   In procurement for system integrator to deploy commercial off-the-shelf
   software product suite and change management techniques, quotation was
   reasonably evaluated as high risk where vendor intended to involve too few
   government personnel in change management process to accomplish important
   change management activities; protester's defense of its ratio of
   government personnel to contractor consultants is undercut by fact that
   ratio was significantly lower than that in typical commercial
   implementations.

   DECISION

   IBM Global Business Services protests the Department of the Air Force's
   issuance of a task order to Computer Sciences Corporation (CSC), under
   request for quotations (RFQ) No. FA8770-05-Q-0022, to serve as the
   Expeditionary Combat Support System (ECSS) System Integrator (SI). IBM
   challenges the terms of the solicitation and the evaluation of quotations.

   We deny the protest.

   The RFQ was issued, in accordance with Federal Acquisition Regulation
   subpart 8.4, to vendors holding Federal Supply Schedule Department of
   Defense Enterprise Resource Planning (ERP) Systems Integration Services
   Blanket Purchase Agreements. The RFQ called for an SI to deploy a
   commercial off-the-shelf (COTS) software product suite and change
   management techniques, which would replace legacy logistics systems and
   antiquated business processes. The SI is required to configure, integrate
   and implement the COTS software, as well as provide leadership and
   resources to redesign and transform Air Force logistics operations so as
   to improve logistics results and reduce support costs. The RFQ generally
   contemplated a phased approach. The initial phase encompasses preparing
   the organization for change and defining how the selected COTS product
   suite will be configured to meet the agency's requirements, which will
   include such activities as: (1) change management, that is, the process by
   which executives, middle managers, and end users are prepared to accept
   changes in business processes; (2) blueprinting, that is, creating
   descriptions of the Air Force's business processes and determining how the
   COTS product suite will support the agency's business needs; and
   (3) continuing efforts of "Pathfinders" to prototype COTS implementations
   and to prove that they can work in an Air Force logistics environment. In
   the second phase, the SI will undertake implementation of ECSS, performing
   all work necessary to configure and test the ECSS solution in preparation
   for deployment; in the third phase, the SI will undertake deployment of
   ECSS into a live production operation; and in the fourth phase, the SI
   will transition ECSS into Air Force managed sustainment. ECSS Request for
   Quote sections 3, 4, 5; ECSS Task Descriptions; ECSS Statement of
   Objectives (SOO); Agency Comments, Feb. 10, 2007, at 11 n.10, 41 n.30, 57.

   Award was to be made to the vendor whose quotation represented the "best
   value" to the government based on three evaluation factors: (1) mission
   capability/quote risk (including subfactors for program execution, change
   management, and program management); (2) past performance; and (3) price
   (comprised of the value of the task order plus the value of any
   government-furnished personnel and property). Under each mission
   capability subfactor, quotations were to be rated X+ for exceeding
   minimum performance or capability objectives in a beneficial way, X for
   meeting minimum objectives, X- for not clearly meeting minimum
   objectives, or F for failing to meet minimum objectives. In addition,
   quote risk ratings were assigned at the subfactor level. Mission
   capability/quote risk and past performance, combined, were significantly
   more important than price.

   CSC and IBM submitted quotations. After conducting extensive discussions
   with the vendors, the agency requested final quotation revisions (FQR). On
   September 8, 2006, the Air Force issued a task order to CSC on the basis
   that its quotation represented the best value to the government. IBM
   thereupon filed several protests with our Office challenging the award
   (B-298833, B-29883.2, B-298833.3). The agency subsequently determined to
   take corrective action, including reevaluation of quotations, and we
   therefore dismissed the protests. After reevaluating quotations, the
   agency determined that CSC's quotation was more advantageous than IBM's
   under each of the evaluation factors, and remained the best value. The
   ratings were as follows:

   +------------------------------------------------------------------------+
   |                                  |IBM           |CSC                   |
   |----------------------------------+--------------+----------------------|
   |Mission Capability                |              |                      |
   |----------------------------------+--------------+----------------------|
   |Program Execution                 |X /low risk   |X /moderate risk     |
   |----------------------------------+--------------+----------------------|
   |Change Management                 |X /high risk  |X+ /low risk         |
   |----------------------------------+--------------+----------------------|
   |Program Management                |X /low risk   |X+ /low risk         |
   |----------------------------------+--------------+----------------------|
   |Past Performance                  |Confidence    |Significant Confidence|
   |----------------------------------+--------------+----------------------|
   |Evaluated Price                   |              |                      |
   |----------------------------------+--------------+----------------------|
   |Task Order Value                  |$747.0 million|$627.8 million        |
   |----------------------------------+--------------+----------------------|
   |Value Government Furnished        |$51.1 million |$118.9 million        |
   |Personnel                         |              |                      |
   |----------------------------------+--------------+----------------------|
   |Value Government-Furnished        |$0 million    |$5.0 million          |
   |Property                          |              |                      |
   |----------------------------------+--------------+----------------------|
   |Total Price                       |$798.1 million|$751.7 million        |
   +------------------------------------------------------------------------+

   CSC's quotation was rated X+ under the change management and program
   management subfactors for exceeding ECSS objectives, while IBM's quotation
   was rated only X. Further, IBM's approach to change management was rated
   as high risk due to IBM's request for an insufficient number of government
   personnel to be furnished to support its approach; this was found likely
   to cause significant disruption of schedule, significant increased cost,
   and significant degradation of performance. Source Selection Decision
   (SSD) at 4-5.

   Both quotations were rated X under the program execution subfactor. In
   this regard, IBM planned to initially deploy ECSS to agency Air Logistic
   Centers, while CSC planned to initially deploy ECSS to Air Force bases.
   IBM's quotation was evaluated as exceeding ECSS objectives by offering
   full operational capability 4 months earlier than specified in the
   notional solicitation schedule (and 5 months earlier than CSC), which
   would potentially accelerate cost savings from retiring legacy systems and
   reducing inventory. CSC's "bases first" approach of deploying a complete
   set of process functionality to end-users at each base also was favorably
   evaluated on the basis that it would make ECSS available to 70 percent of
   the end users up front and (unlike under IBM's approach) in a single
   deployment per site, which would eliminate the need for dual transactions
   by users.[1]

   As for past performance, the agency assigned CSC's quotation a rating of
   significant confidence and IBM's a rating of only confidence, on the basis
   that CSC had better demonstrated the capability to develop and deploy
   large-scale, highly complex COTS/ERP integrated business solutions
   comparable to the magnitude and complexity of the ECSS. On the other hand,
   the agency found that IBM had demonstrated the skills and capabilities
   necessary to transform small-to-medium-scale business operations using
   COTS/ERP applications and industry best practice, but determined that IBM
   (unlike CSC) had not demonstrated the capability to deploy large-scale,
   highly complex COTS/ERP solutions reasonably comparable to the magnitude
   and complexity of the ECSS.

   Given CSC's advantages under the non-price factors and its lower evaluated
   price, the Air Force again made award to CSC. IBM thereupon filed this
   protest.

   In reviewing protests against allegedly improper evaluations, it is not
   our role to reevaluate proposals. Rather, our Office examines the record
   to determine whether the agency's judgment was reasonable and in accord
   with the RFP criteria and applicable procurement statutes and regulations.
   See Rolf Jensen & Assocs., Inc., B-289475.2, B-289475.3, July 1, 2002,
   2002 CPD para. 110 at 5. Here, based on our review of all of IBM's timely
   arguments, we find no basis for questioning the award decision. We discuss
   several of IBM's arguments below.

   GOVERNMENT-FURNISHED PERSONNEL

   IBM raises various arguments concerning the role government-furnished
   personnel were to play in the SI contractor's efforts under the ECSS
   program. In this regard, the RFQ made clear the agency's view that a
   successful implementation of ECSS would require a partnership between the
   agency and the SI. According to the RFQ, "[t]he ECSS COTS Product Suite
   represents industry best practices which, when combined with the
   experience and knowledge of the Air Force Logistics Community, ECSS
   project team, and Systems Integrator (SI), will provide the best
   opportunity to achieve the objective as described in the RFQ documents."
   RFQ, Best Value Selection Factors, sect. 4.1. Likewise, the RFQ (as
   amended after the receipt of initial quotations) indicated that,
   consistent with the "governing umbrella philosophy of Enterprise Resource
   Planning System Integration Services[,] . . . the parties anticipate a
   unique sharing of concerns, knowledge and ideas via a partnership whereby
   Government personnel are made available to provide a source of
   information, and to work side-by-side with the contractor." RFQ amend. 5.

   In order to account for vendors' varying approaches to making use of
   government personnel in designing and implementing ECSS, the RFQ required
   vendors to complete a Government Personnel Matrix, to be incorporated into
   the awarded order as attachment 14, entitled "Government Personnel";
   vendors were to specify by contract year the number of government
   personnel in each of three experience levels (senior, "Mid," and junior)
   and the percentage of their time that would be required under the vendor's
   proposed approach for each ECSS activity (e.g., program management, change
   management, business blueprinting). According to the RFQ, attachment 14
   "defines the maximum amount of Government Personnel available, by skill
   level, per contract year." RFQ amend. 5. For each category of government
   personnel for each contract year, the matrix specified a cost per
   full-time equivalent (FTE) that would be added to the vendor's quoted
   price for purposes of the evaluation in order to "eliminate any
   competitive advantage resulting from the vendor's proposed use of
   Government Personnel." RFQ amend. 4; RFQ, Best Value Selection Factors,
   attach. 14.

   The RFQ further provided that "[i]t is incumbent upon the Government to
   make the Government Personnel available [in accordance with] . . .
   Attachment 14, Government Personnel"; according to the RFQ, the contractor
   "shall be entitled to an equitable adjustment should the Government fail
   to make the required personnel available [in accordance with] . . .
   Attachment 14." RFQ amend. 5. However, the RFQ also provided that the
   contractor's request for government personnel shall be filed so as to
   allow a lead time of 60 days in order to ensure availability, and that

     [b]ecause Government resources availability varies by location, the
     request shall allow flexibility in scheduling. Flexibility may require
     allowance for alternate dates of availability within the same contract
     year, a variance in labor mix or other variances.

   Id. In addition, the government reserved the "right to unilaterally
   substitute qualified Advisory and Assistance Services (A&AS) contractors,
   with similar skill levels, for Government Personnel." Id.

   The RFQ further provided for government personnel to be available to
   support the contractor's ECSS effort, stating as follows:

     The Government recognizes that Subject Matter Experts (SMEs) are
     essential to the success of the ECSS program. Key functional SMEs from
     across the Air Force Logistics Enterprise will be integrated into the
     PMO [Program Management Office]. The Government anticipates that the
     SMEs' team will be comprised of approximately three (3) GS-15 or O-6
     personnel, twelve (12) GS-14 or O-5 personnel, and twenty (20) GS-13 or
     O-4 personnel. At appropriate times these SMEs will be augmented with
     additional functional experts from Air Staff and the MAJCOMs.

   Statement of Objectives sect. 4.1.2. Vendors were to exclude from
   attachment 14, and thus from the price evaluation, government personnel to
   be furnished in support of its ECSS effort who were from the ECSS PMO or
   from the Logistics Transformation Office (LTO). Attach. 13.

   CSC listed in its attachment 14 a total of 1,202.6 government FTEs, spread
   over 7 years, that would be required for its ECSS effort; the agency added
   an adjustment of $118.9 million (at the rates specified by the agency in
   attachment 14) to its evaluated price to reflect the cost of providing
   these government personnel. In addition, pursuant to the above RFQ
   provisions, CSC's quotation specified a total of approximately 163
   unpriced government FTEs, spread over 7 years, which would be required
   from the PMO/LTO. In contrast, IBM listed 486.3 government FTEs, which
   resulted in a $51.1 million adjustment to its evaluated price. IBM did not
   specifically enumerate the government FTEs from the PMO/LTO that it would
   require.

   Government Obligations

   As an initial matter, IBM asserts that issuance of a task order to CSC was
   improper because the Air Force lacked authority to contractually commit
   government personnel to work with CSC on the ECSS effort. This argument is
   untimely. As discussed above, the RFQ made it clear that the agency
   anticipated "a unique sharing of concerns, knowledge and ideas via a
   partnership whereby Government personnel are made available to provide a
   source of information, and to work side-by-side with the contractor." RFQ
   amend. 5. Further, the RFQ included attachment 14, wherein vendors were to
   list the government personnel time that would be required under the
   vendor's proposed approach, with the contractor generally "entitled to an
   equitable adjustment should the Government fail to make the required
   personnel available [in accordance with] . . . Attachment 14." Id. Under
   our Bid Protest Regulations, protests based upon alleged improprieties in
   a solicitation apparent prior to the time set for receipt of initial
   quotations must be filed prior to that time, while alleged improprieties
   subsequently incorporated into the solicitation must be protested not
   later than the next closing time for receipt of quotations. 4 C.F.R.
   21.2(a)(1) (2006). Since it was clear from the solicitation that
   government personnel would be made available to work with the SI under the
   contract, and since IBM did not protest this aspect of the solicitation
   prior to the next closing time, the protest in this regard is untimely.

   In any case, IBM's quotation was based on the same approach about which
   IBM complains with regard to CSC's quotation--it provided in attachment 14
   that the Air Force would be required to furnish 486.3 government FTEs.
   Having submitted a quotation incorporating government-furnished personnel,
   IBM cannot now assert that it was improper for the agency to accept CSC's
   quotation due to its reliance on such personnel. The integrity of the
   protest process does not permit a protester to espouse one interpretation
   or position during the procurement, and then argue during a protest that
   the interpretation or position is unreasonable or otherwise improper.
   Northrop Grumman Sys. Corp., B-298954 et al., Jan. 12, 2007, 2007 CPD
   para. __ at 8; BST Sys., Inc., B-298761, B-298761.2, Dec. 1, 2006, 2006
   CPD para. __ at 6; AAI Eng'g Support, Inc., Nov. 16, 1994, 95-1 CPD para.
   2 at 3-4.

   IBM further asserts that CSC rendered its quotation unacceptable by
   improperly conditioning its performance on the Air Force's furnishing
   government personnel in a certain manner. IBM notes in this regard that
   CSC included in its proposed Performance Work Statement (PWS) an
   "Assumption[] and Constraint[]" that the "change agent roster will remain
   stable with limited turnover and replacements. No more than 10% turnover
   of change agents per year." CSC PWS at 47.

   Here too, however, IBM is challenging an aspect of CSC's quotation that is
   similar to its own. In this regard, while CSC's quotation set forth
   certain "Assumptions and Constraints," IBM's defined numerous "Government
   Roles and Responsibilities." For example, IBM's quotation provided that it
   was the Air Force's responsibility to: [REDACTED] See, e.g., IBM PWS at 3,
   17, 26-27, 59-61, 67, 86, 93. While perhaps not as specific as some of the
   government obligations set forth in CSC's quotation, IBM's definition of
   the "Government['s] Roles and Responsibilities" likewise defined
   government obligations under the contemplated order, and to that extent
   could be viewed as similarly conditioning IBM's quotation. Again, we view
   this argument as inconsistent with INM's position during the procurement,
   and we therefore will not consider it. To the extent that CSC allegedly
   acted inconsistently with the solicitation, so did IBM. See Northrop
   Grumman Sys. Corp., supra; BST Sys., Inc., supra; AAI Eng'g Support, Inc.,
   supra.

   Availability of Government Personnel

   IBM asserts that the Air Force did not adequately account for the risk
   that it would be unable to furnish the 1,202.6 government FTEs, and the
   approximately 163 additional FTEs from the PMO/LTO, specified in CSC's
   quotation.[2] In this regard, the protester notes that the Air Force has
   already decided to acquire the services of up to 28 contract personnel
   under the General Services Administration's Mission Oriented Business
   Integrated Services (MOBIS) contract in order to supplement the LTO staff.
   More generally, the protester notes that the Air Force has announced plans
   to reduce overall staffing by approximately 40,000 active duty personnel
   and 18,000 civilian/guard/reserve personnel. The Air Force responds that
   it reasonably determined that adequate government staffing would be
   available to support either vendor's proposed ECSS approach.

   Based on our review of the record, including testimony at the hearing our
   Office conducted in this matter, we find nothing unreasonable in this
   aspect of the evaluation. As noted by the agency, ECSS is a major program
   that will completely transform the Air Force logistics process by
   replacing over 400 legacy systems and supporting over 250,000 concurrent
   users. The record indicates that, during the course of the procurement,
   the Air Force Director of Transformation for logistics, who determined
   that the agency could furnish the 400-1,200 government personnel (not
   including PMO or LTO staff) the vendors had proposed, repeatedly briefed
   the leadership of the nine Air Force Major Commands involved in the Air
   Force logistics process on the extent of the commitment of agency
   personnel to ECSS that would be necessary. The director testified that the
   ECSS program had received a commitment from the Major Commands to furnish
   the requisite support. Hearing Transcript (Tr.) at 65-67, 96-98, 139-42;
   ECSS Briefing Index; see, e.g., AF Logistics Transformation eLog21
   Briefing Slides, Sept. 13, 2006. Given that the Air Force logistics
   workforce consists of at least 160,000 personnel, and the evident
   commitment of the Major Commands to furnish the resources necessary to
   transform the Air Force logistics process, the agency reasonably could
   conclude that the total of 1,365.6 government FTEs called for under CSC's
   approach would be made available, especially since only a part-time
   commitment would be required for most affected employees. CSC attach.
   14.[3]

   IBM's Change Management Approach

   As noted above, IBM's approach to Change Management was rated as high risk
   due to its identification of and request for an insufficient number of
   government personnel to be furnished. IBM disputes the agency's
   determination that its staffing was inadequate in this regard.

   The evaluation in this area was reasonable. The RFQ described change
   management--that is, the process by which executives, middle managers, and
   end users are prepared to accept changes in business processes--as "the
   key to successful implementation and adoption of a logistics enterprise
   solution." SOO sect. 3.2. The RFQ explained the importance of change
   management as follows: "To succeed, ECSS must gain user buy-in,
   demonstrate the value/benefits of common approaches across the Air Force
   Logistics Enterprise, and create confidence in the ECSS solution with its
   associated processes and supporting organizational changes." Id.

   IBM acknowledged in its quotation that "[t]he importance of change
   management to the ultimate success of the endeavor simply cannot be
   overstated or overemphasized," and that "success is built on a true
   partnership with the AF and that active participation of the AF community
   in all aspects of the program is critical." IBM Technical Proposal sect.
   3.0, Change Management; IBM Response to Evaluation Notice (EN)
   A-PE-CR-255. However, the record supports the agency's determination that
   IBM's quotation provided insufficient participation by Air Force personnel
   in the change management process. In this regard, the agency explains
   that, in the proposed sequence of ECSS activities, commencing with
   planning and continuing through blueprinting, implementation, deployment,
   and transition to sustainment, change management staffing should increase
   at the end of planning and the beginning of blueprinting, and then begin
   to decline during the last part of deployment and the beginning of
   transition to sustainment. The agency found that IBM's approach did not
   follow this staffing pattern. Rather, IBM's attachment 14 requested 21.25
   government FTEs for participation in change management activities in each
   of the first 3 contract years, and then only 17 in year 4, that is, prior
   to implementation, 12.75 in year 5, 8.5 in year 6, and none in year 7. In
   contrast, CSC's approach more closely followed the desired pattern,
   requesting the participation of 4 government FTEs in change management
   activities in year 1, 5.6 in year 2, 76.5 in year 3, 133 in year 4, 111.3
   in year 5, 26.2 in year 6, and none in year 7. CSC FQR attach. 14.

   The agency found as another indication of inadequate involvement of
   government personnel in the change management process, the fact that the
   425 members of IBM's proposed Stakeholder Engagement Team were to
   participate in change management activities for only 5 percent of their
   time in years 1-3, 4 percent in year 4, 3 percent in year 5, 2 percent in
   year 6, and 0 percent in year 7. The agency found that this level of
   participation, after accounting for conferences, meetings and reviews,
   would leave little time for true change management activities. Proposal
   Analysis Report (PAR) at 30.

   Further, the agency determined that another potential approach to
   facilitating change management--relying on super users in the user
   activities--was not part of IBM's approach as priced in attachment 14. In
   this regard, under the RFQ, the SI was to train super users (estimated to
   number 5,000), who would then provide support and advice on the new
   systems and processes to non-super users. The agency noted that, while
   IBM's quotation indicated that "[t]he ECSS program will benefit greatly
   from the involvement of the ECSS super users," IBM did not include in its
   attachment 14 calculations any time for the super users to provide this
   assistance, explaining instead that "the super users will optionally
   choose to support their co-workers." RFQ Technical Parameters sect. 3.2;
   IBM Response to Evaluation Notice (EN) A-PE-CR-299; PAR at 31. The agency
   asked IBM to explain why it characterized super users as "optional
   resources" (and accordingly did not list them in attachment 14 for pricing
   purposes). IBM responded that, while the ECSS program would "benefit
   greatly" from the participation of the super users, "ECSS implementation
   and deployment is not dependent on the estimated 5,000 ECSS super-users
   serving as local area experts and providing local support." IBM Response
   to Evaluation Notice (EN) A-PE-CR-299. The agency ultimately evaluated
   IBM's approach as relying on the super users to facilitate change
   management, since IBM had listed only limited other government
   participation in the change management process. Since, however, the super
   users were not priced in IBM's attachment 14, the agency determined that
   IBM's approach to super users was further evidence of IBM's inadequate
   involvement of government personnel (as priced in attachment 14). Tr. at
   1380; Agency Comments, Feb 13, 2007, at 74.[4]

   Finally, the agency viewed as still further evidence of IBM's inadequate
   involvement of government personnel the fact that IBM's overall ratio of
   government personnel to SI contractor personnel was only approximately
   0.75:1. According to the agency, a ratio of up to 3:1 was recommended in
   large ERP implementations, while the minimum government participation
   level was 1:1. PAR at 30-31. IBM disputes the agency's understanding of
   the appropriate government/contractor personnel ratio. We note, however,
   that IBM itself advised the agency during a February 3, 2005 Industry Day
   briefing that "[t]he typical commercial ERP implementation has 3 to
   4 clients for every consultant." Feb. 3, 2005 Industry Day Briefing
   Slides. According to the protester's briefing, "[m]any public sector
   clients do not commit the level of resources required to successfully
   implement and sustain their program."

   We think the Air Force could reasonably view the above aspects of IBM's
   quotation as indicative that its approach (in attachment 14) to government
   participation was inadequate. IBM has not shown otherwise. Id. In
   particular, given the agency's stated intent to adopt "commercial best
   practices," RFQ sect. 3.1, Project Summary/Strategy, and IBM's own
   recommendation that the government adopt a more commercial approach to ERP
   implementation, IBM's defense of its proposed much more limited level of
   government participation is not persuasive. We thus find no basis to
   question the agency's determination that IBM's approach to government
   participation was inadequate.

   IBM asserts that it was misled by the agency as to the expected level of
   government personnel participation in the ECSS process. Specifically, IBM
   notes that the RFQ stated that "[t]he Government intends for the SI to
   provide as many of the necessary project resources as possible." SOO sect.
   4.2. IBM further notes that, in response to an IBM question during
   discussions as to whether it was "the Government's intention to allow
   unlimited use of Air Force resources," the agency responded as follows:
   "No, the Government has limited resources and must fully understand the
   quantity and timing of personnel required to support each vendor's
   approach." Air Force Memo for File, Apr. 20, 2006.

   We find nothing misleading here. The statements cited by IBM are general
   in nature and, we think, did not warrant an interpretation, essentially,
   that the agency intended vendors to minimize their use of government
   personnel. In particular, we view the agency's statement--in response to
   IBM's inquiry as to whether agency resources were "unlimited"--that the
   government has limited resources as no more than a statement of the
   obvious. Moreover, IBM's assertion that it was misled ignores the fact
   that, in response to its listing of only 146.3 government FTEs in its
   initial response to the April 26, 2006 amendment adding the attachment 14
   government personnel matrix, the agency advised the firm in a deficiency
   notice that "[y]our response to the Government Personnel Template is
   unrealistically low." EN A-AD-DR-294. As evidenced in IBM's subsequent
   more than three-fold increase in the number of requested government FTEs
   listed in its revised attachment 14, ultimately reaching 486.3 FTEs, IBM
   clearly understood that proposing too limited government participation in
   the ECSS process would be evaluated unfavorably.

   PAST PERFORMANCE

   As noted above, the Air Force rated CSC's past performance "significant
   confidence" and IBM's only "confidence." IBM challenges CSC's rating
   primarily on the basis that it failed to reflect CSC's poor performance
   under contracts concerning the Internal Revenue Service's (IRS) Business
   Systems Modernization (BSM) and Integrated Financial System (IFS)
   programs.

   The evaluation of past performance, including the determination of the
   relevance and scope of vendors' performance history to be considered, is a
   matter of agency discretion, which we will not question unless it is
   unreasonable or inconsistent with the solicitation criteria, or
   procurement statutes or regulations. National Beef Packing Co., B-296534,
   Sept. 1, 2005, 2005 CPD para. 168 at 4. The past performance evaluation
   here was unobjectionable.

   The RFQ provided for the agency to evaluate the "recent and relevant
   performance" of the vendor and its three principal subcontractors,
   "focusing on and targeting performance which is relevant to the Mission
   Capability subfactors and the requirements of the solicitation." RFQ, Best
   Value Selection Factors, sect. 5.1. The RFQ listed several factors that
   would be considered, including "[m]anagement actions employed and
   indicators/metrics used to achieve success and overcome problems." Id. The
   RFQ added that "more relevant past performance efforts will be considered
   more heavily in the rating than less relevant past performance efforts."
   Id. In addition, the RFQ defined "recent" past performance as follows: "In
   determining recency, any past performance within the last 60 months (from
   the date of RFQ release) shall be considered." Id.

   CSC's past performance rating of significant confidence was based on its
   having "demonstrated the requisite capabilities to develop and deploy
   large-scale, highly complex COTS/ERP integrated business solutions
   comparable to the magnitude and complexity of the ECSS." SSD at 5;
   Performance Confidence Assessment Group (PCAG) Final Report at 111. In
   particular, the agency noted CSC's strong performance under three contract
   efforts that were evaluated as "Very Relevant large-scale, highly complex,
   integrated business solutions:" (1) a contract under the Army's Logistics
   Modernization Program (LMP), an ERP within a Department of Defense
   logistics environment, under which CSC reduced legacy system operations
   from more than 1,000 non-integrated applications to a single solution; the
   Air Force rated CSC's performance very good (the Army program manager
   evaluated CSC's performance as exceptional); (2)  a contract under General
   Dynamics Land Systems' (GDLS) e-Business program, an ERP within a
   commercial logistics domain, under which CSC implemented an Oracle-based
   ERP solution; CSC's performance was rated very good by the Air Force; and
   (3) a contract under a Dupont Chemical Solutions Enterprise ERP program,
   under which CSC implemented a solution within a commercial logistics
   domain; CSC's performance was rated exceptional by the Air Force. In
   addition, the agency determined that five other CSC contracts and seven
   subcontractor contracts were relevant; the quality of performance was
   rated by the Air Force as exceptional for six of these, very good for
   four, and satisfactory for two--CSC's contracts with IRS for the BSM and
   IFS programs. Five other CSC contracts were considered somewhat relevant;
   CSC's performance was rated by the Air Force as exceptional for four of
   these and very good for the fifth. PCAG at 63-111.

   IBM's past performance rating of confidence was based on its having
   "consistently delivered products or services, developed transformational
   processes, drove best business practices into the operational environment,
   and developed COTS and/or ERP applications which improved their client's
   business operations." PCAG at 57. However, unlike its findings with regard
   to CSC, the agency found "less evidence" that IBM had performed projects
   comparable to ECSS, and rated none of IBM's contracts as very relevant.
   SSD at 5. The agency did identify three relevant contracts that it
   characterized as medium-scale, moderately complex projects: (1) a contract
   for a Delta Airlines ERP implementation within a commercial logistics
   domain; IBM's performance was rated as satisfactory by the Air Force; (2)
   a subcontractor's contract for a Defense Logistics Agency ERP
   implementation within a DOD logistics domain; the subcontractor's
   performance was rated as very good by the Air Force; and (3) a
   subcontractor's contract for an Oracle ERP implementation within a
   commercial logistics domain; the subcontractor's performance was rated as
   exceptional by the Air Force. Twelve additional contracts performed by IBM
   or its subcontractors were found to be relevant; the performance on eight
   of these was rated exceptional, on three as very good, and on one as
   satisfactory. One IBM contract was considered to be only somewhat
   relevant, with IBM's performance rated as satisfactory.

   IBM primarily asserts that CSC encountered significant performance
   problems under the two IRS contracts, and that it was unreasonable for the
   agency to rate CSC's performance on those contracts as satisfactory--the
   same rating IBM received for its relevant Delta contract, during which it
   experienced no significant performance problems.

   This argument is without merit. As noted by the Air Force, while CSC
   encountered performance problems on the IRS contracts--as reflected in
   questionnaires completed by IRS officials, interviews with IRS officials,
   and IRS Contractor Performance Assessment Reports (CPAR)--the record
   indicates that CSC's rating reflected the fact that IRS was viewed as
   sharing some responsibility for those problems, and the fact that CSC's
   performance significantly improved during the contract term. In this
   regard, IRS's BSM program was a complex, multi-billion dollar effort
   encompassing 17 major modernization projects, for which CSC served as the
   prime integrator, responsible for an alliance of subcontractors (including
   IBM). The record indicates that CSC initially appeared to be overwhelmed
   by the magnitude and complexity of the project, and was late in
   recognizing performance problems with several of the subcontractors
   (including, especially, IBM's performance with respect to IRS's Customer
   Account Data Engine (CADE)). CSC received several unsatisfactory ratings
   in IRS CPARs due to resulting delays. The Air Force noted, however, that
   there were reports of role confusion between IRS and CSC; that the IRS
   commissioner at the beginning of the program had stated that it was a
   mistake to believe that any vendor could manage such a huge undertaking
   without significant input from IRS; and that IRS's associate chief
   information officer had not only concurred with the commissioner's view,
   but also attributed some of the failure points to the fact that the
   contract model was a flawed approach. The Air Force also was advised that
   there was a significant improvement when CSC took over CADE development
   from IBM, that IRS was "very happy" with the quality of the CADE end
   product, and that CSC received an overall excellent rating for quality (as
   well as a fair rating for schedule and a good rating for cost control)
   under CSC's most recent BSM CPAR. CSC Performance Documentation, IRS BSM
   Contract.

   Likewise, the record of CSC's performance on the IRS IFS contract, as
   reported to the Air Force, indicated that CSC had successfully overcome
   early performance problems. CSC was tasked under the IFS contract with
   selecting and deploying a new integrated financial system. The record
   indicated that CSC initially failed to assign staff with the requisite
   expertise, was late in discovering problems with its subcontractor's
   performance, and fell behind the required schedule. IRS IFS Performance
   Questionnaire. However, the record also indicated that some of the delay
   had resulted from problems with the joint IRS/CSC requirements development
   process, and that the schedule established by IRS had been very
   aggressive, so that the time CSC required for the project actually
   favorably compared to that required for similar projects. Project Notes,
   IRS IFS. In any case, the record indicated that CSC's performance
   subsequently improved, and that the project was successfully completed.
   Indeed, the record indicated that IRS's new integrated financial system
   resulted in the agency's receiving its first unqualified audit opinion
   from GAO. CSC Performance Documentation, IRS IFS Contract.

   In view of CSC's ultimately successful performance on the IRS contracts,
   as well as the reported extenuating circumstances (i.e., the magnitude and
   complexity of the challenges, IRS having contributed to the performance
   problems, and/or IRS having established an overly aggressive schedule),
   there is no basis for us to object to the Air Force's rating of CSC's
   performance on these contracts as satisfactory.[5]

   Given the Air Force's determination that CSC had exceptional or very good
   performance on three very relevant contracts, and that IBM's quotation
   cited no very relevant contracts, there is no basis for questioning the
   ratings in this area.

   The protest is denied.

   Gary L. Kepplinger
   General Counsel

   ------------------------

   [1] IBM also proposed a rapid porting of the existing Pathfinder
   capabilities into a production environment, thus providing the potential
   for accelerating inventory reductions and improved weapons system
   availability. However, because this approach required rapid approval of a
   waiver from the Office of the Secretary of Defense, which the agency
   viewed as unlikely, this approach was assigned a high risk rating. PAR at
   26, 73.

   [2] IBM also asserts that the agency failed to consider the risk
   associated with having government personnel participate with CSC in the
   ECSS effort, including such risks as errors by government personnel in
   performing their assigned roles. Again, however, since both vendors'
   quotations called for the participation of government personnel in the
   ECSS effort as contemplated by the solicitation, we see little fundamental
   difference between the quotations in this regard.

   [3] Although IBM suggests that there may be greater difficulties
   associated with meeting CSC's request for 163 government FTEs over 7 years
   from PMO/LTO, we note that the record indicates that ECSS is the most
   significant program being undertaken by the 754^th Electronic Systems
   Group, which is part of the 554^th Electronic Systems Wing. Given that
   these units include approximately 1,800 government personnel (as well as
   another 1,600 contractor personnel), we think the agency reasonably could
   conclude that sufficient government personnel can be matrixed to the PMO
   to enable the agency to furnish the requested resources in this area. Tr.
   at 359-64, 504-05.

   [4] IBM asserts that CSC likewise failed to price any super user time in
   its attachment 14. However, since the documents on which this challenge is
   based were furnished to IBM on December 15, 2006, and IBM did not raise
   the argument until January 3, 2007, more than 10 days later, this aspect
   of the protest is untimely. 4 C.F.R. sect. 21.2(a)(2). In any case, IBM's
   argument fails to account for the fact that, while its approach was
   evaluated as effectively requiring the efforts of the super users, CSC's
   approach was based on significantly greater participation by government
   personnel, including a Change Agent Network to serve as change agents in
   the user activities, which were priced in CSC's attachment 14. Thus, CSC,
   unlike IBM, was not dependent upon super users in order to facilitate
   change management. CSC FQR Attach. 14 sect. 2.2.1; Agency Comments, Feb.
   13, 2007, at 73-77.

   [5] IBM also generally questions the process by which the Air Force
   determined the relevance of vendors' contracts. However, IBM has shown
   neither that the agency unreasonably determined that CSC's LMP, GDLS and
   Dupont contracts were very relevant, nor that any of its own contracts
   (that were performed within the 60-month recency window) warranted a
   similar very relevant rating. For example, IBM has failed to rebut the
   agency's determination that its Delta contract was only relevant (not very
   relevant) since, while the contract was for an ERP implementation within a
   logistics domain, the agency learned that much of IBM's work under the
   contract was performed prior to the 60-month recency window, and that
   other work had been performed by another contractor. Tr. at 1444-47;
   Agency Comments, Feb. 10, 2007, at 64-65.