TITLE: InkiTiki Corporation, B-291823.4; B-291823.5, May 16, 2003
BNUMBER: B-291823.4; B-291823.5
DATE: May 16, 2003
**********************************************************************
InkiTiki Corporation, B-291823.4; B-291823.5, May 16, 2003
Decision
Matter of: InkiTiki Corporation
File: B-291823.4; B-291823.5
Date: May 16, 2003
Dr. Susanna Tsai for the protester.
Brian Kau, Esq., Department of the Navy, for the agency.
Katherine I. Riback, Esq., and James A. Spangenberg, Esq., Office of the
General Counsel, GAO, participated in the preparation of the decision.
DIGEST
Protest of agency's decision not to invite the protester to submit a Phase
II proposal under the Department of Defense Small Business Innovation
Research program is denied where the agency reasonably determined, based
on the lack of details in the protester's Phase I written products and
post-award conference, that it could not be determined whether the
protester's approach was innovative or feasible.
DECISION
InkiTiki Corporation protests the Department of the Navy's determination
not to invite it to submit a Phase II proposal under the Department of
Defense's (DOD) Small Business Innovation Research (SBIR) program.
We deny the protest.
The SBIR program is conducted pursuant to the Small Business Innovation
Development Act, 15 U.S.C. S: 638 (2000), which requires certain federal
agencies to reserve a portion of their research and development (R&D)
funds for awards to small businesses. It is a three-phased process of
soliciting proposals and awarding funding for R&D to small businesses for
stated agency needs. Phase I is to determine, insofar as possible, the
scientific, technical, and commercial merit and feasibility of ideas
submitted under the SBIR program. Phase II is the principal R&D effort
demonstrating the Phase I technology, including the delivery of a
prototype. Phase III contemplates that non-SBIR funds will be used to
develop the prototype into a viable product or non-R&D service for sale to
the military or in commercial markets.
To commence the program, the agency issues an SBIR solicitation that sets
forth R&D topics and subtopics that emphasize the need for proposals with
advanced concepts to meet specific agency R&D needs, without providing
detailed specifications to prescribed solutions of the problems. SBIR
Program Policy Directive P: 7.b. Here, DOD issued FY 2002 SBIR Program
Solicitation 02.1 on October 1, 2001, which included the Department of the
Navy's solicited topic, *Innovative Reverse Engineering Protection for
Software.* Under this topic, a contractor was required to *develop usable
and novel approaches to protecting software from reverse engineering.*
DOD Fiscal Year 2002 SBIR Program Solicitation 02.1, Department of Navy
Topic N02-100.
According to the solicitation, the awards of Phase I contracts were to be
based on the following technical evaluation factors:
a. The soundness, technical merit, and innovation of the proposed
approach and its incremental progress toward topic or subtopic solution.
b. The qualifications of the proposed principal/key investigators,
supporting staff, and consultants. Qualifications include not only the
ability to perform the research and development but also the ability to
commercialize the results.
c. The potential for commercial (Government or private sector)
application and the benefits expected to accrue for this
commercialization.
Where technical evaluations were essentially equal in merit, cost was to
be considered in determining the successful offerors. Id. P: 4.2.
Under this solicitation, the same Phase I evaluation factors were to be
used to evaluate Phase II proposals. Id. P: 4.3. The solicitation also
provided that the Phase II awardees were to be selected from firms that
had received Phase I awards on the *basis of results of their Phase I
effort and the scientific, technical, and commercial merit of the Phase II
proposal,* and that the *Phase II proposal evaluation may include on-site
evaluations of the Phase I effort by Government personnel.* Id. P:P: 1.2,
4.3. The solicitation further stated that *[o]nly those Phase I awardees
which have been invited to submit a Phase II proposal by that activity's
proper point of contact . . . during or at the end of a successful Phase I
effort will be eligible to participate for a Phase II award.* Id.,
Department of the Navy Proposal Submission Instructions, at 2-3 (emphasis
in original).
InkiTiki and two other firms received Phase I awards under the Navy topic
in question here. InkiTiki proposed software protection measures using
innovative artificial intelligence techniques. InkiTiki's contract
required the delivery of a number of reports, including monthly status
reports, a Phase I preliminary report, a Phase I final report, and a Phase
II plan. InkiTiki Contract, attach. J.1. The record shows that InkiTiki
furnished the monthly status reports, a Phase II plan and a Phase I
preliminary report; InkiTiki has not submitted a Phase I final report.[1]
Prior to receiving the Phase I final reports, the technical monitor and
two other evaluators reviewed, among other things, the Phase I
contractors' Phase II plans, monthly reports, and Phase I preliminary
reports to decide whether to invite the contractors to submit a Phase II
proposal. In so doing, the evaluators considered the following evaluation
criteria (not listed in the solicitation): quality of communications to
the government; innovativeness of approach to solution and/or product;
past performance during Phase I; clearness of plans and ability to perform
or execute the plans; usefulness of final product to the government and in
commercial applications; best value to the government; best price and/or
return on investment; and ability to work under classified procedures in
Phase II. Based on its evaluation, the agency decided, on November 14,
that it would not invite InkiTiki to submit a Phase II proposal.
The agency's determination not to invite InkiTiki to submit a Phase II
proposal was due in large part to its conclusion that InkiTiki's approach,
as described in InkiTiki's written products and discussed at a Phase I
mid-term conference, did not demonstrate the innovativeness promised in
its Phase I proposal with respect to the use of artificial intelligence in
*obfuscation techniques* to make software tamper-proof. In this regard,
the agency noted that InkiTiki's written submissions and monthly progress
reports lacked detail, and made it impossible to determine whether
InkiTiki's approach was in fact innovative or feasible, or what progress,
if any, the contractor had achieved on its project. Specifically
regarding the required monthly status reports, the evaluators concluded
that they were *very vague and limited in details.* Agency Supplemental
Documents (Feb. 13, 2003), Tab 3a, InkiTiki's Proposal Evaluation, at 1.
The evaluators also found that InkiTiki failed to provide substantial
information evidencing an application of artificial intelligence to
current obfuscation techniques, which was what was supposed to have been
new and innovative about InkiTiki's approach; that InkiTiki had provided
*[n]o details on [its artificial intelligence] approach such as
[artificial intelligence] algorithms, theories, reasoning engine details,
etc., [so that it was] [d]ifficult to assess innovativeness [, and that
InkiTiki had] [g]reat ideas for obfuscation techniques, but again no
obvious innovations with [artificial intelligence].* Agency Supplemental
Documents (Feb. 13, 2003), Tab 3a, InkiTiki's Proposal Evaluation, at 1.
The agency's technical monitor also stated that regardless of what
InkiTiki knew about artificial intelligence, its discussion and
presentation of its proposed use of artificial intelligence during the
mid-term conference was *superficial and lacking in substantive details,*
and *more tutorial* with *[n]ot enough detail to assess true progress or
to communicate real product or design to [agency].* Id.; Agency's
Technical Monitor's Memorandum (Mar. 3, 2003) at 5.
InkiTiki protests the agency's failure to invite it to submit a Phase II
proposal. Our review of a protest involving an SBIR procurement is
limited to determining whether the agency violated any applicable
regulations or solicitation provisions, or acted in bad faith.
Intellectual Properties, Inc., B-280803.2, May 10, 1999, 99-1 CPD P: 83
at 5‑6; Microexpert Sys., Inc., B-233892, Apr. 13, 1989, 89-1 CPD P:
378 at 2.
While InkiTiki generally disputes this evaluation, arguing for example
that its Phase I proposal demonstrated its innovativeness and that the
evaluators lacked expertise to understand its artificial intelligence
approach, it has not shown that the agency's evaluation judgment, as
detailed above, was unreasonable.[2] The record shows that the agency
reasonably concluded that the few details provided by InkiTiki during
Phase I did not show its proposed approach was sound, had technical merit,
or was innovative, or that InkiTiki was making progress towards a
solution.
InkiTiki argues that to the extent that the agency determined that its
written submissions lacked detail and therefore did not show innovation,
this was due to instructions received from the agency's technical monitor
to include only *minimal information* in its monthly reports and
preliminary report due to security considerations, and to hand deliver the
actual detailed research findings only to the technical monitor. InkiTiki
contends that the agency officials acted in bad faith when they instructed
InkiTiki to submit documents with minimal information and then during
evaluation downgraded InkiTiki for not providing more information and
detail in its written submissions.
The agency specifically denies that it instructed or encouraged InkiTiki
to put minimal information in its submissions, and supports this denial
with a number of affidavits from the pertinent agency representatives.
The agency's technical monitor stated that in fact InkiTiki's written
monthly status reports assumed greater importance to the agency due to the
geographical distance between the agency's location in Virginia and the
contractor's location in Hawaii, and the significant time zone difference
between those two areas, which made communication by telephone difficult.
The technical monitor contends that InkiTiki has fabricated this
allegation to *account for the lack of detail in its monthly reports and
other submitted documents (which was obvious in comparison to the
submissions of the other contractors and evident on their face even to lay
persons).*[3] Agency's Technical Monitor's Affidavit (Apr. 14, 2003) at
8.
In response, InkiTiki asserts that the agency representatives' statements
should be disregarded because they are false. However, the protester has
provided no evidence that supports its version of these events.[4] We
have reviewed the record and find no support for InkiTiki's allegation of
bad faith on the part of Navy officials in the evaluation of its
proposal. In this regard, prejudicial motives will not be attributed to
contracting officials on the basis of unsupported allegations, inference,
or supposition. Matrix Int'l Logistics, Inc., B-277208, B-277208.2, Sept.
15, 1997, 97-2 CPD P: 94 at 13 n.9.
InkiTiki also asserts that it was unfair to consider the mid-term
conference as part of the evaluation for Phase II, or to decide which
Phase I contractors to solicit for Phase II prior to receiving the final
Phase I reports, because the agency did not give InkiTiki proper notice
that it would do so. However, the solicitation clearly put InkiTiki on
notice that its Phase I contract performance would be considered in the
Phase II evaluation and that the decision of which contractors to solicit
for Phase II could be made during Phase I prior to the submission of the
final report. DOD Fiscal Year 2002 SBIR Program Solicitation 02.1
P:P: 1.2, 4.3; Department of the Navy Proposal Submission Instructions, at
2-3.
The protester finally argues that the agency's use of unstated evaluation
criteria to determine which contractors to solicit Phase II proposals from
was improper and inconsistent with the SBIR solicitation evaluation
scheme. While it is true that the criteria used to determine whether to
solicit the contractors for Phase II were not stated in the solicitation,
the agency explains that it regarded these criteria as *relevant subsets*
of the evaluation factors listed in the solicitation. Agency's Technical
Monitor's Memorandum (Mar. 3, 2003) at 2. Regardless of whether the
specific criteria used were all encompassed by the solicitation's
evaluation scheme, the record shows that the reasons InkiTiki was not
solicited to submit a proposal for Phase II clearly fell under the first
Phase II factor listed in the solicitation, *the soundness, technical
merit, and innovation of the proposed approach and its incremental
progress toward topic or subtopic solution.*
In sum, we find no basis to question the agency's determination not to
solicit InkiTiki for Phase II.
The protest is denied.
Anthony H. Gamboa
General Counsel
------------------------
[1] The agency granted an InkiTiki request for an extension of the
delivery date for its final report. InkiTiki did not meet the extended
due date and, as noted, has never provided a final Phase I report.
[2] The protester claims that its work was characterized by the agency's
technical monitor as important to the government. The technical monitor
has persuasively responded in detail to the protester's attacks on the
evaluation, and asserts that he did not compliment InkiTiki's
demonstrations or submissions during Phase I. Agency's Technical
Monitor's Affidavit (Apr. 14, 2003) at 6.
[3] Consistent with the technical monitor's view that InkiTiki may not
have developed the details supporting its approach is the fact that, as
noted above, InkiTiki has never submitted its final Phase I report, even
though it was granted an extension. See Agency Submission (April 14,
2003) at 8 n.4.
[4] We attempted to develop this issue further by means of a hearing where
the credibility of the parties could be directly assessed. InkiTiki
declined to participate. Therefore, this issue was further developed by
means of questions by our Office, to which the agency and InkiTiki
responded.