TITLE: B-311344; B-311344.3; B-311344.4; B-311344.6; B-311344.7; B-311344.8; B-311344.10; B-311344.11, The Boeing Company, June 18, 2008
BNUMBER: B-311344; B-311344.3; B-311344.4; B-311344.6; B-311344.7; B-311344.8; B-311344.10; B-311344.11
DATE: June 18, 2008
*********************************************************************************************************************************
B-311344; B-311344.3; B-311344.4; B-311344.6; B-311344.7; B-311344.8; B-311344.10; B-311344.11, The Boeing Company, June 18, 2008
DOCUMENT FOR PUBLIC RELEASE
The decision issued on the date below was subject to a GAO Protective
Order. This redacted version has been approved for public release.
Decision
Matter of: The Boeing Company
File: B-311344; B-311344.3; B-311344.4; B-311344.6; B-311344.7;
B-311344.8; B-311344.10; B-311344.11
Date: June 18, 2008
Rand L. Allen, Esq., Paul F. Khoury, Esq., Scott M. McCaleb, Esq., Martin
P. Willard, Esq., Nicole J. Owren-Wiest, Esq., Kara M. Sacilotto, Esq.,
Nicole P. Wishart, Esq., Jon W. Burd, Esq., Stephen J. Obermeier, Esq.,
and Heidi L. Bourgeois, Esq., Wiley Rein LLP; and Charles J. Cooper, Esq.,
Michael W. Kirk, Esq., and Howard C. Neilson, Esq., Cooper & Kirk; Lynda
Guild Simpson, Esq., and Stephen J. Curran, Esq., The Boeing Company, for
the protester.
Neil H. O'Donnell, Esq., Allan J. Joseph, Esq., David F. Innis, Esq.,
Thomas D. Blanford, Esq., Aaron P. Silberman, Esq., Tyson Arbuthnot, Esq.,
Michelle L. Baker, Esq., James Robert Maxwell, Esq., and Suzanne M.
Mellard, Esq., Rogers Joseph O'Donnell; and Joseph O. Costello, Esq.,
Northrop Grumman Systems Corporation, for the intervenor.
Bryan R. O'Boyle, Esq., Col. Neil S. Whiteman, James A. Hughes, Esq., Col.
Timothy Cothrel, Robert Balcerek, Esq., Maj. Christopher L. McMahon, W.
Michael Rose, Esq., Stewart L. Noel, Esq., Gerald L. Trepkowski, Esq.,
Lynda Troutman O'Sullivan, Esq., John J. Thrasher III, Esq., Lt. Col.
Thomas F. Doyon, Anthony P. Dattilo, Esq., Bridget E. Lyons, Esq., John R.
Hart, Esq., Ronald G. Schumann, Esq., Maj. Steven M. Sollinger, Maj.
Sandra M. DeBalzo, and John M. Taffany, Esq., Department of the Air Force,
for the agency.
Guy R. Pietrovito, Esq., and James A. Spangenberg, Esq., Office of the
General Counsel, GAO, participated in the preparation of the decision.
DIGEST
1. Protest is sustained, where the agency, in making the award
decision, did not assess the relative merits of the proposals in
accordance with the evaluation criteria identified in the solicitation,
which provided for a relative order of importance for the various
technical requirements, and where the agency did not take into account the
fact that one of the proposals offered to satisfy more "trade space"
technical requirements than the other proposal, even though the
solicitation expressly requested offerors to satisfy as many of these
technical requirements as possible.
2. Protest is sustained, where the agency violated the solicitation's
evaluation provision that "no consideration will be provided for exceeding
[key performance parameter] KPP objectives" when it recognized as a key
discriminator the fact that the awardee proposed to exceed a KPP objective
relating to aerial refueling to a greater degree than the protester.
3. Protest is sustained, where the record does not demonstrate the
reasonableness of the agency's determination that the awardee's proposed
aerial refueling tanker could refuel all current Air Force fixed-wing
tanker-compatible receiver aircraft in accordance with current Air Force
procedures, as required by the solicitation.
4. Protest is sustained, where the agency conducted misleading and
unequal discussions with the protester, where the agency informed the
protester that it had fully satisfied a KPP objective relating to
operational utility, but later determined that the protester only
partially met this objective, without advising the offeror of this change
in its assessment and while continuing to conduct discussions with the
awardee relating to its satisfaction of the same KPP objective.
5. Protest is sustained, where the agency unreasonably determined
that the awardee's refusal to agree to the specific solicitation
requirement that it plan and support the agency to achieve initial organic
depot-level maintenance within 2 years after delivery of the first
full-rate production aircraft was an "administrative oversight," and
improperly made award, despite this clear exception to a material
solicitation requirement.
6. Protest is sustained, where the agency's evaluation of military
construction costs in calculating the offerors' most probable life cycle
costs for their proposed aircraft was unreasonable, where the evaluation
did not account for the offerors' specific proposals, and where the
calculation of military construction costs based on a notional
(hypothetical) plan was not reasonably supported.
7. Protest is sustained, where the agency improperly added costs to
an element of cost (non-recurring engineering costs) in calculating the
protester's most probable life cycle costs to account for risk associated
with the protester's failure to satisfactorily explain the basis for how
it priced this cost element, where the agency did not determine that the
protester's proposed costs for that element were unrealistically low.
8. Protest is sustained, where the agency's use of a "Monte Carlo"
simulation model to determine the protester's probable cost of
non-recurring engineering associated with the system demonstration and
development portion of the acquisition was unreasonable, where the model's
inputs concerned total weapons systems at an overall program level and
there is no indication that this is a reliable predictor of anticipated
growth of the protester's non-recurring engineering costs.
9. Protester is not required to file a "defensive protest" when
during the procurement it is apprised of an agency's evaluation judgments
with which it disagrees or where it believes the evaluation is
inconsistent with the solicitation's evaluation scheme, because GAO's Bid
Protest Regulations, 4 C.F.R. sect. 21.2(a)(2) (2008), require that where
the protest involves a procurement conducted on the basis of competitive
proposals under which a debriefing is requested and, when requested, is
required, these protest grounds can only be raised after the offered
debriefing.
10. While an agency, in an appropriate case, may request under GAO's Bid
Protest Regulations, 4 C.F.R. sect. 21.3(d) (2008), that a protester
provide specific relevant documents, of which the agency is aware and does
not itself possess, this does not allow for "wide-open" document requests
by an agency of broad categories of documents.
DECISION
The Boeing Company protests the award of a contract to Northrop Grumman
Systems Corporation under request for proposals (RFP) No.
FA8625-07-R-6470, issued by the Department of the Air Force, for aerial
refueling tankers.[1] Boeing challenges the Air Force's technical and cost
evaluations, conduct of discussions, and source selection decision.[2]
As explained below, we find that the agency's selection of Northrop
Grumman's proposal as reflecting the best value to the government was
undermined by a number of prejudicial errors that call into question the
Air Force's decision that Northrop Grumman's proposal was technically
acceptable and its judgment concerning the comparative technical
advantages accorded Northrop Grumman's proposal. In addition, we find a
number of errors in the agency's cost evaluation that result in Boeing
displacing Northrop Grumman as the offeror with the lowest evaluated most
probable life cycle costs to the government. Although we sustain Boeing's
protest on grounds related to these errors, we also deny many of Boeing's
challenges to the award.
Specifically, we sustain the protest, because we find that (1) the Air
Force did not evaluate the offerors' technical proposals under the key
system requirements subfactor of the mission capability factor in
accordance with the weighting established in the RFP's evaluation
criteria; (2) a key technical discriminator relied upon in the selection
decision in favor of Northrop Grumman relating to the aerial refueling
area of the key system requirements subfactor, was contrary to the RFP;
(3) the Air Force did not reasonably evaluate the capability of Northrop
Grumman's proposed aircraft to refuel all current Air Force fixed-wing,
tanker-compatible aircraft using current Air Force procedures, as required
by the RFP; (4) the Air Force conducted misleading and unequal discussions
with Boeing with respect to whether it had satisfied an RFP objective
under the operational utility area of the key system requirements
subfactor; (5) Northrop Grumman's proposal took exception to a material
solicitation requirement related to the product support subfactor; (6) the
Air Force did not reasonably evaluate military construction (MILCON) costs
associated with the offerors' proposed aircraft consistent with the RFP;
and (7) the Air Force unreasonably evaluated Boeing's estimated
non-recurring engineering costs associated with its proposed system
development and demonstration (SDD).
BACKGROUND
Aerial refueling is a key element supporting the effectiveness of the
Department of Defense's (DoD) air power in military operations and is, as
such, an important component of national security. See AR, Tab 333,
Capability Development Document, Dec. 27, 2006, at 2, 7; see also Air
Force Refueling: The KC-X Aircraft Acquisition Program, Congressional
Research Service (CRS) Report for Congress, No. RL34398, Feb. 28, 2008, at
1. Currently, the Air Force uses two types of aircraft for aerial
refueling: the KC-135, which is considered to be a medium-sized airplane,
and the larger KC-10. The Air Force's fleet of KC-135 aircraft currently
has an average age of 46 years and is the oldest combat weapon system in
the agency's inventory;[3] for the newer KC-10 aircraft, the average age
is over 20 years. Defense Acquisitions: Air Force Decision to Include a
Passenger and Cargo Capability in Its Replacement Refueling Aircraft Was
Made without Required Analyses, GAO-07-367R, Mar. 6, 2007, at 1.
To begin replacing the aging refueling tanker fleet, the Air Force
established a three-pronged approach under which it intended to first
conduct a procurement to replace the older KC-135 tankers, while
maintaining the remaining KC-135 and KC-10 tankers; the first procurement,
which is the acquisition protested here, was identified by the Air Force
as the KC-X procurement or program. See AR, Tab 4, Acquisition Strategy
Plan Briefing, at 9-10. The Air Force intends to replace the remaining
KC-135 and KC-10 aircraft in later procurements under programs the agency
identified as the KC-Y and KC-Z.
Solicitation
The RFP, issued January 30, 2007, provided for the award of a contract
with cost reimbursement and fixed-price contract line items. In this
regard, offerors were informed that, although the agency would procure up
to 179 KC-X aircraft over a 15 to 20-year period, the initial contract
would be for the SDD of the KC-X aircraft and the procurement of up to 80
aircraft, beginning with the delivery of four SDD aircraft and two low
rate initial production (LRIP) aircraft.[4] Offerors were also informed
that the agency contemplated receiving an existing commercial, Federal
Aviation Administration (FAA) or equivalent certified transport aircraft
modified to meet the agency's requirements. RFP, Statement of Objectives
(SOO) for KC-X SDD, at 1.
A detailed system requirements document (SRD) was provided in the RFP that
presented the technical performance requirements for the KC-X aircraft. In
this regard, the SRD stated that
[t]he primary mission of the KC-X is to provide world-wide, day/night,
adverse weather aerial refueling . . . on the same sortie to receiver
capable United States (U.S.), allied, and coalition military aircraft
(including unoccupied aircraft). [The KC-X aircraft will] provide
robust, sustained [aerial refueling] capability to support strategic
operations, global attack, air-bridge, deployment, sustainment,
employment, redeployment, homeland defense, theater operations, and
special operations. Secondary missions for KC-X include emergency aerial
refueling, airlift, communications gateway, aeromedical evacuation (AE),
forward area refueling point (FARP), combat search and rescue, and
treaty compliance.
RFP, SRD sect. 1.2.1. The SRD identified the minimum and desired
performance/capability requirements for the aircraft. The minimum
performance capabilities of the aircraft were identified in nine key
performance parameters (KPP), which the Air Force summarized as follows:
+------------------------------------------------------------------------+
|KPP| Parameter | Required Performance |
|---+-------------------------------+------------------------------------|
| 1 |Tanker Air Refueling Capability|Air refueling of all current and |
| | |programmed fixed wing receiver |
| | |aircraft |
|---+-------------------------------+------------------------------------|
| 2 |Fuel Offload and Range |Fuel, offload, range chart |
| | |equivalent to KC-135 |
|---+-------------------------------+------------------------------------|
| 3 |Communications, Navigation, |Worldwide flight operations at all |
| |Surveillance/Air Traffic |times in all civil and military |
| |Management |airspace |
|---+-------------------------------+------------------------------------|
| 4 |Airlift Capability |Carry passengers, palletized cargo, |
| | |and/or aeromedical patients on |
| | |entire main deck |
|---+-------------------------------+------------------------------------|
| 5 |Receiver Air Refueling |Refueled in flight from any boom |
| |Capability |equipped tanker aircraft |
|---+-------------------------------+------------------------------------|
| 6 |Force Protection |Operate in chemical/biological |
| | |environments |
|---+-------------------------------+------------------------------------|
| 7 |Net-Ready |Meet enterprise-level joint critical|
| | |integrated architecture requirements|
|---+-------------------------------+------------------------------------|
| 8 |Survivability |Operate in hostile environments |
| | |(night vision and imaging systems, |
| | |electromagnetic pulse, defensive |
| | |systems: infrared detect and |
| | |counter, radio frequency detect, no |
| | |counter) |
|---+-------------------------------+------------------------------------|
| 9 |Multi-point Refueling |Multi-point drogue[5] refueling |
+------------------------------------------------------------------------+
AR, Tab 46, Source Selection Evaluation Team (SSET) Final Briefing to
Source Selection Advisory Council (SSAC) and Source Selection Authority
(SSA), at 18.
The RFP provided for award on a "best value" basis and stated the
following evaluation factors and subfactors:
+------------------------------------------------------------------------+
|Mission Capability |
|------------------------------------------------------------------------|
| |Key System Requirements |
| |---------------------------------------------------|
| |System Integration and Software |
| |---------------------------------------------------|
| |Product Support |
| |---------------------------------------------------|
| |Program Management |
| |---------------------------------------------------|
| |Technology Maturity and Demonstration |
|------------------------------------------------------------------------|
|Proposal Risk |
|------------------------------------------------------------------------|
|Past Performance |
|------------------------------------------------------------------------|
|Cost/Price |
|------------------------------------------------------------------------|
|Integrated Fleet Aerial Refueling Assessment (IFARA) |
+------------------------------------------------------------------------+
Offerors were informed that the mission capability, proposal risk, and
past performance factors were of equal importance and individually more
important than the cost/price or IFARA factors, and that the cost/price
and IFARA factors were of equal importance. The subfactors within the
mission capability factor were stated to be of descending order of
importance. RFP sect. M.2.1.
The RFP stated that the agency, in its evaluation of proposals under the
mission capability subfactors, would assign one of the color ratings
identified in the solicitation,[6] and one of the proposal risk ratings
that were also identified.[7] RFP sections M.2.2, M.2.3. In this regard,
offerors were informed that proposal risk would only be assessed at the
mission capability subfactor level and for only the first four subfactors.
RFP sect. M.2.3.
With respect to the key system requirements subfactor, the most important
mission capability subfactor, offerors were informed that the agency would
assess the offerors' understanding of, and substantiation of their ability
to meet, the requirements of the SRD (with the exception of the logistics
requirements that were to be evaluated under the product support
subfactor). The RFP provided that the offerors' approaches to meeting the
SRD requirements would be evaluated under the key system requirements
subfactor in the following five areas: aerial refueling, airlift,
operational utility, survivability, and "other system requirements." RFP
sect. M.2.2.1.2.
In order for a proposal to be found acceptable under this subfactor (and
overall), an offeror was required to meet the various identified minimum,
mandatory KPP "thresholds" identified in the SRD for each of the nine
KPPs. The SRD also identified KPP "objectives" relating to some, but not
all of, the identified KPP thresholds. In this regard, the RFP stated that
[a]ll KPP thresholds [relating to the aerial refueling, airlift,
operational utility, and survivability areas] must be met. Depending on
substantiating rationale, positive consideration will be provided for
performance above the stated KPP thresholds up to the KPP objective
level. No consideration will be provided for exceeding KPP objectives.
If there is no stated objective and, depending on substantiating
rationale, positive consideration will be provided when the specified
capability above the KPP threshold is viewed as advantageous to the
Government.
RFP sect. M.2.2.1.1.a.
Among the minimum requirements identified in the SRD was a KPP No. 1
threshold that required the offeror's proposed aircraft to be "capable of
aerial refueling all current [Air Force] tanker compatible fixed wing
receiver aircraft using current [Air Force] procedures . . . ." RFP, SRD
sect. 3.2.10.1.1.9. Another minimum requirement was a KPP No. 2 threshold
that required the offeror's aircraft to be capable of satisfying the fuel
offload versus unrefueled radius range as depicted in a linear graph
contained in the RFP; this threshold charted the minimum pounds of fuel an
aircraft must be capable of offloading to a receiver aircraft at a given
distance of unrefueled flight by the tanker.[8] See RFP, SRD
sect. 3.2.1.1.1.1. Also identified under KPP No. 2, as an objective, was
that the "aircraft should be capable of exceeding the fuel offload versus
unrefueled radius range as depicted in" this chart. RFP, SRD
sect. 3.2.1.1.1.2.
In addition, the SRD identified numerous key system attributes (KSA) for
the aerial refueling, airlift, operational utility, survivability, and
"other system requirements" areas, as well as numerous other "non-KPP/KSA
requirements" for these areas that were desired but not required.[9] The
RFP provided that these "requirements" did not have to be satisfied by the
offerors, but were desired and considered part of the offerors' "design
trade space."[10] RFP sect. M.2.2.1.1.b. With respect to these aspects of
the evaluation of the key system requirements subfactor, offerors were
informed that
[f]or non-KPP requirements, the Government may give consideration for
alternate proposed solutions or capabilities below the stated SRD
requirement, depending on substantiating rationale. The Government may
give additional consideration if the offeror proposes to meet (or exceed
if there is an objective) the SRD threshold or requirement, depending on
substantiating rationale.
RFP sect. M.2.2.1.1.b. The RFP further stated that the Air Force sought an
affordable KC-X system that not only met all of the KPP threshold
requirements, but as many KSA and other SRD requirements as possible. RFP,
SOO for KC-X SDD, at 2.
Finally, with regard to the overall evaluation of the key system
requirements subfactor, the RFP stated that "evaluation of the offeror's
proposed capabilities and approaches against the SRD requirements will be
made in the following descending order of relative importance: KPPs, KSAs,
and all other non-KPP/KSA requirements." RFP sect. M.2.2.1.1.c.
With respect to the aerial refueling area of the key system requirements
subfactor, offerors were informed that the agency's evaluation would
include "tanker aerial refueling, receiver aerial refueling, fuel offload
versus radius range, drogue refueling systems (including simultaneous
multi-point refueling), the operationally effective size of the boom
envelope, the aerial refueling operator station and aircraft fuel
efficiency." RFP sect. M.2.2.1.2.a. With respect to airlift area, the RFP
provided that the agency's evaluation would include "airlift efficiency,
cargo, passengers, aero-medical evacuation, ground turn time, and cargo
bay re-configuration." RFP sect. M.2.2.1.2.b. Offerors were instructed
with regard to this area to provide an airlift efficiency calculation,
based upon a calculation procedure stated in the solicitation, that would
result in a "payload pounds - nautical miles per pound fuel used"
calculation (in other words, the weight of cargo per pound of fuel
burned). RFP sect. L.4.2.2.4.1. Under the operational utility area, the
agency's evaluation would include "aircraft maneuverability, worldwide
airspace operations, communications/information systems (including
Net-Ready capability), treaty compliance support, formation flight,
intercontinental range, 7,000-foot runway operations, bare base airfield
operations, and growth provisions for upgrades." RFP sect. M.2.2.1.2.c.
The survivability area evaluation would include "situational awareness,
defensive systems against threats, chemical/biological capability,
[electromagnetic pulse] protection, fuel tank fire/explosion protection,
and night vision capability." RFP sect. M.2.2.1.2.d. The remaining "other
system requirements" area evaluated SRD requirements were not included in
any of the other areas. RFP sect. M.2.2.1.2.e.
Under the system integration and software subfactor, the evaluation was to
consider the offeror's ability to implement a systems engineering approach
and software development capability to satisfy the KC-X performance
requirements, considering a number of listed attributes. RFP sect.
M.2.2.2.
Under the product support subfactor, the evaluation was to consider the
offeror's product support approach that includes logistics planning and
analysis; interim contractor support; transition to organic two-level
maintenance support; approach and rationale for proposed operational
availability, reliability and maintainability and mission capable rates;
logistics footprint; site activation/beddown; and training. RFP sect.
M.2.2.3.
With respect to the program management subfactor, offerors were informed
that the agency would assess whether "the offeror's proposal demonstrates
a capability to effectively and efficiently implement and manage the KC-X
Program." RFP sect. M.2.2.4. Included in this evaluation was whether the
offeror demonstrated a "sound approach to achieving FAA
Certification/Validation" and a "feasible, effective, low risk
manufacturing and quality assurance approach to integrating military
capability into the commercial baseline aircraft and transition to full
rate production." RFP sections M.2.2.4.C, M.2.2.4.F.
With respect to the past performance factor, the RFP informed offerors
that the agency's performance confidence assessment group (PCAG) would
conduct an in-depth review and evaluation of all performance data to
determine how closely the work performed under those efforts related to
the effort solicited under the RFP. The RFP provided that for this factor
the agency would assess the degree of confidence that the agency had in an
offeror's ability to perform the tanker contract, based upon an assessment
of the offeror's demonstrated record of performance, and focusing on
performance in five areas: the four mission capability subfactors and the
cost/price factor.[11] RFP sect. 2.4.1. In this regard, the RFP stated
that the agency would consider each offeror's, and its major/critical
subcontractor's, demonstrated record of performance. Offerors were also
informed that, in assessing an offeror's past performance, the agency
would consider the relevance of an offeror's (and its subcontractor's,
joint venture's, and teaming partner's) present and past performance, and
that "[m]ore recent and more relevant performance by the same
division/organization may have a greater impact on the performance
confidence assessment than less recent or less relevant effort." Id.
sect. M.2.4.5.3. With respect to an offeror's performance problems, the
RFP stated:
Where relevant performance records indicate performance problems, the
Government will consider the number and severity of the problems and the
appropriateness and effectiveness of any corrective actions taken (not
just planned or promised). The Government may review more recent
contracts or performance evaluations to ensure corrective actions have
been implemented and to evaluate their effectiveness.
RFP sect. M.2.4.4.
With respect to the IFARA evaluation factor, the RFP provided that the
agency would assess the utility and flexibility of a fleet of the
offeror's proposed aircraft "by evaluating the number of aircraft required
to fulfill the peak demand of the aerial refueling elements evaluated in
the 2005 Mobility Capabilities Study."[12] Specifically, offerors were
informed that the Air Force would analyze offeror-provided data in the
evaluation scenario "primarily using the Combined Mating and Ranging
Planning System (CMARPS) modeling and simulation tool" to calculate a
"fleet effectiveness value," and would report this finding to the source
selection authority (SSA), along with "any major insights and observations
gleaned from the evaluation."[13] To calculate the fleet effectiveness
value, the agency, using the CMARPS modeling tool, would calculate the
number of KC-135R aircraft and the number of the offeror's proposed
aircraft needed to satisfy the scenario, and then divide the number of
KC-135R aircraft required by the number of the offeror's aircraft. The RFP
stated that, with respect to this ratio, a fleet effectiveness value of
1.0 would be equal in effectiveness to the KC-135R, while a value in
excess of 1.0 would be viewed as more advantageous to the agency. RFP
sect. M.2.6.
Under the cost/price factor, the RFP provided that offerors' proposed
costs and prices would be evaluated for realism and reasonableness,
respectively. RFP sect. M.2.5. Offerors were also informed that the agency
would calculate a most probable life cycle cost (MPLCC) estimate for each
offeror, which was described by the solicitation to be "an independent
government estimate, adjusted for technical, cost, and schedule risk, to
include all contract, budgetary and other government costs associated with
all phases of the entire weapon system life cycle (SDD, [Production and
Deployment], and Operations and Support (O&S))." RFP sect. 2.5.2. The RFP
provided that, as part of the "other government costs," the agency would
evaluate anticipated MILCON costs associated with the offerors' proposed
aircraft. RFP sect. 2.5.2.4. The RFP also provided that the agency would
assess "technical, cost, and schedule risk for the entire most probable
life cycle cost estimate based upon the offeror's proposed approach," and
that the "impact of technical, schedule, and/or cost risk will be
quantified (dollarized), where applicable, and included in the MPLCC." RFP
sect. M.2.5.2.5.
The RFP instructed the offerors to provide detailed cost information
supported by a basis of estimate. Offerors were informed that the basis of
estimate must
completely describe the cost element content . . . philosophy, and
methodology used to develop the estimate including appropriate
references to any historical supporting cost date.
RFP sect. L.6.4.7. The basis of estimate was required to include a
"narrative with supporting data explaining how the proposed cost estimates
(SDD, [production and deployment], O&S) were created." RFP sect. L.6.2.
With respect to proposed O&S costs, which include fuel costs, offerors
were informed that they should assume a 25-year system life from the date
each aircraft is delivered and "calculate their O&S costs for 2 years
beyond the date of their final production delivery"; to support their O&S
cost projections, offerors were required to provide all "assumptions,
ground rules, methodology, and supporting data."[14] RFP sections
L.6.1.1.13, L.6.4.9. In this regard, the offerors were informed that if
the historical data did not support the proposed prices, the cost
documentation would be considered adequate only if the agency could
understand the technical content, estimating methodology, and the
"build-up" of the offerors' costs. RFP sect. L.6.4.7.
Proposals
The Air Force received proposals from Boeing and Northrop Grumman in
response to the RFP. Boeing proposed as its KC-X aircraft the KC-767
Advanced Tanker, a derivative of its commercial 767-200 LRF (long range
freighter) aircraft.[15] The KC-767 was composed of elements of a number
of Boeing commercial aircraft, including the 767-200ER, 767-300F, 767-400
ER, 737, and 777 models. AR, Tab 61, Boeing Initial Technical Proposal,
Executive Summary, at V1-ES-1. Boeing's proposed production plan for its
SDD and production KC-X aircraft was to build the 767-200 LRF baseline
aircraft at the Everett, Washington facility of its commercial division,
Boeing Commercial Airplanes (BCA), and then fly the aircraft to its
Wichita, Kansas facility for installation of military equipment and
software by its military division, Integrated Defense Systems (IDS). Id.
at V1-ES-2.
Northrop Grumman proposed the KC-30 aircraft, which was a derivative of
the Airbus A330-200 commercial aircraft.[16] AR, Tab 140, Northrop Grumman
Initial Technical Proposal, Executive Summary, at I-1. Northrop Grumman
proposed a production plan that provided for a number of changed locations
for the production, assembly, and modification of its SDD and LRIP
aircraft. For the first SDD aircraft, Northrop Grumman proposed to build
the commercial A330 aircraft in sections in various European locations,
then assemble the aircraft in Toulouse, France, add the cargo door in
Dresden, Germany, and complete militarization of the aircraft in Madrid,
Spain. For the second and third SDD aircraft, Northrop Grumman proposed
using its own Melbourne, Florida facility, in place of EADS's Madrid
facility, to complete militarization. For the last SDD aircraft, Northrop
Grumman proposed replacing its Melbourne facility with a new facility it
proposed to build in Mobile, Alabama. For the first LRIP aircraft,
Northrop Grumman proposed to have the Toulouse facility not only assemble
the commercial baseline aircraft but also install the cargo door, and the
Mobile facility would complete the militarization of the aircraft.
Beginning with the second LRIP aircraft, and thereafter through the
production phase, Northrop Grumman proposed to build the A330 baseline
aircraft in sections at various locations in Europe and then ship those
sections to the Mobile facility, which would assemble the aircraft,
install the cargo door, and complete militarization of the aircraft. Id.
at I-6; see also Hearing Testimony (HT) at 1343-52.[17]
SSET Evaluation
The proposals were evaluated by the agency's SSET, which initiated
discussions with the offerors by issuing evaluation notices (EN).[18]
After evaluating the offerors' EN responses, the SSET provided a
"mid-term" evaluation briefing to the SSAC and SSA. Because there were
"concerns regarding how to properly show that all SRD requirements had
been evaluated," the SSET prepared and provided another briefing to the
SSA that detailed how each offeror's proposal was evaluated against each
SRD requirement. COS at 24. Following the SSA's approval of the mid-term
briefing, the SSET provided mid-term briefings to Boeing and Northrop
Grumman, at which each offeror was provided with the agency's evaluation
ratings of their respective proposals.[19] AR, Tabs 129, 130, Boeing's
Mid-Term Briefings; Tabs, 199, 200, Northrop Grumman's Mid-Term Briefings.
Following the offerors' mid-term briefings, the SSET provided a
MPLCC/schedule risk assessment briefing to the SSAC and SSA, and
subsequently the SSET provided MPLCC/schedule risk assessment briefings to
the offerors. AR, Tab 133, Boeing's MPLCC/Schedule Risk Assessment
Briefing; Tab 203, Northrop Grumman's MPLCC/Schedule Risk Assessment
Briefing.
Extensive discussions were conducted with each offeror, after which a
"pre-final proposal revision" briefing was provided to the SSAC and SSA by
the SSET that presented updated evaluation ratings of Boeing's and
Northrop Grumman's proposals and discussion responses. Following approval
of this briefing by the SSA, the SSET again provided to each offeror the
agency's evaluation ratings of their respective proposals. AR, Tab 135,
Boeing's Pre-Final Proposal Revision Briefing; Tab 205, Northrop Grumman's
Pre-Final Proposal Revision Briefing.
"Final revised proposals" were received from the offerors. Although
intended by the agency to be the final proposal revisions, shortly after
receipt of these proposals, the Air Force reopened discussions with the
offerors in response to the enactment of the National Defense
Authorization Act for Fiscal Year 2008, Pub. L. No. 110-181, 122 Stat. 3,
208-12, 222-24 (2008).[20] As a part of these discussions, the Air Force
provided offerors with additional information concerning the firms'
respective IFARA evaluations and with a "clarified chart on Airlift
Efficiency." COS at 25. Subsequently, the agency received the firms' final
proposal revisions.
The protester's and awardee's final proposal revisions were evaluated by
the SSET as follows:
+------------------------------------------------------------------------+
| | Boeing |Northrop Grumman|
|--------------------------------------+---------------------------------|
|Mission Capability/Proposal Risk | |
|--------------------------------------+---------------------------------|
| |Key System Requirements | Blue/Low | Blue/Low |
| |------------------------------------+----------------+----------------|
| |System Integration/Software | Green/Moderate | Green/Moderate |
| |------------------------------------+----------------+----------------|
| |Product Support | Blue/Low | Blue/Low |
| |------------------------------------+----------------+----------------|
| |Program Management | Green/Low | Green/Low |
| |------------------------------------+----------------+----------------|
| |Technology Maturity/Demonstration | Green | Green |
|--------------------------------------+----------------+----------------|
|Past Performance | Satisfactory | Satisfactory |
| | Confidence | |
| | | Confidence |
|--------------------------------------+----------------+----------------|
|Cost/Price (MPLCC) |$108.044 Billion|$108.010 Billion|
|--------------------------------------+----------------+----------------|
| |Cost Risk | Moderate/Low | Low/Low |
| | | | |
| |SDD Phase/Production & Deployment | | |
| |Phase | | |
|--------------------------------------+----------------+----------------|
|IFARA Fleet Effectiveness Value | 1.79 | 1.9 |
+------------------------------------------------------------------------+
AR, Tab 46, SSET Final Briefing to SSAC and SSA, at 508, 532.
As indicated by the nearly identical evaluation ratings received by both
firms' technical proposals and the nearly identical evaluated MPLCCs, the
competition was very close, and, as evaluated, both firms' proposals were
found to be advantageous to the government. Ultimately, the SSAC
concluded, however, that Northrop Grumman's proposal was more advantageous
to the agency than Boeing's under the mission capability, past
performance, cost/price, and IFARA factors; the two firms were found to be
essentially equal under the proposal risk factor. AR, Tab 55, Proposal
Analysis Report (PAR), at 46-48.
SSAC's Mission Capability Factor Evaluation
Northrop Grumman's evaluated advantage under the mission capability factor
was largely based upon the firm's perceived superiority under the key
system requirements and program management subfactors; the two firms were
found essentially equal under the remaining three subfactors. Id. at
46-47.
The SSAC assigned both firms' proposals, under the key system requirements
subfactor (the most important mission capability subfactor), blue, low
risk ratings, noting:
Both Offerors proposed to meet all KPP Thresholds. Both Offerors
proposed capability beyond KPP Thresholds and offered significant trade
space KSA capability. Additionally, both offered numerous non-KPP/KSA
trade space capabilities deemed beneficial to the Government.
Id. at 12. This assessment was documented in the SSAC's PAR, which
identified evaluated "major discriminators," "discriminators offering less
benefit" and weaknesses in each offeror's proposal in the aerial
refueling, airlift, operational utility, survivability, and "other system
requirements" areas of this subfactor.[21] Id. at 13-28.
In the aerial refueling area, the SSAC noted "major discriminators" in
favor of Boeing under several KPP No. 1 objectives, including its
capability to [Deleted] and [Deleted], and for a "noteworthy non-KPP/KSA
capability to [Deleted]. Id. at 13.
The SSAC also noted a number of "major discriminators" in favor of
Northrop Grumman in the aerial refueling area, including one under the KPP
No. 2 objective for Northrop Grumman's proposal to exceed the RFP's fuel
offload versus unrefueled radius range (Boeing's aircraft was also
evaluated as exceeding this KPP objective but to a lesser degree),[22] and
for a number of non-KPP/KSA requirements, including the proposal of a
better aerial refueling efficiency (more pounds of fuel offload per pound
of fuel used) than Boeing's; a "boom envelope" that was [Deleted] times
greater than that defined by the Allied Technical Publication (ATP)-56[23]
(Boeing proposed a boom envelope that was [Deleted] times greater than
that defined by the publication); and a higher offload and receive fuel
rate than Boeing. Id. at 13-14.
In the aerial refueling area, the SSAC also identified five
"discriminators offering less benefit" for Boeing that were assessed under
14 different SRD requirements and one such discriminator for Northrop
Grumman that was assessed under 2 SRD requirements. Id. at 15-16.
The SSAC found that Boeing's proposal had no weaknesses in the aerial
refueling area, but identified the following two weaknesses in Northrop
Grumman's proposal:
The first weakness is related to the specified lighting around the fuel
receptacle of the KC-30. The specified lighting for refueling as a
receiver may provide [Deleted]. The second weakness is related to
Northrop Grumman's boom approach. The [Deleted].
Id. at 16. The concern that Northrop Grumman's [Deleted] was assessed
under a KPP No. 1 threshold; the other weaknesses were assessed under
non-KPP/KSA requirements. No schedule or cost risk was assigned by the
SSET or SSAC for either of Northrop Grumman's evaluated weaknesses. See
AR, Tab 46, SSET Final Briefing to SSAC and SSA, at 196, 198; Tab 55, PAR,
at 16.
In the airlift area, the SSAC found that both offerors met all threshold
requirements for the airlift KPP (there was only one KPP in this area),
and that both offerors exceeded the threshold requirement for efficiently
transporting equipment and personnel. AR, Tab 55, PAR, at 16. There were
no KPP objectives identified by the SRD in the airlift area.
The SSAC identified one "major discriminator" in favor of Boeing in the
airlift area: Boeing satisfied the non-KPP/KSA requirement for the
capability to [Deleted]. Id. at 17. With respect to Northrop Grumman, the
SSAC identified a number of "major discriminators" in the airlift area.
That is, with respect to carrying cargo, the SSAC found that Northrop
Grumman had a better airlift efficiency capability than Boeing, showing an
improvement of [Deleted] percent over that of the KC-135R, while Boeing's
airlift efficiency showed only a [Deleted]-percent improvement over the
KC-135R.[24] The SSAC noted that the KC-30 could carry more 463L
pallets[25] than Boeing,[26] and that Northrop Grumman offered the
capability to carry 463L pallets on both the main cargo deck and a lower
cargo compartment, while Boeing only offered the single cargo deck. The
SSAC also identified "major discriminators" in Northrop Grumman's proposal
for passenger carriage ([Deleted] passengers to Boeing's [Deleted]
passengers) and for aeromedical evacuation capability (Northrop Grumman
could carry more litters and ambulatory patients). Id. at 18-19.
Three "discriminators offering less benefit" were identified for Boeing in
the airlift area and one such discriminator identified for Northrop
Grumman. No proposal weaknesses were identified for either offeror in the
airlift area. Id. at 19-20.
In the operational utility area, the SSAC found that both offerors
satisfied the three KPP thresholds identified in this area, and partially
met the one KPP objective identified.[27] The SSAC also found that both
offerors met the KSA thresholds and objectives in this area. Id. at 20.
Two "major discriminators" were identified for Boeing in this area:
(1) [Deleted] and (2) [Deleted]. Id. at 21. Two "major discriminators"
were also identified for Northrop Grumman: (1) the KC-30 could operate
from a 7,000-foot runway carrying approximately [Deleted] percent more
fuel than the KC-767,[28] and (2) the KC-30 provided a ferry range of
[Deleted] nautical miles as compared to the KC-767's ferry range of
[Deleted] nautical miles.[29] Id. at 21-22. Numerous "discriminators
offering less benefit" were identified for both Boeing and Northrop
Grumman. Among such discriminators identified for Boeing was the KC-767's
smaller ground footprint, which the SSAC found would enable the KC-767 to
operate from bare base airfields with confined ramp space.[30] Id. at 22.
No proposal weaknesses were identified for either offeror in this area.
Ultimately, the SSAC concluded, largely based upon Northrop Grumman's
evaluated advantages in the aerial refueling and airlift areas, that
Northrop Grumman's proposal was superior to Boeing's under the key system
requirements subfactor.[31] Specifically, the SSAC noted:
While [the] KC-767 offers significant capabilities, the overall
tanker/airlift mission is best supported by the KC-30. [The] KC-30
solution is superior in the core capabilities of fuel capacity/offload,
airlift efficiency, and cargo/passenger/aeromedical carriage. These
advantages in core capabilities outweigh the flexibility advantages of
the attributes which Boeing offered (e.g. [Deleted], etc.)
Id. at 29.
Under the program management subfactor, the SSAC assigned both offerors
green, low risk ratings, identifying no strengths, deficiencies, or
uncertainties in either firm's proposal. Id. at 34. Nevertheless, the SSAC
concluded that Northrop Grumman's program management approach was superior
to that of Boeing, finding:
Northrop Grumman's approach of providing four "green" aircraft for use
early in SDD, by leveraging the existing A330 commercial production
line, is deemed to be of benefit to the Government by reducing program
risk. Northrop Grumman's approach adds value for the Government through
increased confidence in overall program management.
Id. at 46-47.
Past Performance Factor Evaluation
The SSAC found that both offerors had equal confidence ratings in four of
the five past performance areas; the only difference in ratings was with
respect to the program management area, under which Northrop Grumman's
past performance was assessed as "satisfactory confidence" but Boeing's
proposal was assessed as "little confidence."[32] Id. at 36. Boeing's
little confidence rating for the program management area was based upon
the Air Force's assessment of Boeing's past performance of the [Deleted]
contract with [Deleted], of the [Deleted] contract with the [Deleted], and
of the [Deleted] with the [Deleted]. The Air Force evaluated as marginal
Boeing's past performance of these contracts, which were assessed as "very
relevant." Id. at 37-38.
IFARA Factor Evaluation
The SSET also calculated a fleet effectiveness value for each proposed
aircraft based upon offeror-provided data, which was analyzed under a
variety of scenarios using the CMARPS modeling and simulation tool.[33] As
noted above, the fleet effectiveness value reflected the quantity of an
offeror's aircraft that would be required to perform the scenarios in
relation to the number of KC-135R aircraft that would have been required.
See RFP sect. M.2.6. The agency concluded that, whereas [Deleted] KC-135R
aircraft would be required to perform the identified scenarios, the
offerors' aircraft could perform the scenarios with fewer aircraft, that
is, [Deleted] KC-30 aircraft and [Deleted] KC-767 aircraft. AR, Tab 55,
PAR, at 45. The SSET calculated a fleet effectiveness value of 1.79 for
the KC-767, and a higher (superior) value of 1.90 for the KC-30. Id. at
44.
The SSAC also noted a number of insights and observations concerning the
IFARA evaluation of the offerors' aircraft. With respect to Boeing's
proposed aircraft, the agency stated that, as compared to the KC-135R in
the peak demand scenario:
[the] KC-767 used [Deleted]% more ramp space (without requiring
additional bases), burned [Deleted]% more fuel and was able to
accomplish the scenarios with [Deleted] fewer aircraft when taking the
aerial refueling receptacle into account. Additional aircraft were
needed if every runway in the scenario were interdicted to 7,000 feet.
In the base denial scenarios, when a base was closed, [Deleted]% of the
Air Tasking Order (ATO) could be completed by basing KC-767s within the
remaining bases' ramp space. Within the scenarios, [the] KC-767
offloaded between [Deleted]% and [Deleted]% of its fuel.
Id. at 45. With respect to Northrop Grumman's aircraft, the agency stated:
[the] KC-30 used [Deleted]% more ramp space (needing some additional
bases), burned [Deleted]% more fuel and was able to accomplish the
scenarios with [Deleted] fewer aircraft when taking the aerial refueling
receptacle into account. In the base denial sensitivity assessment, in
some cases when a base was closed, the [Deleted]. [The] KC-30 has
exceptional short field capability if the runway is interdicted to 7,000
feet (as noted in Subfactor 1.1). Within the scenarios, [the] KC-30
offloaded between [Deleted]% and [Deleted]% of its fuel.
Id.
Cost/Price Evaluation
The Air Force calculated a MPLCC for each offeror, which, as noted above,
was intended to be an independent government estimate of each proposal,
adjusted for technical, cost and schedule risk and including all contract,
budgetary and other government costs associated with all phases of the
aircraft's entire life cycle (SDD, production and deployment, and O&S).
See RFP sect. 2.5.2; COS at 124.
With respect to Boeing's proposal, the Air Force made a number of
adjustments in Boeing's proposed costs in calculating its MPLCC. For
example, the agency added an additional $[Deleted] million to Boeing's
proposed costs of $[Deleted] billion for SDD because the agency concluded
that the firm had not adequately supported its basis of estimate for these
costs, despite repeated discussions on this issue. Most of this adjustment
($[Deleted] million) was associated with a moderate risk rating that was
assigned to Boeing's cost proposal to account for the agency's concern
that Boeing had not adequately supported its proposed $[Deleted] billion
for non-recurring engineering costs that Boeing estimated it would incur
in the development of its proposed aircraft. As another example, the Air
Force added $[Deleted] billion to Boeing's proposed costs for the
production and deployment lots 6 through 13 (the budgetary aircraft)
because the agency concluded that Boeing had not substantiated an
approximately [Deleted]-percent decrease in proposed costs for these lots
following the fixed-price production lots (lots 1 through 5). The Air
Force also upwardly adjusted Boeing's MPLCC by $[Deleted] billion for
"other government costs," the bulk of which ($[Deleted] billion) reflected
additional O&S repair costs because the Air Force did not accept Boeing's
estimating methodology of these costs. The agency also added additional
costs to Boeing's MPLCC to account for the agency's estimated MILCON costs
of $[Deleted] billion. AR, Tab 46, SSET Final Briefing to SSAC and SSA, at
451-76; Tab 55, PAR, at 40-42.
The Air Force also made a number of adjustments in Northrop Grumman's
proposed costs, including upwardly adjusting the proposed SDD costs by
$[Deleted] million and the firm's estimated costs for lots 6 through 13
(budgetary aircraft) by $[Deleted] million. In addition, the Air Force
added additional costs to Northrop Grumman's MPLCC to account for the
agency's estimated MILCON costs of $[Deleted] billion. AR, Tab 46, SSET
Final Briefing to SSAC and SSA, at 479-502; Tab 55, PAR, at 42-43.
The Air Force calculated a MPLCC for Boeing of $108.044 billion and a
MPLCC for Northrop Grumman of $108.010 billion.
In comparing the firms' evaluated costs, the SSAC noted that Northrop
Grumman had a lower evaluated MPLCC, but that the firms' evaluated MPLCCs
were within $34 million of each other (approximately a .03-percent
difference). The SSAC noted, however, that Boeing's slightly higher
evaluated MPLCC was "driven" primarily by the firm's much higher SDD
costs, "which reflected Boeing's more complex design, development, and
integration activities." AR, Tab 55, PAR, at 43. In addition, the SSAC
accepted the SSET's evaluation that Boeing's proposal presented a moderate
cost risk for SDD. Northrop Grumman's proposal was assessed as a low cost
risk for SDD costs. The SSAC viewed this difference in cost risk for the
SDD phase to be the discriminator under this factor. Id. at 44.
SSAC Recommendation
Ultimately, the SSAC recommended to the SSA that [the SSA] select Northrop
Grumman's proposal for award, because the SSAC concluded that Northrop
Grumman's proposal was more advantageous under the mission capability,
past performance, cost/price, and IFARA evaluation factors. With respect
to cost/price, the SSAC noted that, although the difference between the
two proposals' MPLCC was "negligible," Northrop Grumman's risk rating
under this factor (low risk) for the SDD phase was lower than that
assigned to Boeing's proposal (moderate cost/price risk) for the SDD
phase. Id. at 46-48.
Selection Decision
As noted above, the SSA was presented with the SSET's evaluation results
in a number of briefings at various stages in the procurement. In
addition, the SSA was briefed by the SSAC with respect to that council's
recommendation for award and was presented with the SSAC's detailed PAR,
which documented the SSAC's weighing of the offerors' respective strengths
and weaknesses and the SSAC's award recommendation.
The SSA agreed with the SSAC's recommendation that Northrop Grumman's
proposal reflected the best value to the agency, and [the SSA] identified
Northrop Grumman's evaluated superiority under the mission capability,
past performance, cost/price, and IFARA factors as supporting this
conclusion; [the SSA] also concluded that neither offeror had an advantage
under the proposal risk factor. With respect to the mission capability
factor, the SSA emphasized that Northrop Grumman's evaluated superiority
in the aerial refueling and airlift areas of the key system requirements
subfactor were key factors in [the SSA's] decision.[34] AR, Tab 54, Source
Selection Decision, at 9. Although not key to [the SSA's] determination
that Northrop Grumman's proposal was more advantageous than Boeing's under
the key system requirements subfactor, the SSA noted Boeing's evaluated
superiority in the survivability area; [the SSA] also noted that neither
offeror had an advantage in the operational utility area. Id. at 8-9.
With respect to the aerial refueling area, the SSA noted that Northrop
Grumman exceeded the KPP objective for fuel offload capability for the
unrefueled radius range to a greater degree than did Boeing; this, the SSA
found, demonstrated that a "single KC-30 can refuel more receivers or
provide more fuel per receiver than a single KC-767." AR, Tab 54, Source
Selection Decision, at 5-6. In addition, the SSA noted that Northrop
Grumman offered a larger boom envelope than Boeing, and proposed a
superior fuel offload and receive rate than Boeing. Id. at 6. Although
Northrop Grumman had weaknesses in the aerial refueling area, and Boeing
did not, the SSA agreed with the SSAC that the weaknesses (associated with
receiver lighting and the firm's boom design) would have no impact on
program cost and schedule. Id. at 6-7.
With respect to the airlift area, the SSA noted Northrop Grumman's
superior airlift efficiency, dual cargo deck configuration, and ability to
carry more passengers and aeromedical litters and patients. The SSA
concluded that the KC-30's airlift capability was "compelling to my
decision." Id. at 7.
In sum, the SSA selected Northrop Grumman's proposal for award, finding
Northrop Grumman's proposal was better than Boeing's proposal in four of
the five factors evaluated and equal in one. Northrop Grumman's offer
was clearly superior to that of Boeing's for two areas within KC-X's
Mission Capability factor: aerial refueling and airlift. Additionally,
Northrop Grumman's KC-30's superior aerial refueling capability enables
it to execute the IFARA scenario described in the RFP with [Deleted]
fewer aircraft than Boeing's KC-767 -- an efficiency of significant
value to the Government. I am confident that Northrop Grumman will
deliver within the cost, schedule, and performance requirements of the
contract because of their past performance and the lower risk of their
cost/price proposal.
Id. at 19.
Award was made to Northrop Grumman on February 29, 2008, and following
receipt of a required debriefing,[35] Boeing protested to our Office on
March 11.
DISCUSSION
In its protest, which was supplemented numerous times as evaluation
documents were provided during the development of the case,[36] Boeing
challenges the Air Force's evaluation of technical and cost proposals,
conduct of discussions, and source selection decision. In this regard, the
protester identifies what it alleges are prejudicial errors under each of
the RFP's evaluation factors and subfactors, and contends that, if the
proposals had been evaluated in accordance with the RFP, its proposal
would have been selected for award. As discussed below, we find a number
of significant errors in the Air Force's evaluation under the key system
requirements and product support subfactors of the mission capability
evaluation factor and in its cost evaluation, and that the agency
conducted misleading and unequal discussions with Boeing.[37]
Document Production
During the development of the record, Boeing requested that the Air Force
provide various documents pursuant to our Bid Protest Regulations, 4
C.F.R. sect. 21.3(g). We granted Boeing's requests in part where we were
persuaded that the requested documents were relevant to the protest
issues.
The Air Force also requested that Boeing produce certain broad categories
of documents bearing upon, among other things, Boeing's interpretation of
the solicitation and several of its protest allegations. Boeing objected
to that request, asserting that the documents sought were not relevant.
The agency responded that its request was reasonable and limited, and
sought relevant documents, which would be "necessary to allow GAO to
perform a complete and accurate review of the issues in Boeing's
protests." Air Force's Response to Boeing's Objection to Air Force's
Document Production Request (Apr. 11, 2008) at 1.
Our Bid Protest Regulations provide, in pertinent part, that "[i]n
appropriate cases, the contracting agency may request that the protester
produce relevant documents, or portions of documents, that are not in the
agency's possession." 4 C.F.R. sect. 21.3(d).
We denied the Air Force's request, because our regulations do not provide
for broad agency request for documents whose existence and relevance are
not at all apparent.[38] Rather, our regulations are intended to permit a
contracting agency, in an appropriate case, to request a specific relevant
document or documents, of which the agency is aware and does not itself
possess. See 60 Fed. Reg. 40737, 40738 (wherein, in establishing this
rule, we indicated that our regulations were not intended to allow
"wide-open" document requests of protesters); see also Bid Protests at
GAO: A Descriptive Guide, 8^th ed. 2006, at 28, in which our Office
described the purpose for our "reverse discovery" rule as follows:
Occasionally, the agency may be aware of the existence of relevant
documents that only the protester possesses. In appropriate cases, the
agency may request that the protester produce those documents.[39]
Dismissal Requests
Prior to the submission of the agency's report, the Air Force and Northrop
Grumman requested that we summarily dismiss a substantial portion of
Boeing's protest as untimely. The agency and intervenor argued that some
of Boeing's protest grounds were untimely challenges to alleged, apparent
solicitation improprieties. They also argued that some of Boeing's
challenges to the agency's evaluation of proposals were untimely because
Boeing was allegedly aware of the bases of these protest grounds during
the competition, but did not protest until after award and the firm's
receipt of a post-award debriefing.
Our Bid Protest Regulations contain strict rules for the timely submission
of protests. These timeliness rules reflect the dual requirements of
giving parties a fair opportunity to present their cases and resolving
protests expeditiously without disrupting or delaying the procurement
process. Peacock, Myers & Adams, B-279327, Mar. 24, 1998, 98-1 CPD para.
94 at 3-4; Professional Rehab. Consultants, Inc., B-275871, Feb. 28, 1997,
97-1 CPD para. 94 at 2. Under these rules, a protest based on alleged
improprieties in a solicitation that are apparent prior to closing time
for receipt of proposals must be filed before that time. 4 C.F.R. sect.
21.2(a)(1). Protests based on other than alleged improprieties in a
solicitation must be filed not later than 10 days after the protester knew
or should have known of the basis for protest, whichever is earlier. 4
C.F.R. sect. 21.2(a)(2). Our regulations provide an exception to this
general 10-day rule for a protest that challenges "a procurement conducted
on the basis of competitive proposals under which a debriefing is
requested and, when requested, is required." Id. In such cases, as here,
with respect to any protest basis which is known or should have been known
either before or as a result of the requested and required debriefing, the
protest cannot be filed before the debriefing date offered, but must be
filed not later than 10 days after the date on which the debriefing is
held. Id.; see Bristol-Myers Squibb Co., B-281681.12, B-281681.13,
Dec. 16, 1999, 2000 CPD para. 23 at 4.
We did not, and do not now, agree with the Air Force and Northrop Grumman
that Boeing's protest is a challenge to the ground rules established by
the RFP for this procurement. We find that Boeing, rather than objecting
to any of the RFP's requirements or evaluation criteria, is instead
protesting that the Air Force failed to reasonably evaluate proposals in
accordance with the RFP's identified requirements and evaluation
criteria.[40] We also do not agree with the agency and intervenor that,
because Boeing was informed during the competition of the agency's view of
the merits of its proposal and/or how the proposals were being evaluated,
Boeing was required to protest the agency's evaluation or evaluation
methodology prior to award and to the protester's receipt of its required
debriefing. Even where the protester is apprised of agency evaluation
judgments with which it disagrees or where it believes the evaluation is
inconsistent with the solicitation's evaluation scheme, our Bid Protest
Regulations require that these protest grounds be filed after the receipt
of the required debriefing.[41] See 4 C.F.R. sect. 21.2(a)(2); see also 61
Fed. Reg. 39039, 39040 (July 26, 1996) ("to address concerns regarding
strategic or defensive protests, and to encourage early and meaningful
debriefings," a protester shall not file an initial protest prior to its
required debriefing); Rhonda Podojil--Agency Tender Official, B-311310,
May 9, 2008, 2008 CPD para. 94 at 3 (application of debriefing exception
to A-76 competitions conducted on the basis of competitive proposals).
Key System Requirements Subfactor Evaluation
Boeing complains that the Air Force failed to evaluate the firms'
proposals under the key system requirements subfactor--the most important
subfactor of the mission capability factor--in accordance with the RFP's
identified evaluation scheme. As noted above, under this subfactor, the
agency was to assess the offerors' understanding of, and ability to meet,
the various SRD requirements. Boeing argues that the Air Force did not
reasonably consider the weighting assigned to the various SRD requirements
by the RFP in making its source selection,[42] even though they involve
the "major discriminators" referenced in justifying the award to Northrop
Grumman.[43] Boeing also asserts that the evaluation did not account for
the fact that the RFP specifically requested offerors to propose as many
of these "trade space" requirements as possible. In this regard, Boeing
complains that the agency assigned no credit for the fact that Boeing's
aircraft satisfied significantly more trade space SRD requirements than
did Northrop Grumman's under the key system requirements subfactor. See
Boeing's Post-Hearing Comments at 18.
The Air Force and Northrop Grumman deny that the agency failed to evaluate
the firms' proposals in accordance with the solicitation criteria. They
contend that the SSET performed an elaborate evaluation, "identifying
specifically how Boeing and [Northrop Grumman] met or exceeded KPP
thresholds, and how each traded, partially met or met desired requirements
(trade space)." See, e.g., Air Force's Memorandum of Law at 62. The Air
Force notes that the SSET identified potential strengths, which the SSAC
categorized, as relevant here, as "major discriminators" or
"discriminators offering less benefit." Id. The agency argues:
Because Boeing and [Northrop Grumman] offered significant trade space
and the benefits for each [SRD] reference line capability were not of
equal value, the SSAC gave positive consideration for additional
capability beyond the applicable threshold based upon the magnitude of
benefit to the Air Force. The offerors' approaches, their relative
benefits, advantages, and operational contributions for the five areas
within [the] Key System Requirements [subfactor] were evaluated by the
SSAC for accomplishing the comparative analysis. The SSAC deliberated
extensively, using expert technical, engineering, and operational
judgment to carefully evaluate the capabilities offered, consistent with
the RFP Measures of Merit and the priorities of KPP, KSA, and
non-KPP/KSAs. Both offerors proposed highly capable solutions to the
requirements that offered tremendous benefit above current Air Force
tanker capability.
Id. at 63-64.
An agency is obligated to conduct an evaluation consistent with the
evaluation scheme set forth in the RFP. FAR sect. 15.305(a); Serco, Inc.,
B-298266, Aug. 9, 2006, 2006 CPD para. 120 at 8. We recognize that
proposal evaluation judgments are by their nature often subjective;
nevertheless, the exercise of these judgments in the evaluation of
proposals must be reasonable and must bear a rational relationship to the
announced criteria upon which competing offers are to be selected. Systems
Research and Applications Corp.; Booz Allen Hamilton, Inc., B-299818 et
al., Sept. 6, 2007, 2008 CPD para. 28 at 11. In order for our Office to
perform a meaningful review, the record must contain adequate
documentation showing the bases for the evaluation conclusions and source
selection decision. Southwest Marine, Inc.; American Sys. Eng'g Corp.,
B-265865.3, B-265865.4, Jan. 23, 1996, 96-1 CPD para. 56 at 10.
Here, we agree that the SSET's evaluation identified and documented the
SRD requirements under which the firms' evaluated strengths and weaknesses
were assessed. Nevertheless, the record does not establish that the SSAC
and SSA, in considering those strengths and weaknesses, applied the
relative weights identified in the RFP for the various SRD requirements
(under which the KPPs were most important). Moreover, the record does not
show any consideration by the SSAC or SSA of the fact that Boeing's
proposal was evaluated as satisfying significantly more SRD requirements
than Northrop Grumman's.
For example, the record shows that most of Boeing's evaluated "major
discriminators" in the aerial refueling area were assessed under KPP
requirements, and conversely most of Northrop Grumman's evaluated "major
discriminators" were assessed under less important non-KPP/KSA
"requirements." Specifically, the SSAC identified as "major
discriminators" the following requirements that Boeing's aircraft
satisfied but Northrop Grumman's aircraft did not: (1) the capability to
[Deleted] (a KPP No. 1 objective); (2) the capability, [Deleted] (another
KPP No. 1 objective); (3) the capability to [Deleted] (another KPP No. 1
objective); and (4) the capability to [Deleted] (a "noteworthy"
non-KPP/KSA requirement).[44] See AR, Tab 55, PAR, at 13. The SSAC
identified as a "major discriminator" for Northrop Grumman that firm's
satisfaction of one KPP objective (KPP No. 2 objective for exceeding the
fuel offload unrefueled range), where Boeing also satisfied this objective
but to a lesser degree. The Air Force also identified as "major
discriminators" for Northrop Grumman under this area the firm's better air
refueling efficiency, larger boom envelope, and better offload and receive
rates, all of which were non-KPP/KSA requirements.
Although the record thus evidences that most of Boeing's evaluated "major
discriminators" were assessed under KPP requirements, and conversely most
of Northrop Grumman's evaluated "major discriminators" were assessed under
less important non-KPP/KSA requirements,[45] we have found no document in
the contemporaneous evaluation record that shows that the SSAC or SSA gave
any meaningful consideration to the weights that were to be assigned to
the various KPP, KSA, and other requirements. That is, the SSAC's briefing
slides to the SSA and its PAR do no more than identify the SRD
requirements for which the evaluated discriminators were assessed, but do
not evidence any consideration of the descending order of importance
assigned to these various SRD requirements.
In its briefing to the SSA, the SSAC merely reports each of the firms'
"advantages" without any analysis of whether or not Boeing's "advantages"
(which as indicated above are mostly derived from KPP objectives) were
entitled to greater weight than Northrop Grumman's advantages (which are
mostly derived from less important non-KPP/KSA requirements). See, e.g.,
AR, Tab 55, SSAC Recommendation Briefing to SSA, at 6-7 (aerial refueling
discriminators). Similarly, in the PAR, the SSAC duly reports the relative
order of importance that was to be assigned to the KPP, KSA and other
requirements, see AR, Tab 55, PAR, at 4, but there is no suggestion that
the assigned weights to these requirements were applied in any of the
SSAC's comparative analyses of the firms' evaluated discriminators. See,
e.g., id. at 13-14 (aerial refueling discriminators). Thus, although it is
true that the SSAC reported in the PAR that it considered the "priorities
of KPP, KSA, and non-KPP/KSAs," see id. at 29, the record does not provide
any evidence of such a weighing.
Similarly, in [the SSA's] selection decision, the SSA reports that the
evaluation of the "offerors' proposed capabilities and approaches against
the SRD requirements were made in the following order of importance: KPPs,
KSAs, and all other non-KPP/KSA requirements." See AR, Tab 54, Source
Selection Decision, at 5. Despite this reported recognition of the varying
weights assigned to the different SRD requirements, the SSA's decision
document does not evidence any consideration of the fact that Boeing's
assessed "major discriminators" were derived from requirements that were
identified as being more important than most of the requirements from
which Northrop Grumman's discriminators were derived. See id. at 5-7.
We agree with the Air Force that it is permissible to identify relative
strengths found under less important evaluation factors to be
discriminators for selection purposes, where there are lesser relative
differences favoring another proposal under more important evaluation
factors. However, we find no evidence in this record that any such
analysis, which considered the relative weight of the KPPs, KSAs and
non-KPP/KSA requirements, was performed here.
The Air Force also identified more "discriminators offering less benefit"
for Boeing's proposal than for Northrop Grumman's proposal in the aerial
refueling area. Specifically, the SSAC identified five such discriminators
for Boeing that were assessed under 13 different SRD requirements, and
only one such discriminator for Northrop Grumman that was assessed under 2
SRD requirements. As noted above, the RFP requested that offerors satisfy
as many of the "trade space" SRD requirements "as possible." See RFP, SOO
for KC-X SDD, at 2. Despite having solicited proposals that satisfy as
many SRD requirements as possible, there is no evidence in the record
showing that either the SSAC or the SSA accounted for the fact that
Boeing's proposal was evaluated as satisfying significantly more SRD
requirements in the aerial refueling area than did Northrop Grumman's
proposal. In fact, in deciding that Northrop Grumman had a significant
advantage in the aerial refueling area, the SSA did not even discuss the
fact that Boeing had more "discriminators offering less benefit" than did
Northrop Grumman, much less that Boeing satisfied far more SRD
requirements than did Northrop Grumman in this area.
As noted by the Air Force, the assignment of adjectival ratings and the
source selection should generally not be based upon a simple count of
strengths and weaknesses, but upon a qualitative assessment of the
proposals. See Kellogg Brown & Root Servs., Inc., B-298694.7, June 22,
2007, 2007 CPD para. 124 at 5. Such a qualitative assessment must be
consistent with the evaluation scheme, however. Here, although the RFP
expressly encouraged offerors to satisfy as many of the "trade space" SRD
requirements "as possible," see RFP, SOO for KC-X SDD, at 2, the record
shows no evidence that the Air Force gave any consideration to Boeing's
offer to satisfy significantly more trade space SRD requirements. This, in
our view, is not a matter of simply counting strengths and weaknesses, but
of evaluating the advantages and disadvantages of competing proposals
consistent with the RFP's evaluation scheme. See, e.g., Systems Research
and Applications Corp.; Booz Allen Hamilton, Inc., supra, at 14.
In short, our review of the record indicates that, as illustrated by the
aerial refueling area examples above, the Air Force failed to evaluate
proposals in accordance with the RFP's evaluation criteria.[46] That is,
the record evidences that the Air Force failed to assess the relative
merits of the offerors' proposals based upon the importance assigned to
the various SRD requirements by the RFP or to account for the fact that
Boeing proposed to satisfy far more SRD requirements than did Northrop
Grumman.
Fuel Offload versus Unrefueled Radius Range KPP Objective
Boeing protests that one of the key discriminators relied upon by the SSA
in [the SSA's] selection decision was contrary to the RFP's evaluation
criteria. This contention concerns the significant discriminator assessed
by the Air Force under the aerial refueling area of the key system
requirements subfactor. The assessed significant discriminator reflects
the conclusion that Northrop Grumman's proposed aircraft exceeded to a
greater degree than Boeing's aircraft a KPP objective to exceed the RFP's
identified fuel offload to the receiver aircraft versus the unrefueled
radius range of the tanker. The SSA noted in this regard that Northrop
Grumman's aircraft exceeded the threshold by [Deleted] percent at 1,000
nautical miles and by [Deleted] percent at 2,000 nautical miles, whereas
Boeing's aircraft exceeded the threshold by [Deleted] percent at
1,000 nautical miles and by [Deleted] percent at 2,000 nautical miles. AR,
Tab 54, Source Selection Decision, at 5. This was a key reason supporting
the SSA's determination that Northrop Grumman's proposed aircraft was more
advantageous than Boeing's aircraft in the aerial refueling area and was
superior overall to Boeing's. See id. at 6-7, 9, 19.
The RFP informed offerors that the agency would evaluate the offerors'
approach to meeting the SRD requirements related to aerial refueling,
which would include the fuel offload versus radius range. RFP
sect. M.2.2.1.2.a. With respect to fuel offload versus unrefueled range,
the RFP identified as a KPP threshold (a mandatory minimum requirement)
the range that offerors must satisfy to be found acceptable. See RFP, SRD
sect. 3.2.1.1.1.1. The RFP also identified as a KPP objective that the
offerors' "aircraft should be capable of exceeding" the threshold. See
RFP, SRD sect. 3.2.1.1.1.2. Finally, the RFP specifically informed
offerors that "[n]o consideration will be provided for exceeding KPP
objectives." RFP sect. M.2.2.1.1.a.
Boeing argues that section M.2.2.1.1.a. of RFP unambiguously prohibited
crediting Northrop Grumman for exceeding the fuel offload versus
unrefueled range objective to a greater extent than Boeing. Boeing asserts
that this limitation on providing credit for exceeding KPP objectives
"played an important role in shaping . . . how offerors designed and
selected the aircraft that was proposed to meet the stated SRD
requirements," see Protester's Comments at 14, and states that, had Boeing
known of the agency's desire for a larger aircraft which can carry more
fuel, it likely would have offered the agency an aircraft based upon the
777 aircraft platform.[47] See Protest at 2, 40.
The Air Force and Northrop Grumman respond that the agency "appropriately
found [Northrop Grumman's] superior ability to offload fuel at radius to
be a major discriminator and of operational benefit to the Air Force." Air
Force's Memorandum of Law at 70; see Northrop Grumman's Comments at 18-19.
In this regard, the agency and intervenor argue, despite the plain
solicitation language cited above by the protester, that the RFP, read as
a whole, indicated to offerors that the agency would consider, and award
credit for, the amount by which offerors proposed to exceed the fuel
offload versus unrefueled radius range chart identified in the KPP. In
this regard, the Air Force and Northrop Grumman argue that this KPP
objective did not identify an objective level, and therefore this
particular objective was "unbounded," such that unlimited credit could be
provided for exceeding this KPP objective. See, e.g., Air Force's Request
for Partial Dismissal at 19; Northrop Grumman's Post-Hearing Comments
at 102. The Air Force argues that:
[t]he RFP made clear that the Air Force desired maximum fuel offload at
radius because it described the objective in qualitative rather than
quantitative terms, thereby placing both offerors on notice that the
extent to which each offeror's proposed solution exceeded the threshold
could become a potential discriminator between the offerors.
Air Force's Memorandum of Law at 70. The agency also argues that, reading
this KPP objective to exceed the fuel offload versus radius range
threshold, see RFP, SRD sect. 3.2.1.1.1.2, in conjunction with the
non-KPP/KSA trade space requirement that the aircraft "should operate with
maximum fuel efficiency," see RFP, SRD sect. 3.2.1.1.1.3, offerors should
have known that the agency would be giving credit under this KPP objective
for the degree to which the offerors would exceed the charted KPP
threshold with no upward limits. See Air Force's Request for Partial
Dismissal at 17. Northrop Grumman contends that Boeing's reading of this
provision is inconsistent with the general nature of what the Air Force
sought, which Northrop Grumman argues was "a greater refueling capacity,
including the possibility of reducing the number of airplanes required to
complete a mission." Northrop Grumman's Comments at 27.
Where, as here, a dispute exists as to the actual meaning of a particular
solicitation provision, our Office will resolve the matter by reading the
solicitation as a whole and in a manner that gives effect to all its
provisions; to be reasonable, an interpretation of a solicitation must be
consistent with such a reading. Stabro Labs., Inc., B-256921, Aug. 8,
1994, 94-2 CPD para. 66 at 4.
We find from our review of the solicitation that the offerors were
unambiguously informed that their proposals would not receive additional
consideration or credit for exceeding a KPP objective. This is true
whether we look to the express provision itself, the meaning of which is
plain, or whether we view this restriction within the context of the whole
solicitation. The only reasonable interpretation of the KPP objective here
is that an offeror would be credited for meeting the fuel offload versus
unrefueled radius range objective if its aircraft exceeded the charted KPP
threshold, and that no additional credit would be provided for exceeding
the charted threshold amount to a greater degree than other proposed
aircraft.
Contrary to the Air Force's and Northrop Grumman's positions that this KPP
objective was "unbounded" because no finite number or level is stated as
part of the objective, the plain language of section M.2.2.1.2.a. of the
RFP unequivocally prohibited any consideration for exceeding the stated
KPP objective and the RFP did not suggest that the stated objective must
be finite or be at an objective level in order for this section to be
applicable. To read this provision as suggested by the intervenor and
agency would render meaningless section M.2.2.1.2.a, and be inconsistent
with identification of an objective for this KPP threshold. See Brown &
Root, Inc. and Perini Corp., a joint venture, B-270505.2, B-270505.3,
Sept. 12, 1996, 96-2 CPD para. 143 at 8 (a solicitation should be
reasonably read to give effect to all of its provisions). We do not find
such a reading reasonable.
The Air Force, as the drafter of the RFP, could have provided for
unbounded consideration of the degree to which offerors exceeded the fuel
offload versus unrefueled range, but did not. In fact, the last sentence
in section M.2.2.1.1.a. states that "[i]f there is no objective and,
depending on substantiating rationale, positive consideration will be
provided when the specified capability above the KPP threshold is viewed
as advantageous to the Government." Thus, according to the RFP,
"unbounded" credit could be given for exceeding the KPP where no KPP
objective is stated (depending on the substantiating rationale and when
advantageous to the government).[48] Indeed, the solicitation contained a
number of KPP thresholds that did not have corresponding KPP objectives,
see, e.g., RFP, SRD sect. 3.2.1.6.1. (KPP No. 4, Airlift Capability);
sect. 3.2.8 (KPP No. 8, Survivability), but that is not the case with
respect to this KPP threshold.
We also note that the RFP elsewhere specifically informed offerors of
other objectives for which their proposals could receive additional
consideration for exceeding objectives; that is, with respect to non-KPP
requirements, the RFP stated that the agency may give "additional
consideration if the offeror proposes to meet (or exceed if there is an
objective) the SRD threshold or requirement, depending on the
substantiating rationale." See RFP sect. M.2.2.1.1.b. In addition,
offerors were informed with regard to certain non-KPP objectives that they
should try to exceed the requirement by as much as possible. See, e.g.,
RFP, SRD sect. 3.2.10.1.5.2.2 ("The boom envelope should exceed the ATP-56
envelope as much as possible (OBJECTIVE).")
We also agree with Boeing that the RFP, read as whole, established a
complex set of trade-offs for offerors to consider in determining what
aircraft to propose to the agency, and we do not agree that "common sense"
mandates that "unbounded" refueling capabilities were being sought by the
RFP. [49] Although it is apparent that a larger aircraft could provide
greater refueling capabilities, there could be associated disadvantages
with respect to costs and space constraints. Thus, given that the RFP did
not establish any size requirements or limitations upon the aircraft that
could be proposed, the restriction on credit for exceeding this KPP
objective provided offerors with a key consideration in determining what
sort of aircraft to offer, as well as how to best structure their
proposals.
As indicated above, the Air Force and Northrop Grumman argued that
Boeing's protest of the agency's evaluation of the firms' proposal under
this KPP objective is untimely because it is actually a challenge to the
terms of the solicitation.[50] They base this argument upon their
contention that Boeing learned of the agency's interpretation from the
agency's briefings during the competition. However, we agree with Boeing's
contention that the agency's briefings supported Boeing's understanding
that no credit would be given for exceeding this KPP objective. For
example, in Boeing's mid-term briefing, the Air Force reported to Boeing
with regard to the aerial refueling area of the key system requirements
that, although its aircraft exceeded the fuel offload versus unrefueled
range and the agency identified by how much Boeing's aircraft exceeded the
range, its proposal was evaluated to have "met" the objective.[51] See AR,
Tab 129, Mid-term Briefing to Boeing, at 26. Similarly, in its pre-Final
Proposal Revision Briefing, Boeing was informed that its offer to exceed
the KPP threshold for this requirement was evaluated as having "met" the
objective.[52] See AR, Tab 135, Pre-Final Proposal Revision Briefing to
Boeing, at 30. Based on our review of the record, Boeing was not informed
in its briefings of the SSA's and SSAC's interpretation that the RFP
allowed "unbounded" credit to be given for exceeding the fuel offload
versus unrefueled radius range KPP objective, and only became aware of the
agency's interpretation from the redacted source selection decision that
was provided to Boeing at its post-award required debriefing.[53]
In sum, we find that a key discriminator relied upon by the SSA in making
[the SSA's] selection decision--that is, the assessment related to the KPP
objective to exceed the fuel offload versus unrefueled range--was not
consistent with the RFP. It is a fundamental principle of competitive
procurements that competitors be treated fairly, and fairness in
competitions for federal procurements is largely defined by an evaluation
that is reasonable and consistent with the terms of the solicitation. For
that reason, agencies are required to identify the bases upon which
offerors' proposals will be evaluated and to evaluate offers in accordance
with the stated evaluation criteria. See Competition in Contracting Act of
1984, 10 U.S.C. sect. 2305(a)(2)(A), (b)(1) (2000); FAR sections
15.304(d), 15.305(a); Sikorsky Aircraft Co.; Lockheed Martin Sys.
Integration-Owego, B-299145 et al., Feb. 26, 2007, 2007 CPD para. 45 at 4.
The Air Force did not fulfill this fundamental obligation here.
KC-30 Overrun and Breakaway Capability
Boeing also complains that the Air Force did not reasonably assess the
capability of Northrop Grumman's proposed aircraft to refuel all current
Air Force fixed-wing tanker-compatible aircraft using current Air Force
procedures, as required by a KPP No. 1 threshold under the aerial
refueling area of the key system requirements subfactor.[54] See RFP, SRD
sect. 3.2.10.1.1.9. Specifically, Boeing notes that current Air Force
refueling procedures require that the tanker aircraft be capable of
"overrun" and "breakaway" procedures when necessary, which would require
the tanker aircraft to fly faster than the receiver aircraft or quickly
accelerate during refueling.[55] Boeing's Second Supplemental Protest at
29. Boeing contends that the Air Force unreasonably determined that
Northrop Grumman's proposed aircraft would meet these requirements.
With regard to the overrun issue, the record shows that Northrop Grumman
was twice informed by the Air Force during discussions that the firm's
initially identified maximum operational airspeed of [Deleted] Knots
Indicated Air Speed (KIAS) would not be sufficient under current Air Force
overrun procedures to achieve required overrun speeds of [Deleted] KIAS
for various fighter aircraft, including the [Deleted], or [Deleted] KIAS
for the [Deleted].[56] See AR, Tab 184, EN NPG-MC1-003, at 2;
EN NPG-MC1-003a, at 1-2. Ultimately, Northrop Grumman informed the Air
Force that a [Deleted] limited the aircraft's operational speed, but that
Northrop Grumman proposed to include a [Deleted] to achieve the necessary
overrun speed.[57] See id., Northrop Grumman Response to EN NPG-MC1-003a,
at 2-7. The Air Force accepted Northrop Grumman's proposed solution as
satisfying this KPP threshold. HT at 628.
Boeing complains that Northrop Grumman's proposed solution of [Deleted] to
achieve overrun speed requires [Deleted], which is not consistent with the
Air Force's current procedures as is required by the KPP.[58] See Boeing's
Second Supplemental Protest at 29-32; Boeing's Comments at 64. Boeing also
argues that the agency did not note that Northrop Grumman qualified its
promise to increase its maximum operational airspeed in its EN response.
Specifically, Boeing points out that Northrop Grumman stated that,
[Deleted], the KC-30 had a maximum airspeed of [Deleted] KIAS, and not the
[Deleted] KIAS evaluated by the Air Force. See AR, Tab 184, Northrop
Grumman Response to EN NPG-MC1-003a, at 9.
At the hearing that our Office conducted in this protest, the Air Force
produced its SSET mission capability factor team chief to testify
regarding the agency's evaluation of the capability of Northrop Grumman's
aircraft to satisfy this KPP threshold.[59] This witness, in response to
direct examination, stated that the SSET found that [Deleted] would allow
the KC-30 to achieve the necessary airspeed to perform the required
overrun and breakaway procedures. Specifically, he testified that the SSET
was convinced that, by [Deleted], the KC-30 could achieve an operational
airspeed of [Deleted] KIAS, because Northrop Grumman had informed the
agency in its EN response that the commercial A330 aircraft had a maximum
"dive velocity"[60] of 365 KIAS and had been flight tested to a dive
velocity of [Deleted] KIAS, and that analysis had been done showing that
the A330 could even achieve a dive velocity of [Deleted] KIAS. HT at
626-27. The mission capability factor team chief testified that the SSET
evaluated Northrop Grumman's response to indicate that the [Deleted], see
HT at 637-38, and that in any event Air Force current procedures did not
require the use of the [Deleted] during aerial refueling operations. HT
at 638-39.
From this record, we cannot conclude that the Air Force reasonably
evaluated the capability of Northrop Grumman's proposed aircraft to
satisfy the KPP threshold requirement to refuel all current Air Force
fixed-wing tanker-compatible aircraft using current Air Force procedures.
The contemporaneous record, as explained by the hearing testimony, does
not establish that the Air Force understood Northrop Grumman's response in
discussions concerning its ability to satisfy the solicitation
requirements, nor does it demonstrate that the agency had a reasonable
basis upon which to accept Northrop Grumman's promises of compliance.
First, we agree with Boeing that the SSET erred in concluding that the
[Deleted] in tanker refueling operations was not a current Air Force
procedure.[61] See HT at 638, 735; Air Force's Post-Hearing Comments at
19. As noted above, the contemporaneous evaluation record shows that the
agency interpreted the solicitation requirement to comply with "current
[Air Force] procedures" to mean compliance with the procedures set forth
in the agency's flight manuals for the KC-135 and KC-10 tanker aircraft,
and expressly informed Northrop Grumman during discussions that the flight
manuals for the KC-135 and KC-10 established the current Air Force
procedures for refueling operations. See AR, Tab 184, EN NPG-MC1-003a, at
1, wherein the agency stated "[a]erial refueling procedures were contained
in T.O. 1-1C-1-3 and 1-1C-1-33 for the KC-135 and KC-10 respectively when
the RFP was released."[62] These manuals show that current Air Force
procedures provide that tanker pilots [Deleted] in refueling operations.
For example, the KC-135 manual under Section IV, Air Refueling Procedures,
warns tanker pilots that they "must be prepared to assume aircraft control
[Deleted]," and under Section V, Emergency Air Refueling Procedures,
instructs tanker pilots that in a breakaway situation, if a climb is
required, they must "[Deleted]." See AR, Tab 289, Flight Manual KC-135
(Tanker) Flight Crew Air Refueling Procedures, Supp. III, T.O. 1-1C-1-3,
Jan. 1, 1987, as revised Sept. 1, 2004, at [Deleted]. Similarly, the KC-10
flight manual provides under Section III, Air Refueling Procedures, that
the "[Deleted]." Id., Flight Manual, KC-10A Aircraft, Flight Crew Tanker
Air Refueling Procedures, USAF Series, T.O. 1-1C-1-33, Sept. 1, 2002, as
revised Jan. 31, 2005, at [Deleted]. In this regard, Boeing provided the
statement of a retired Air Force pilot, who had extensive experience as
both a KC-10 and KC-135 tanker pilot and had operated each aircraft as
both a tanker and a receiver in refueling missions; this individual
stated:
Refueling is more demanding and difficult for both tanker and receiver
aircraft if the tanker [Deleted]. For the tanker pilot, [Deleted]. For
the receiver pilot, [Deleted]. Due to these realities, existing
refueling guidelines dictate that [Deleted] should be used for refueling
under normal circumstances. [Citations omitted.] Beginning aerial
refueling [Deleted] should it become necessary, violates this policy. As
previously noted, [Deleted].
Boeing's Comments, attach. 14, Declaration of Retired Air Force Pilot, at
3-4. Although the Air Force and Northrop Grumman generally disagree with
Boeing's consultant that the Air Force's current procedures provide for
the [Deleted], neither the agency or intervenor have directed our
attention to anything in the KC-135 or KC-10 flight manuals or to any
other source that would establish that Boeing's view, which appears to be
reasonable on its face, is in error.
We also find unsupported the agency's conclusion that Northrop Grumman's
proposed solution of [Deleted] did not also involve [Deleted]. In its EN
response, Northrop Grumman informed the Air Force that 330 KIAS was the
normal design maximum operating velocity of the commercial A330 aircraft,
and that "selection of a [maximum operating velocity] drives overall
design characteristics of the aircraft, specifically aerodynamic and
structural design limits, handling quality definition, and thrust." See
AR, Tab 184, Northrop Grumman Response to EN NPG-MC1-003a, at 2. Northrop
Grumman explained that its [Deleted] limited the aircraft to its maximum
operating velocity, but that the firm could [Deleted] to exceed the
maximum operating velocity. The awardee then stated "three cases . . . to
illustrate the performance of the KC-30 with and without [Deleted]." Id.
at 3. The three cases that Northrop Grumman identified and separately
described were (1) KC-30 [Deleted]; (2) KC-30 [Deleted]; and (3) KC-30
[Deleted], which indicated that the KC-30 could only meet the overrun
requirement under the third case where both the [Deleted]. Id. at 3-6.
The SSET read, as described by the testimony of its mission capability
factor team chief, Northrop Grumman's EN response to describe a "fourth
case" (although not identified as such) under the "third case" heading,
but located at the end of that section, where, the agency contends, the
KC-30's [Deleted] but the [Deleted]. See HT at 664. However, we are unable
to accept such a reading of Northrop Grumman's EN response. It ignores the
logical structure of Northrop Grumman's response to the agency, which only
identified and described three cases. Moreover, nowhere in its response to
the agency's EN does Northrop Grumman suggest a "fourth case" where the
[Deleted]; rather, the only reference to both the [Deleted] in the third
case expressly states that the [Deleted] ("Case 3: KC-30 [Deleted]").[63]
See AR, Tab 184, Northrop Grumman Response to EN NPG-MC1-003a, at 6. In
any event, given the uncertainty surrounding the agency's interpretation
of Northrop Grumman's solution to a matter the agency believed could
render the firm's proposal unacceptable, see HT at 625, 649, this is
something the agency should have continued to clarify and resolve during
discussions with the firm.[64]
Even apart from the agency's apparent misreading of Northrop Grumman's EN
response and disregard of the current Air Force procedure to [Deleted],
the record does not establish that the agency had a reasonable basis for
concluding that Northrop Grumman's proposed solution would allow its
aircraft to obtain the requisite overrun airspeeds to satisfy this KPP
threshold. The witness that the Air Force produced to support its
arguments on this point testified that the SSET had concluded that the
KC-30 had the "inherent capability" of reaching airspeeds greater than
[Deleted] KIAS (the aircraft's certified maximum operational airspeed)
based upon the far greater airspeed ([Deleted] KIAS) identified by the
firm for its certified dive velocity.[65] See HT at 624-28; Air Force's
Post-Hearing Comments at 17-18. In this regard, the SSET apparently
believed that simply [Deleted] would enable the aircraft to achieve its
indicated dive velocity airspeed as its operational airspeed.
Although the SSET mission capability factor team chief repeatedly
testified that the dive speed indicated that the aircraft would have the
structural ability to fly at the dive speed limitation, see, e.g., HT at
674, he also admitted under cross examination that he did not know what
the relationship was between maximum operating airspeed and design dive
speed:
Q: What's your understanding of what the general margin is between maximum
operational velocity and dive velocity?
A: I'm not aware.
Q: Was there somebody on your team that was advising you about what the
general margin is or difference is between maximum operational velocity
and dive velocity?
A: There could have been. We had advisors for handling qualities.
Q: I know you had advisors. I'm asking you, were there any advisors who
actually helped you with understanding the difference between dive
velocity and maximum operational velocity?
A: They did not help me, no.
Q: Did they help the team?
A: Not that I'm aware of.
HT at 669-70. The SSET mission capability factor team chief's (and
presumably the SSET's) lack of knowledge concerning the relationship
between maximum operating airspeed and design dive airspeed[66] is
particularly troubling given the definition of maximum operating limit
speed in FAA's regulations:
The maximum operating limit speed . . . is a speed that may not be
deliberately exceeded in any regime of flight (climb, cruise, or
descent), unless a higher speed is authorized for flight test or pilot
training operations. [The maximum operating limit speed] must be
established so that it is not greater than the design cruising speed
. . . and so that it is sufficiently below [dive speed and velocity] to
make it highly improbable that the latter speeds will be inadvertently
exceeded in operations.
14 C.F.R. sect. 25.1505.
In sum, despite having identified, as an issue for the hearing, the
capability of Northrop Grumman's proposed aircraft to satisfy the airspeed
requirements of this KPP threshold, we have been presented with no
testimony or documented analysis that explains why simply [Deleted] on the
KC-30 would ensure that the proposed aircraft would achieve required
overrun airspeeds that are in excess of its FAA certified maximum
airspeed.[67] Furthermore, neither the Air Force nor Northrop Grumman has
directed us to any documentation establishing that the agency analyzed
what would be entailed in designing the KC-30 to exceed the certified
maximum operational airspeed limit.[68] Given Northrop Grumman's
recognition in its EN response that selection of the maximum operational
airspeed limit "drives overall design characteristics of the aircraft,
specifically the aerodynamic and structural design limits, handling
quality definition, and thrust," see AR, Tab 184, Northrop Grumman
Response to EN NPG-MC1-003a, at 2, it would seem apparent that some design
and FAA re-certification efforts could be necessary.
Boeing also complains that the Air Force did not reasonably evaluate the
capability of Northrop Grumman's aircraft to initiate emergency breakaway
procedures when refueling the [Deleted].[69] Current Air Force procedures,
as reflected by the KC-135 flight manual, specifies that the tanker will
refuel the [Deleted] at an airspeed of [Deleted] KIAS, see AR, Tab 289,
Flight Manual KC-135 (Tanker) Flight Crew Air Refueling Procedures, Supp.
III, T.O. 1-1C-1-3, Jan. 1, 1987, as revised Sept. 1, 2004, at [Deleted],
and Northrop Grumman's EN response indicates that the KC-30's airspeed is
limited to [Deleted] KIAS with the aircraft's [Deleted]. See AR, Tab 184,
Northrop Grumman Response to EN NPG-MC1-003a, at 9. Boeing contends,
citing the statement of its former tanker/receiver pilot consultant, that
there is insufficient margin between airspeed at which [Deleted] are
refueled and the KC-30's operational airspeed limit during refueling (a
[Deleted]-KIAS margin) to allow for emergency breakaway maneuvers. See
Boeing's Comments, attach. 14, Declaration of Retired Air Force Pilot, at
3-4.
As was true with respect to whether the KC-30 can satisfy the current Air
Force procedures with respect to overrun airspeed, there is no
documentation in the record setting forth an analysis of whether Northrop
Grumman's proposed aircraft has sufficient operational airspeed when
refueling the [Deleted] to initiate an emergency breakaway procedure. The
agency's counsel provided a proffer at the hearing that the SSET's
analysis of whether the KC-30 was capable of performing a breakaway
maneuver with the [Deleted] was contained in the SSET's Final Evaluation
Summary Report for Northrop Grumman. See HT at 784; see AR, Tab 215,
Evaluation Summary Report for Northrop Grumman, at 3. Neither the page
referenced by agency counsel or any other part of that document contains
any analysis of whether Northrop Grumman's proposed aircraft can perform a
breakaway procedure while refueling the [Deleted]; rather, the page
referenced by agency counsel merely states that "[t]he Offeror has
substantiated the ability to deliver a KC-X aircraft that meets (minimum
requirement) all KPP thresholds associated with aerial refueling," and
provides no reasons or analysis supporting this conclusion. AR, Tab 215,
Evaluation Summary Report for Northrop Grumman, at 3.
Although the SSET mission capability factor team chief was examined
extensively about the SSET's consideration of the KC-30's ability to
perform breakaway procedures, he recalled little about the SSET's
discussions in this regard. His testimony does indicate, however, that the
SSET accepted that the KC-30's maximum operational airspeed when refueling
([Deleted]) was [Deleted] KIAS, and that the SSET apparently believed
that, to initiate the emergency breakaway procedure, with Northrop
Grumman's proposed aircraft, the tanker would have to start accelerating
and [Deleted] simultaneously. See HT at 706. During cross examination, the
SSET mission capability factor team chief admitted that he did not know
how long it would take [Deleted] Northrop Grumman's proposed [Deleted] or
what the procedure was for [Deleted], nor was he aware of whether this was
ever analyzed by the agency in its evaluation.[70] HT at 685-87, 707.
In sum, we conclude that the record does not demonstrate that the agency
reasonably determined that Northrop Grumman's proposed aircraft would be
able to refuel all current Air Force fixed-wing tanker-compatible receiver
aircraft in accordance with current Air Force procedures as was required
by this KPP No. 1 threshold.
Operational Utility Area
Boeing also complains that the Air Force unreasonably evaluated the firms'
proposals in the operational utility area under the key system
requirements subfactor. The RFP provided that evaluation of this area
would consist of an assessment of the offeror's approach to meeting (or
exceeding, where appropriate) SRD requirements, "including the following:
aircraft maneuverability, worldwide airspace operations,
communication/information systems (including Net-Ready capability), treaty
compliance support, formation flight, intercontinental range, 7,000-foot
runway operations, bare base airfield operations, and growth provisions
for upgrades." RFP sect. M.2.2.1.2.c. Boeing contends that its proposal
should have been found technically superior to Northrop Grumman's in this
area, and not essentially equal, as the SSA and SSAC concluded. See AR,
Tab 54, Source Selection Decision, at 8.
As noted above, the SSET found that both offerors satisfied the three KPP
thresholds identified in this area, and partially met the one KPP
objective identified; the SSET also found that both offerors met all
associated KSA thresholds and objectives. See AR, Tab 55, PAR, at 20-21.
The SSAC also identified two "major discriminators" in each of the firms'
respective proposals; the discriminators for Boeing were the firm's
(1) [Deleted] and (2) [Deleted], and for Northrop Grumman were (1) the
ability of the KC-30 to operate from a 7,000-foot runway carrying more
fuel than the KC-767, and (2) the KC-30's longer ferry range compared to
the KC-767's. Id. Boeing's two "major discriminators" were assessed under
17 different SRD requirements, while Northrop Grumman's two discriminators
were assessed under only 2 SRD requirements. The SSAC also identified a
number of "discriminators offering less benefit" for each firm: six such
discriminators for Boeing assessed under 19 SRD requirements, and five
such discriminators for Northrop Grumman assessed under 6 SRD
requirements. Id. at 22-24.
Here, too, as we described above with respect to the aerial refueling
area, the record does not evidence that the SSAC and SSA, in determining
that the firms' proposals were essentially equal in the operational
utility area, gave any consideration to the fact that Boeing's proposal
was evaluated as satisfying more SRD requirements than Northrop Grumman's
in this area, as was sought by the RFP. Given this failure by the SSAC and
SSA to address Boeing's apparent advantage in meeting more SRD
requirements than Northrop Grumman, we conclude that the agency's
evaluation and selection decision was unreasonable in this regard.
Boeing also complains that the agency conducted misleading discussions
with Boeing with respect to whether Boeing had fully satisfied the KPP No.
7 objective, Net-Ready Capability. RFP, SRD sect. 3.2.4.1.1; app. A,
Net-Ready Capability KPP, at 3. The KPP No. 7 objective provides that the
offeror's "system should be capable of accomplishing all operational
activities identified in Table 5." RFP, SRD, app. A, at 4. Table 5 of the
appendix identified a number of information exchange requirements. Id.
at 15-25.
Specifically, Boeing complains that at its mid-term briefing it was
informed of an uncertainty regarding the firm's net ready capability, see
AR, Tab 129, Mid-Term Briefing to Boeing, at 77, and that ultimately the
firm responded to an EN concerning the firm's System Requirements Matrix
and System Specification with respect to complying with the SRD
requirements for KPP No. 7. See AR, Tab 210, Boeing Response to EN
BOE-MC1-041. Boeing believed that its EN response charted how its proposal
met the KPP No. 7 thresholds and objective in total, see Boeing's Comments
at 29, and during the firm's Pre-Final Proposal Revision Briefing the Air
Force informed Boeing that the firm "met" both the KPP thresholds and the
objective requirements for KPP No. 7. See AR, Tab 135, Boeing's Pre-Final
Proposal Revision Briefing, at 57. Accordingly, Boeing made no further
revisions to its proposal in this area. Boeing's Second Supplemental
Protest at 53. The Air Force, however, changed its evaluation rating of
this aspect of Boeing's proposal to "partially met" the KPP objective (the
same rating that Northrop Grumman received) without further notice to
Boeing.[71] Boeing contends that the Air Force's misleading discussions
prevented the firm from addressing the agency's concerns with respect to
this objective.
The Air Force does not dispute that it informed Boeing during discussions
that the firm had satisfied all of the thresholds and the objective under
KPP No. 7, but contends that at the time it later determined that Boeing
had not fully satisfied this objective, discussions had already been
closed. See Second Supplemental COS at 77. The agency argues that, in any
event, it was under no obligation to inform Boeing of the changed
evaluation rating associated with this objective because the objective
"constituted trade space," the absence of which would not be a deficiency
or weakness. Agency Memorandum of Law at 131.
We do not agree with the Air Force that the agency was permitted, after
informing Boeing that its proposal fully met this objective, to change
this evaluation conclusion without affording Boeing the opportunity to
satisfy this requirement. It is a fundamental precept of negotiated
procurements that discussions, when conducted, must be meaningful,
equitable, and not misleading. See 10 U.S.C. sect. 2305(b)(4)(A)(i); AT&T
Corp., B-299542.3, B-299542.4, Nov. 16, 2007, 2008 CPD para. 65 at 6.
Here, by informing Boeing prior to the submission of the firm's final
proposal revision that it satisfied all aspects of KPP No. 7, the Air
Force deprived the firm of the opportunity to further address these
particular requirements. See AT&T Corp., supra, at 12; see also Bank of
Am., B-287608, B-287608.2, July 26, 2001, 2001 CPD para. 137 at 13.
In contrast, the Air Force informed Northrop Grumman prior to the
submission of that firm's final proposal revision that it had only
partially met this KPP objective, which permitted that firm the
opportunity to further address the KPP objective requirements. See AR, Tab
205, Northrop Grumman's Pre-Final Proposal Revision Briefing, at 61.
Moreover, Boeing submitted its final submission addressing this KPP
objective several months prior to the pre-FPR briefing, and, as indicated
above, the agency actually reopened discussions on other subjects after
submission of the FPRs and obtained revised FPRs. Boeing's Protest at 66;
Boeing's Second Supplemental Protest at 53. In short, the Air Force misled
Boeing when the agency advised the firm that it met this objective, but
later determined that Boeing did not fully meet this objective, and did
not reopen discussions with Boeing on this issue. The Air Force also
treated the firms unequally when it provided Northrop Grumman, but not
Boeing, with continued discussions on this same objective. It is axiomatic
that procuring agencies may not conduct discussions in a manner that
favors one offeror over another. See FAR sect. 15.306(e)((1); Chemonics
Int'l, Inc., B-282555, July 23, 1999, 99-2 CPD para. 61 at 8-9.
We also find a reasonable possibility that Boeing was prejudiced by the
Air Force's misleading and unequal discussions, given the greater weight
that KPPs were supposed to receive in the agency's evaluation. In this
regard, if Boeing had been evaluated as fully satisfying this KPP
objective, which was the only KPP objective in the operations utility
area, it could well have been considered to be superior in this area to
Northrop Grumman, which was evaluated as only partially satisfying this
KPP objective.
Other Key System Requirements Subfactor Issues
Boeing also protests the Air Force's conclusion in the aerial refueling
area that Northrop Grumman's proposed larger boom envelope (relative to
that offered by Boeing) offered a meaningful benefit to the Air Force. See
AR, Tab 55, PAR, at 14. From our review of the record, including hearing
testimony on this issue, we do not find a basis to object to the Air
Force's judgment that Northrop Grumman had offered a larger boom envelope
and that this offer provided a measurable benefit.[72]
Boeing also challenges the Air Force's evaluation judgment in the airlift
area that Northrop Grumman's proposed aircraft offered superior cargo,
passenger, and aeromedical evacuation capability than did Boeing's
aircraft. From our review of the record, including the hearing testimony,
we see no basis to conclude that the Air Force's evaluation that Northrop
Grumman's aircraft was more advantageous in the airlift area is
unreasonable.
Product Support Subfactor Evaluation
Boeing also complains that the Air Force misevaluated Northrop Grumman's
proposal under the product support subfactor. This subfactor required the
agency to evaluate the "offeror's proposed product support approach for an
efficient, effective and comprehensive support program for the service
life of the KC-X fleet." RFP sect. M.2.2.3. Specifically, Boeing contends
that the Air Force improperly ignored Northrop Grumman's refusal to commit
to providing the required support necessary to allow the agency to achieve
initial organic depot-level maintenance capability within the time
required by the RFP, namely, within 2 years after delivery of the first
full-rate production aircraft.[73] Boeing's Post-Hearing Comments at 84.
The Air Force evaluated Boeing's and Northrop Grumman's proposals to be
essentially equal under the product support subfactor.[74] See AR, Tab 54,
Source Selection Decision Document, at 10; Tab 55, PAR, at 34.
Offerors were informed that the long-term support concept for the KC-X
program was for two levels of organic maintenance: organization level and
depot level, and that a program objective was a product support approach
that effectively addressed all the integrated support elements, including
"[t]imely, cost effective transition to organic support." RFP, SOO for
KC-X SDD, at 1-2. One of the specific minimum program tasks required by
the SOO with regard to "logistics" was for the contractor to
[p]lan for and support the Government to achieve an initial organic
[depot]-level maintenance capability in accordance with the [Source of
Supply Assignment Process] for core-designated workloads, at a minimum,
within two years after delivery of the first full-rate production
aircraft.
Id. at 14; see also RFP, SOO for KC-X LRIP and Full-Rate Production, at
1.[75] The RFP instructed offerors to ensure that their proposed
contractual statements of work (SOW) would "conform to the Government's
SOO" and that "[t[he proposed SOWs shall define the tasks required for the
KC-X program, ensuring all minimum requirements of the Government provided
SOOs and preliminary [work breakdown structure] have been addressed." See
RFP sections L.2.1, L.8.3.7.2.
The Air Force recognized in its evaluation that, although Northrop Grumman
promised to provide the necessary planning and support for the agency to
achieve an initial depot-level maintenance capability, the firm did not
commit to providing this required support within 2 years after delivery of
the first full-rate production aircraft, as required by the RFP. Thus, at
the mid-term briefing, Northrop Grumman was informed that the timing of
the firm's proposed depot level maintenance support was "unclear," see AR,
Tab 199, Northrop Grumman's Mid-Term Briefing, at 134, and then again at
the pre-final proposal briefing, Northrop Grumman was informed that the
agency had assigned it a weakness for its failure "to include the time
frame for initial organic depot standup in Offeror's Production SOW (SOO
states within two years after delivery of the first full-rate production
aircraft)."[76] See AR, Tab 205, Northrop Grumman's Pre-Final Proposal
Revision Briefing, at 141. Northrop Grumman did not resolve its failure to
commit to the 2-year timeframe for this product support requirement during
the procurement. In the firm's final proposal revision, Northrop Grumman
stated in one place that resolution of this "timing issue will be
determined in coordination with the Government at contract award" and, in
another place, that action to "resolve government identified weaknesses"
would occur "after contract award." See AR, Tab 187, Northrop Grumman's
Final Proposal Revision, KC-X Program Summary Document, at 2-3.
In its final evaluation, the SSET evaluated Northrop Grumman's refusal to
commit to providing these product support services within the 2-year
timeframe as a weakness. AR, Tab 46, SSET Final Briefing to SSAC and SSA,
at 360, 362. The SSAC concluded that this was an "administrative
documentation oversight" because Northrop Grumman had promised to provide
the required services and its "cost/schedule documentation is consistent
with standing up depot capability within two years of delivery of the
first full-rate production aircraft." AR, Tab 55, PAR, at 34. The SSA
concurred with the SSAC that this was "merely an administrative
oversight." AR, Tab 54, Source Selection Decision, at 10.
We agree with Boeing that Northrop Grumman's refusal to commit to the
required 2-year timeframe within which to provide these depot-level
maintenance planning and support services cannot be reasonably viewed as
an administrative or documentation oversight. As noted above, Northrop
Grumman was clearly informed several times by the Air Force of the
agency's concern that the firm had not committed to the required
timeframe, and Northrop Grumman responded that it was not resolving this
failure before award. Although throughout the protest and during the
hearing, the agency steadfastly asserted that Northrop Grumman's failure
to so commit was an "oversight,"[77] see, e.g., Air Force's Memorandum of
Law at 151-53, in its post-hearing rebuttal comments, the agency admitted
for the first time that Northrop Grumman's "omission" appeared to be a
conscious decision. See Air Force's Post-Hearing Rebuttal Comments at 9.
Northrop Grumman also finally admits in its rebuttal comments that its
decision to not commit to the 2-year timeframe was "intentional."[78]
Northrop Grumman's Post-Hearing Rebuttal Comments at 29 n.13.
The Air Force and Northrop Grumman argue, however, that, apart from
Northrop Grumman's refusal to commit to the 2-year timeframe, Northrop
Grumman committed generally and specifically to performing the planning
and support services solicited by the RFP in its proposal and proposal
revisions, and that the firm would otherwise be obligated to perform the
required services under whatever schedule the agency chooses. See, e.g.,
Air Force's Post-Hearing Rebuttal Comments at 11; Northrop Grumman's
Post-Hearing Rebuttal Comments at 29. The parties disagree as to whether
Northrop Grumman's proposal demonstrates the ability to provide the
required services within 2 years of delivery of the first full-rate
production aircraft, and based on our review of Northrop Grumman's
proposal and revisions, we find that it is far from clear whether or not
Northrop Grumman's proposed schedule establishes that it would perform
these services within the 2-year time frame.
Whether or not Northrop Grumman's proposed schedule accommodates providing
these product-support services within the 2-year timeframe misses the
point, however. By explicitly refusing to contractually commit to the
2-year timeframe for providing these services in the SOW as it was
repeatedly requested to do, we think that Northrop Grumman has taken
exception to this solicitation requirement. See C-Cubed Corp., B-272525,
Oct. 21, 1996, 96-2 CPD para. 150 at 3. It is a fundamental principle in a
negotiated procurement that a proposal that fails to conform to a material
solicitation requirement is technically unacceptable and cannot form the
basis for award. See TYBRIN Corp., B-298364.6; B-298364.7, Mar. 13, 2007,
2007 CPD para. 51 at 5.
The Air Force and Northrop Grumman also argue that the 2-year requirement
is not a material solicitation provision. However, their arguments in this
regard are belied by the agency's contemporaneous actions during the
procurement and the testimony of the SSET product support subfactor team
chief. As noted above, the agency repeatedly raised this matter with
Northrop Grumman during discussions in an unsuccessful effort to have the
firm commit to this solicitation requirement, and Northrop Grumman just as
steadfastly refused to commit. Moreover, the SSET product support
subfactor team chief identified the purpose or intent of this particular
SOO requirement as follows: "It was a binding function to bind it to a
specific time line," see HT at 1216, and that this 2-year requirement was
"an important requirement." HT at 1245. We find, from our review of the
record, that the requirement to plan for and support the agency's
achieving an initial organic depot-level maintenance capability within 2
years after delivery of the first full-rate production aircraft was a
material requirement.
In sum, the Air Force improperly accepted Northrop Grumman's proposal,
where that proposal clearly took exception to a material solicitation
requirement.[79]
System Integration and Software Subfactor Evaluation
Boeing also complains that, although both firms were evaluated as
acceptable but with a moderate risk under the system integration and
software subfactor, the Air Force should have viewed Northrop Grumman's
proposal as riskier than Boeing's. See Boeing's Comments at 100-01. The
Air Force states that it viewed both firms' offers of substantial software
reuse to be risky, and this, with other weaknesses the agency noted in
each firm's proposal under this subfactor, resulted in an assignment of a
moderate risk. We see no basis in this record to object to the agency's
evaluation under this subfactor.
Program Management Subfactor Evaluation
Boeing also complains that the Air Force did not reasonably assess
schedule or cost risks under the program management subfactor with respect
to Northrop Grumman's proposed changes during contract performance in its
production approach and production lines. See Boeing's Comments at 75-96.
The Air Force contends that Northrop Grumman agreed to appropriate
mitigation measures that supported the agency's conclusion that the firm
presented low cost or schedule risk under the program management
subfactor. From our review of the record, including hearing testimony on
this issue, we do not find a basis to object to the Air Force's evaluation
of Northrop Grumman's proposal under this subfactor.
Past Performance Factor Evaluation
Boeing also challenges the Air Force's evaluation of Boeing's and Northrop
Grumman's past performance, arguing that the agency's assessment of the
relevance of contracts to be considered was unreasonable, that the agency
treated the offerors disparately, and that the past performance evaluation
judgments were not adequately documented. See Boeing's Comments at 148. We
find from our review of the record no basis to object to the Air Force's
past performance evaluation, under which both firms' past performance
received a satisfactory confidence rating. We also find no basis to
question the SSA's judgment that, despite the equal confidence ratings
that the firms received under this factor overall, Northrop Grumman's
higher "satisfactory confidence" rating, as compared to Boeing's "little
confidence" rating, under the program management area was a reasonable
discriminator. The Air Force evaluated Boeing's past performance as
marginal in this area based on the agency's judgments as to Boeing's
program management performance under the [Deleted] contract, the [Deleted]
contract, and the [Deleted] contract. We have no basis, on this record, to
find the Air Force's judgment unreasonable.
IFARA Factor Evaluation
Boeing also challenges the Air Force's evaluation of the firms' proposals
under the IFARA evaluation factor. Boeing complains that the Air Force
unreasonably concluded that Northrop Grumman's proposed aircraft was
superior to Boeing's under this factor based only upon the fleet
effectiveness value and without considering evaluated major insights and
observations, which Boeing asserts favored its proposal. See Boeing's
Comments at 146. Our review of the record discloses that the SSAC and SSA
did consider the agency's evaluated insights and observations in their
evaluation of the firms' proposals under this factor, and therefore find
no basis to object to the agency's evaluation.
Evaluation of MILCON Costs
Boeing also complains that the Air Force did not reasonably evaluate the
firms' cost/price proposals in accordance with the RFP. As noted above,
the solicitation provided that the Air Force would calculate an MPLCC
estimate for each offeror, which reflected the agency's independent
estimate of all contract, budgetary, and other government costs associated
with all phases of the aircraft's life cycle from SDD through production
and deployment and O&S; MILCON costs were specifically identified as a
cost that the agency would evaluate in calculating the firms' MPLCCs. See
RFP sect. M.2.5.2. Boeing contends that the Air Force's evaluation of
MILCON costs greatly understated the difference between the firms' MILCON
costs and that Northrop Grumman's much larger and heavier aircraft would
have correspondently higher MILCON costs. See Boeing's Comments at 110-18;
Boeing's Post-Hearing Comments at 117-18.
The Air Force disputes Boeing's complaint, contending that it reasonably
assessed the likely life cycle costs associated with each firm's proposed
aircraft. In this regard, the agency states that, because it did not know
at which bases ("beddown sites") the new KC-X aircraft would be assigned,
it conducted site surveys at four airbases ([Deleted] Air Force Base
(AFB), [Deleted] AFB, [Deleted] AFB, and [Deleted] AFB) to determine what
military construction would be required at those bases for the offerors'
proposed aircraft. The agency then extrapolated those results to six other
airbases to calculate the agency's MILCON costs for the offerors. Air
Force's Memorandum of Law at 221-22; Air Force's Post-Hearing Comments
at 120-22. As indicated above, the agency added $[Deleted] billion in
MILCON costs to Boeing's MPLCC and $[Deleted] billion in MILCON costs to
Northrop Grumman's MPLCC. AR, Tab 55, PAR, at 40-43.
An agency's life cycle cost evaluation, like other cost analyses, requires
the exercise of informed judgment concerning the extent to which proposed
costs or prices represent a reasonable estimation of future costs. Our
review of the agency's cost analysis is limited to the determination of
whether the evaluation was reasonable and consistent with the terms of the
RFP. See Cessna Aircraft Co., B-261953.5, Feb. 5, 1996, 96-1 CPD para. 132
at 21. The agency's analysis need not achieve scientific certainty;
rather, the methodology employed must be reasonably adequate to provide
some measure of confidence that the agency's conclusions about the most
probable costs under an offeror's proposal are realistic in view of other
cost information reasonably available to the agency at the time of its
evaluation. See Information Ventures, Inc., B-297276.2 et al., Mar. 1,
2006, 2006 CPD para. 45 at 7.
As a threshold matter, the Air Force admits that in "defending this
protest" it discovered five errors in its assessment of MILCON costs,
which, when corrected, would result in Boeing displacing Northrop Grumman
as the offeror with the lowest evaluated MPLCC. Specifically, the Air
Force states that it underestimated Northrop Grumman's MILCON costs by
$122.5 million, and overestimated Boeing's costs by $3.3 million. After
correction of these $125.8 million in errors, Boeing's MPLCC would be
$108.041 billion and Northrop Grumman's would be $108.133 billion.[80] Air
Force's Memorandum of Law at 201-02.
Here, the record shows that the agency's MILCON cost evaluation was
otherwise flawed. In this regard, the RFP contemplated that the agency's
MILCON cost evaluation would be based upon "the offeror's proposed KC-X
aircraft solution," see RFP sect. M.2.5.2.4, which is consistent with the
rule that an agency must consider an offeror's proposed approach in
estimating the likely costs associated with that offeror's proposal. See
Hughes STX Corp., B-278466, Feb. 2, 1998, 98-1 CPD para. 52 at 8. The
record shows, however, that the agency's evaluation of MILCON costs was
based upon site surveys that were conducted prior to the receipt of
proposals in response to the RFP. HT at 472-73, 1293; Air Force's
Post-Hearing Comments at 120. Admittedly, the agency's site surveys were
based upon the size and dimensions of the A330-200 and 767-200, the
commercial aircraft from which the offerors' proposed KC-X aircraft were
derived. See, e.g., AR, Tab 297, Site Survey Report for [Deleted] Air
Force Base, at 3. However, it is equally clear that the Air Force could
not and did not evaluate MILCON costs associated with some aspects of the
offerors' proposed aircraft because the site surveys were conducted before
the receipt of proposals, and no further evaluation of the additional
MILCON costs for the improvements/changes necessary to support each of
these particular aircraft was performed after the proposals were received.
For example, although the Air Force recognizes that there will be a "need
for seat storage" associated with the KC-X aircraft, the survey teams were
unable to assess the likely MILCON costs associated with this need
because, at the time of the surveys, the agency did not know the number of
seats associated with the firms' respective aircraft.[81] See Air Force's
Post-Hearing Comments at 127. Accordingly, at [Deleted] AFB, the team
assumed that the offerors' aircraft had seating capacities similar to that
of the KC-10 and, on this basis, concluded that the facilities at
[Deleted] AFB were adequate. HT at 497. The KC-10, however, has only 75
seats, which is far less than the [Deleted] seats carried by the KC-30 and
less than the [Deleted] seats carried by the KC-767. Similarly, at
[Deleted] AFB, the survey team assigned no MILCON costs associated with
seat storage because it determined, without any actual knowledge of the
number of seats the proposed aircraft would carry, that there would be
adequate storage available. Air Force's Post-Hearing Comments at 128. At
[Deleted] AFB, the survey team concluded that there would be insufficient
storage space to accommodate the seats and that an additional storage
facility would need to be constructed; the cost of this facility
($[Deleted] million) was estimated to be the same for both offerors
because the team did not know how many seats the aircraft carried and
therefore "assigned a seat requirement the same for both aircraft." HT
at 499-500.
As another example, the survey team at [Deleted] AFB noted that the
battery shop at the base may not have enough capability to service the
batteries for the KC-X aircraft, if the new aircraft used different
batteries from the other aircraft (the KC-135 and C-17 aircraft) at the
base. AR, Tab 297, Site Survey Report for [Deleted] AFB, at 13. The team
assigned no cost for this concern:
Battery concern was noted because we did not know what the requirements
were for the two different batteries, since we did not know the battery
type on the A330, does that mean you only need to be separated by
certain amount of spaces in the facility? Could you put up a wall? Would
you actually need a whole new facility? So we didn't have enough detail
to know if we needed to build anything or if there was going to be no
cost.
HT at 506. Other hearing testimony indicated that Boeing's proposed
aircraft uses the same batteries as the [Deleted] aircraft, but that
Northrop Grumman's aircraft may not. HT at 546-47; see also Boeing's
Comments at 116.
Also unexplained in the contemporaneous record is the agency's failure to
consider in its evaluation of MILCON costs the offerors' own estimates of
likely MILCON at Fairchild AFB that were included in their proposals.
Specifically, the RFP instructed offerors, as part of its response to the
product support subfactor, to
describe the offeror's approach to meet the government's 2-level
maintenance requirements. This proposal shall lay out:
* * * *
KC-X facilities, infrastructure requirements and design criteria.
Facilities required to support the first operational bed down location
at (assume Fairchild AFB, WA), including requirements for space,
utilities or special requirements (such as clean rooms, special storage,
etc.) with sufficient detail to assess installation capabilities to
support the KC-X. The offeror shall describe facilities recommended to
support the KC-X aircraft. The offeror shall, at a minimum, address the
square footage for parking, maintenance facilities, infrastructure
(e.g., power requirements, compressed air, office requirements,
storage), personnel, and support equipment required to operate two
squadrons of 16 aircraft for Main Operating Base (MOB) 1 and MOB 2. MOB
3, MOB 4 and MOB 5 will be determined at a later date.
RFP sections L.4.2.4.4, L.4.2.4.4.5, L.4.2.4.4.6.
Northrop Grumman informed the Air Force in its proposal that based upon a
[Deleted].
AR, Tab 167, Northrop Grumman's Pre-Final Proposal Revision, vol. II,
Mission Capability/Proposal Risk, Book 2, at II-SF3-48. Northrop Grumman
also informed the agency that, among other changes that would be needed,
[Deleted] in an identified building on Fairchild AFB would require
"[Deleted]." Id. at II-SF3-49.
The Air Force argues that it was reasonable to ignore the offerors' views
as to the sufficiency of the facilities at Fairchild AFB with respect to
their proposed aircraft because this information was requested in the
solicitation instructions for the product support subfactor, and the
offerors were not informed that this information would be used in the
agency's evaluation of MPLCCs.[82] See Air Force's Post-Hearing Rebuttal
Comments at 22-23. The agency does not explain, however, for what purpose
this information was requested if not to aid in its evaluation of the
facilities that would be needed to support the KC-X aircraft at Fairchild
AFB. Given that both offerors responded to this solicitation instruction,
it is apparent that neither offeror was confused as to the purpose of this
instruction, which plainly sought the offerors' views as to whether the
facilities at Fairchild AFB were adequate for their respective aircraft.
In short, we find no reasonable basis to ignore the information that both
offerors provided with respect to the adequacy of, or need for changes to,
facilities with respect to their proposed aircraft.
We also find, as described below, that the record does not otherwise
demonstrate the reasonableness of the Air Force's notional (hypothetical)
methodology for assessing likely MILCON costs. A notional beddown plan was
developed because the agency did not know where the KC-X aircraft would be
assigned. Under this scheme, the KC-X aircraft would be assigned in
specified numbers to a test base (Air Force Materiel Command), a training
base (Air Education & Training Command), three major operating bases (Air
Mobility Command) within the continental United States (CONUS), four air
reserve command (ARC) bases, and two major operating bases outside the
continental United States (OCONUS).[83] See AR, Tab 309, Notional KC-X
Beddown Plan Memorandum, June 29, 2007. As noted above, to assess the
MILCON costs associated with each offeror's aircraft, the agency conducted
site surveys at [Deleted] AFB, [Deleted] AFB, and [Deleted] AFB (major
operating bases) and at [Deleted] AFB (a training base). See Air Force's
Memorandum of Law at 221-22. The agency then extrapolated the results of
its [Deleted] AFB survey to six other bases (four unspecified air reserve
command bases and two unspecified OCONUS major operating bases) to
calculate the agency's MILCON costs for the offerors. With respect to the
two OCONUS airbases, the agency added a 10-percent premium to the
extrapolated costs. Air Force's Post-Hearing Comments at 122. The sole
reason identified by the Air Force for selecting [Deleted] AFB as the base
from which it would extrapolate costs to the four ARC airbases and two
OCONUS major operating bases was that a roughly comparable number of
aircraft would be assigned at each of these bases. See HT at 63,
1299-1300; Air Force's Post-Hearing Comments at 122.
Where, as here, anticipated requirements cannot be reasonably ascertained,
an agency may establish a reasonable hypothetical, or notional, plan to
provide for a common basis for evaluating costs. See, e.g., PWC Logistics
Servs., Inc., B-299820, B-299820.3, Aug. 14, 2007, 2007 CPD para. 162 at
11-15 Aalco Forwarding, Inc., et al., B-277241.15, Mar. 11, 1998, 98-1 CPD
para. 87 at 11. But that said, we are unable to conclude on this record
that the agency's extrapolation of the [Deleted] AFB MILCON costs to the
ARC airbases provided a reasonable basis to evaluate these costs. In this
regard, Boeing argues that [Deleted] AFB, as a former Strategic Air
Command, bomber base, has "a great deal more infrastructure" than do ARC
airbases and thus cannot be used as a reasonable forecast of potential
MILCON costs, such as for pavement improvements for runways, ramps, and
parking aprons, at other bases. See Boeing's Post-Hearing Comments at
136-41.
Although the Air Force dismisses Boeing's argument as being speculative
and argues that many ARC airbases have substantial infrastructure, see Air
Force's Post-Hearing Rebuttal Comments at 28, the agency has not produced
any explanation for selecting [Deleted] AFB other than its similar
squadron size, nor presented any evidence, either through testimony or by
reference to documentation in the record, showing why the infrastructure
at [Deleted] AFB would be similar enough to ARC airbases to find that the
costs evaluated for [Deleted] AFB are a reasonable representation of the
MILCON costs to be expected for ARC airbases.
Similarly, no evidence has been presented by the Air Force to explain why
the application of a 10-percent premium to the costs assessed for
[Deleted] AFB provides a reasonable estimate of MILCON costs for the
OCONUS major operating bases. In this regard, Boeing contends, with no
rebuttal, that the OCONUS airbases would have different issues associated
with MILCON costs, such as compliance with foreign labor laws and foreign
exchange rates, and that overseas bases would have to accommodate parking
for all of the assigned KC-X aircraft, as opposed to the 75 percent of
assigned aircraft that was done for CONUS airbases, such as [Deleted] AFB.
See Boeing's Post-Hearing Comments at 137. The only evidence supporting
the 10-percent factor is the testimony of the agency's SSET cost/price
factor team chief that it was "based on estimator judgment" of one of the
cost/price factor evaluators. HT at 209. No contemporaneous written
description or explanation of that judgment has been provided for the
record, however.
In sum, we do not find reasonable support in the record for the agency's
evaluation of the MILCON costs.
Evaluation of Boeing's Non-recurring Engineering Costs
Boeing also protests the Air Force's MPLCC adjustment for Boeing's
estimated non-recurring engineering costs in the SDD phase of the
contract. The Air Force added $[Deleted] million to the MPLCC beyond the
$[Deleted] billion for non-recurring engineering that Boeing estimated for
the SDD phase. Boeing states that its proposal approach is to acquire the
baseline 767-200 LRF aircraft (which Boeing asserts and the Air Force
concedes is a commercial item) from its commercial division, BCA, under a
fixed-price subcontract, and that its estimated non-recurring engineering
costs are included in the subcontract's fixed price.[84] Boeing argues
that the agency unreasonably did not accept Boeing's commercial data in
support of its estimated non-recurring engineering costs, and that it was
improper to add costs to its MPLCC, given that the non-recurring
engineering costs are part of a fixed-price subcontract for a commercial
item. See Boeing's Comments at 118-22.
The Air Force responds that, despite repeated discussions with Boeing
regarding the firm's need to substantiate its estimated non-recurring
engineering costs, see, e.g., AR, Tab 116, EN BOE-CP-001, EN BOE-CP-023,
Boeing did not adequately support its estimated non-recurring engineering
costs, and that the agency therefore concluded that there was a moderate
risk associated with Boeing's non-recurring engineering cost estimate,
although the agency did not determine that Boeing's estimated
non-recurring engineering costs were unrealistic. COS at 136-37; HT at
111-12.
The agency also decided that it was necessary to upwardly adjust the MPLCC
by $[Deleted] million to reflect this risk. To calculate this amount, the
agency used a "Monte Carlo" analysis,[85] and concluded that Boeing was
likely to incur a 36-percent cost growth with respect to its non-recurring
engineering costs during the SDD phase of the procurement, which the
agency then adjusted to account for a cost sharing provision that Boeing
had proposed. COS at 139-40; HT at 221. The agency also states that as a
"crosscheck" it looked at Boeing's P-8A Poseidon Multi-Mission Maritime
Aircraft contract with the Navy, under which the Air Force contends that
Boeing had an overall [Deleted]-percent cost growth, and that this
compared favorably with the overall 36-percent cost growth it forecast
using its Monte Carlo model. Air Force's Memorandum of Law at 205-06.
We find reasonable the agency's assignment of a moderate risk to Boeing's
proposal because of its failure to adequately substantiate its SDD
non-recurring engineering costs. As noted above, the RFP placed upon the
offerors the responsibility for substantiating their cost estimates. See,
e.g., RFP sections L.6.1.2, 6.4.7. Here, the Air Force found, reasonably
we conclude, that despite repeated requests during discussions Boeing
failed to substantiate its SDD non-recurring engineering cost estimate. In
this regard, we disagree with Boeing that, even if its purchase of the
baseline aircraft from its commercial division, is considered to be the
purchase of a commercial item, this prohibited the Air Force from
requesting substantiating cost information from Boeing. Although FAR
sect. 15.403-1(b)(3) provides that a contracting officer should not
request the submission of certified cost or pricing data when a commercial
item is being procured, this does not limit the right of the agency to
request other cost information to determine price reasonableness or
realism. See FAR sect. 15.403-3(c). We also note that it is not clear that
the subcontract between Boeing and its commercial division is a
fixed-price subcontract, as Boeing asserts, given Boeing's response in
discussions that indicated that the price would not be fixed until the
aircraft's configuration specifications were established, which had not
yet happened. See AR, Tab 119, Boeing Response to EN BOE-K-015, at 2-3;
Tab 259, Subcontract between Boeing and BCA.
Nevertheless, as discussed below, we conclude for a somewhat different
reason that the Air Force's MPLCC adjustment of Boeing estimated
non-recurring engineering costs for SDD was unreasonable. When an agency
evaluates proposals for the award of a cost-reimbursement contract, an
offeror's proposed estimated cost of contract performance is not
considered controlling since, regardless of the costs proposed by an
offeror, the government is bound to pay the contractor its actual and
allowable costs. Earl Indus., LLC, B-309996, B-309996.4, Nov. 5, 2007,
2007 CPD para. 203 at 8. As a result, a cost realism analysis is required
to determine the extent to which an offeror's proposed costs represent the
offeror's likely costs in performing the contract under the offeror's
technical approach, assuming reasonable economy and efficiency. See FAR
sections 15.305(a)(1), 15.404-1(d)(1). A cost realism analysis involves
independently reviewing and evaluating specific elements of each offeror's
cost estimate to determine whether the estimated proposed cost elements
are realistic for the work to be performed, reflect a clear understanding
of the requirements, and are consistent with the unique methods of
performance and materials described in the offeror's proposal. FAR
sect. 15.404-1(d)(1); Advanced Commc'n Sys., Inc., B-283650 et al., Dec.
16, 1999, 2000 CPD para. 3 at 5. Based on the results of the cost realism
analysis, an offeror's proposed costs should be adjusted "to realistic
levels based on the results of the cost realism analysis." FAR
sect. 15.404-1(d)(2)(ii).
Here, the record shows that the Air Force made no determination that
Boeing's estimated $[Deleted] billion for SDD non-recurring engineering
costs was unrealistic. See Air Force's Post-Hearing Comments at 90-91. In
this regard, the SSET cost/price factor team chief testified under cross
examination as follows:
Q: Yes. You're supposed to look at whether what -- what Boeing proposed
for the [non-recurring engineering], for the fixed price [non-recurring
engineering] was realistic for the work to be performed. . . .
A: No.
* * * *
Q: But when you made your adjustment, for example, one of the things that
I would expect you would do is you would try to make an adjustment to make
it, looking at the third item there, consistent with unique methods of
performance and materials described in the offeror's technical proposal.
Did you make any adjustments consistent with the unique methods of
performance and materials described in Boeing's proposal when you adjusted
upward using this Rand study?
A: We added cost risk.
HT at 111-12.
The Air Force and Northrop Grumman argue that section M.2.5.2.5 of the RFP
provided for the quantification of "pure cost risk," and for including
that quantified dollar amount in the agency's evaluated MPLCC. Air Force's
Post-Hearing Comments at 87; Northrop Grumman's Post-Hearing Comments at
27-28. We disagree. This solicitation section states in its entirety:
Risk Adjustments. The Government will assess the technical, cost, and
schedule risk for the entire most probable life cycle cost estimate
based upon the offeror's proposed approach. The Government will perform
a Schedule Risk Assessment (SRA) and quantify the schedule risk
accordingly. The Government will also assess risks associated with
technical content as identified in the evaluation of the Mission
Capability factor/subfactors 1 through 4, and other pure cost risks as
identified during the cost evaluation. The impact of technical,
schedule, and/or cost risk will be quantified (dollarized), where
applicable, and included in the MPLCC. Additionally, the Government
reserves the right to adjust budgetary estimates for technical, cost,
and schedule risk.
RFP sect. M.2.5.2.5.
We do not agree that this section allows the agency to upwardly adjust the
cost element of an offeror's "probable" costs of performance where the
agency does not conclude that the proposed cost element is unrealistic or
not probable. Rather, we find that this section allows the agency to
assess the risk associated with an offeror's probable costs and, "where
applicable," to quantify that risk and add the quantified amount in the
agency's evaluated MPLCC for an offeror.[86] The increase to the MPLCC is
"applicable" where the agency concludes that the higher number is more
probable or more realistic than the lower one. The Air Force's and
Northrop Grumman's reading is also inconsistent with other sections of the
RFP that provided that the Air Force would assess the realism of offerors'
proposed costs in accordance with FAR sect. 15.404-1 and that the agency's
evaluated MPLCCs would be the agency's estimates of the probable or likely
life cycle costs associated with the offerors' aircraft. See RFP sections
M.2.5.1.1, M.2.5.2. Such a reading is also inconsistent with FAR sect.
15.404-1, which, as noted above, provides for adjusting an offeror's
proposed costs "to realistic levels based on the results of the cost
realism analysis." See FAR sect. 15.404-1(d)(2)(ii).
Moreover, even assuming a cost realism adjustment would have been proper
in this case, we do not find reasonable the agency's use here of its Monte
Carlo simulation model. Although we have recognized that a Monte Carlo
model can be a useful evaluation tool, see TRW, Inc., supra, at 5, the
validity of a Monte Carlo simulation, like all cost estimation models,
depends upon the quality of the data used in the simulation or model. See
Cost Assessment Guide: Best Practices for Estimating and Managing Program
Costs, supra, at 144. Here, the cost evaluators used three inputs, "best
case, worst case, and most likely case," in the Monte Carlo simulation to
provide for a triangular distribution. HT at 29. Those three inputs were:
(1) no cost growth (the best case); (2) 28-percent cost growth, which was
derived from a GAO report, AR, Tab 281, Defense Acquisitions: Major Weapon
Systems Continue to Experience Cost and Schedule Problems under DoD's
Revise Policy, GAO-06-368, April 2006, (the most likely case); and (3)
58-percent cost growth, which was derived from a Rand Corporation study,
AR, Tab 282, Historical Cost Growth of Completed Weapon System Programs,
(RAND 2006), (the worst case). See COS at 139. These reports, however, are
discussing weapon systems and cost growth at an overall program level, and
the reported cost growth would likely be attributable to a number of
factors, including program changes and delays. In any event, we fail to
see how overall program cost growth is a reliable predictor of anticipated
growth in a single cost element, such as non-recurring engineering costs,
nor has the Air Force or Northrop Grumman provided any explanation as to
why that should be so.[87]
Cost Evaluation Errors Conclusion
In sum, we find that the Air Force unreasonably evaluated the MILCON costs
associated with the firms' proposed aircraft and unreasonably adjusted
Boeing's estimated non-recurring engineering costs, without finding those
costs to be unrealistic. The correction of these errors in the Air Force's
cost evaluation result in Boeing's MPLCC being lower than that of Northrop
Grumman's.[88]
Other Cost Issues
Boeing also challenges a number of other aspects of the Air Force's
evaluation of its cost proposal, including the Air Force's addition of
$[Deleted] billion to Boeing's proposed costs for budgetary aircraft (lots
6 through 13) and the addition of $[Deleted] billion to reflect additional
O&S repair costs. In addition, Boeing challenges a number of aspects of
the Air Force's evaluation of Northrop Grumman's proposed costs, including
that the agency did not evaluate the fuel costs associated with that
firm's larger and heavier aircraft and the costs of upgrades (such as the
[Deleted]) that may be added to Northrop Grumman's aircraft in the future.
We find no basis from our review of the record to object to the agency's
evaluation of these other aspects of the Air Force's evaluation of
costs.[89]
CONCLUSION AND RECOMMENDATION
This decision should not be read to reflect a view as to the merits of the
firms' respective aircraft. Judgments about which offeror will most
successfully meet governmental needs are largely reserved for the
procuring agencies, subject only to such statutory and regulatory
requirements as full and open competition and fairness to potential
offerors. Foundation Health Fed. Servs., Inc.; QualMed, Inc., B-254397.4
et al., Dec. 20, 1993, 94-1 CPD para. 3 at 43. Here, we find, as described
above, a number of errors in the Air Force's conduct of this procurement,
including the failure to evaluate proposals in accordance with the RFP
criteria and requirements and to conduct discussions in a fair and equal
manner. But for these errors, we believe that Boeing would have had a
substantial chance of being selected for award.[90] Accordingly, we
sustain Boeing's protest of the Air Force's award of a contract to
Northrop Grumman for the aerial refueling tankers.
The protest is sustained.
We recommend that the Air Force reopen discussions with the offerors,
obtain revised proposals, re-evaluate the revised proposals, and make a
new source selection decision, consistent with this decision. If the Air
Force believes that the RFP, as reasonably interpreted, does not
adequately state its needs, the agency should amend the solicitation prior
to conducting further discussions with the offerors. If Boeing's proposal
is selected for award, the Air Force should terminate the contract awarded
to Northrop Grumman. We also recommend that Boeing be reimbursed the
reasonable costs of filing and pursuing the protest, including reasonable
attorneys' fees. 4 C.F.R. sect. 21.8(d)(1). Boeing should submit its claim
for costs, detailing and certifying the time expended and costs incurred,
with the contracting agency within 60 days after receipt of this decision.
4 C.F.R. sect. 21.8(f)(1).
Gary L. Kepplinger
General Counsel
------------------------
[1] This acquisition has been identified as a Major Defense Acquisition
Program. See Agency Report (AR), Tab 8, Acquisition Strategy Report, at 1.
[2] The record in this case, which the agency largely provided
electronically to GAO and the private parties, is voluminous and complex,
and some of the record is classified. Although we considered the
classified information, it is not discussed in this decision.
[3] The Air Force acquired 732 KC-135A aircraft between 1957 and 1965. In
the 1980s, a number of KC-135A aircraft were upgraded to the KC-135E
aircraft, and later other KC-135A aircraft were upgraded to the KC-135R
aircraft. Currently, the Air Force has 85 KC-135E aircraft and 418 KC-135R
aircraft in its fleet. The agency also has 75 newer KC-10A aircraft in its
fleet. See Air Force Refueling: The KC-X Aircraft Acquisition Program, CRS
Report for Congress, at 4-5.
[4] LRIP is defined as:
The first effort of the Production and Deployment (P&D) phase. The
purpose of this effort is to establish an initial production base for
the system, permit an orderly ramp-up sufficient to lead to a smooth
transition to Full Rate Production (FRP), and to provide production
representative articles for Initial Operational Test and Evaluation
(IOT&E) and full-up live fire testing. This effort concludes with a Full
Rate Production Decision Review (FRPDR) to authorize the Full Rate
Production and Deployment (FRP&D) effort.
Glossary of Defense Acquisition Acronyms & Terms, Defense Acquisition
University, 12^th ed., July 2005, at B-96-97.
[5] A drogue is a small windsock placed at the end of a flexible hose that
trails from a tanker aircraft in flight in order to stabilize the hose and
to provide a funnel for the receiver aircraft, which inserts a probe into
the hose to receive fuel. See Aerial Refueling Methods: Flying Boom versus
Hose-and-Drogue, CRS Report for Congress, No. RL32910, June 5, 2006, at 1.
[6] For example, a "blue" rating reflected an exceptional proposal that
[e]xceeds specified minimum performance or capability requirements in a
way beneficial to the Government; proposal must have one or more
strengths and no deficiencies to receive a blue.
A "green" rating reflected an acceptable proposal that
[m]eets specified minimum performance or capability requirements
delineated in the [RFP]; proposal rated green must have no deficiencies
but may have one or more strengths.
RFP sect. M.2.2.
[7] For example, a "low" risk rating reflected a proposal that
[h]as little potential to cause disruption of schedule, increased cost
or degradation of performance. Normal contractor effort and normal
Government monitoring will likely be able to overcome any difficulties.
A "moderate" risk rating reflected a proposal that
[c]an potentially cause disruption of schedule, increased cost, or
degradation of performance. Special contractor emphasis and close
Government monitoring will likely be able to overcome difficulties.
RFP sect. M.2.3.
[8] For example, the graph indicated an aircraft must be capable of
offloading 117,000 pounds of fuel at a radius of 500 nautical miles and
94,000 pounds at a radius of 1,000 nautical miles. RFP, SRD
sect. 3.2.1.1.1.1, Figure 3-1, Fuel Offload vs. Radius Range.
[9] Although identified as "requirements" by the RFP, these non-KPP
"requirements" were not mandatory, but reflect features and performance of
the aircraft that the agency desired. There were thresholds and objectives
identified for some of the KSAs and the other SRD requirements.
[10] The Air Force described "trade space" as follows:
[the RFP] also provided the offerors considerable "trade space," meaning
some performance parameters of the tanker were required, while others
were not. The optional capabilities or attributes could be traded away
for better or different performance in other areas depending on the
offeror's unique approach. . . . Essentially, this asked the offerors to
tender their best proposals, and encouraged them to be creative in doing
so. With such a structure, the RFP harnessed the power of the commercial
marketplace competition to drive innovation as well as efficiency.
Air Force's Memorandum of Law at 5.
[11] The RFP provided that the PCAG would assign a confidence rating of
high confidence, significant confidence, satisfactory confidence, unknown
confidence, little confidence, or no confidence.
[12] The 2005 Mobility Capabilities Study assessed the mobility
capabilities of DoD against the backdrop of a revised National Security
Strategy; the study was intended to support decisions on future strategic
airlift, aerial refueling, aircraft, and sealift procurements needed to
meet varying military requirements. See Defense Transportation: Study
Limitations Raise Questions about the Adequacy and Completeness of the
Mobility Capabilities Study and Report, GAO-06-938, at 6.
[13] CMARPS is a system that is comprised of the Contingency Mating and
Ranging Program, Tanker Mating and Ranging Program, and Graphically
Supported Interactive Control System user interface. Using inputs, such as
aircraft performance characteristics, and assumptions and ground rules
(such as the maximum number of a particular aircraft that could be located
at a particular base given "ramp geometrics and aircraft dimensions" and
pavement strengths of ramps and runways), the agency would conduct
simulations, or "runs," where a proposed tanker fleet attempts to satisfy
tanker demand; the results of these simulations are intended to reflect
the effectiveness of those runs. See Fourth Supplemental Contracting
Officer's Statement (COS) at 9-12.
[14] With regard to fuel costs, offerors were requested to provide a
fuel-consumption "sample calculation" for an average mission-ready KC-X,
including fuel, crew, and mission equipment on board, in gallons per hour
per primary aircraft assigned multiplied by the number of flying hours in
a given fiscal year. Offerors were required to document the source of the
input data and rationale. RFP sect. L, attach. 15, KC-X O&S Data Form, at
7.
[15] Boeing stated in its proposal that the "767-200LRF is a new minor
model (a family of variants as defined by the FAA such as 767-200,
767-300F, or 767-400ER) that includes design features that satisfy KC-X
requirements." AR, Tab 61, Boeing Executive Summary, at V1-ES-1.
[16] Airbus is a division of European Aeronautic Defence & Space Company
(EADS), Northrop Grumman's principal subcontractor for this procurement.
After award, the Air Force changed the designation of Northrop Grumman's
aircraft to the KC-45; throughout this decision, however, we refer to
Northrop Grumman's aircraft by the firm's KC-30 designation.
[17] Although not requested by the parties, we conducted a hearing to
receive testimony from a number of Air Force witnesses to complete and
explain the record. In this regard, we provided a detailed description of
the hearing issues to the parties in a pre-hearing conference and in a
written Confirmation of Hearing notice. We also expressly informed the
parties that identification of some of the protest issues as hearing
issues did not indicate GAO's views as to the merits of any issue in the
case. The Air Force was informed that it was responsible for identifying
and producing those witnesses who could knowledgeably testify with respect
to the identified issues. Although invited to do so, neither Boeing nor
Northrop Grumman offered any witnesses. At the conclusion of the hearing,
the parties were informed that they could address any aspect of the
protest in their post-hearing comments and rebuttal comments. HT at 1524.
[18] The Air Force conducted numerous rounds of written and oral
discussions with the firms; in total, Boeing received 271 ENs, and
Northrop Grumman received 295 ENs. AR, Tab 46, SSET Final Briefing to SSAC
and SSA, at 9-10.
[19] Limited information was provided to the offerors in the mid-term
briefing with respect to the agency's schedule risk assessment and its
impact on the offeror's MPLCC. COS at 24.
[20] Discussions were conducted with the offerors to address any possible
impact on their proposals from section 804 of the National Defense
Authorization Act for Fiscal Year 2008 (related to Buy American Act
requirements with respect to specialty metals) and section 815 of the Act
(related to treatment of major defense acquisition program systems,
components, and spare parts as commercial items). Air Force's Memorandum
of Law at 20 n.6.
[21] A "major discriminator" was defined to be
an offered feature evaluated as a strength that provided extensive
capability and a substantial difference in magnitude of benefit to the
Air Force, when compared to the other Offeror.
A "discriminator offering less benefit" was defined to be
an offered feature evaluated as a strength that provided some capability
and some difference in benefit to the Air Force when compared to the
other Offeror.
AR, Tab 55, PAR, at 12.
[22] The SSAC reported that the KC-30 met the objective by offering a fuel
offload versus unrefueled range capability of [Deleted] pounds at 1,000
nautical miles and [Deleted] pounds at 2,000 nautical miles, which
exceeded the threshold by [Deleted] percent at 1,000 nautical miles and by
[Deleted] percent at 2,000 nautical miles. The KC-767 was also found to
meet the objective by offering a capability of [Deleted] pounds at
1,000 nautical miles and [Deleted] pounds at 2,000 nautical miles, which
exceeded the threshold by [Deleted] percent at 1,000 nautical miles and
[Deleted] percent at 2,000 nautical miles. AR, Tab 55, PAR, at 13-14.
[23] The ATP is an aerial refueling publication issued by the North
Atlantic Treaty Organization.
[24] Airlift efficiency was calculated using the following formula:
(pounds of payload) x (nautical miles)/(pounds of fuel). The SSET
performed this calculation at various distances for the offerors to derive
a payload-range curve to provide for a comparative analysis. AR, Tab 55,
PAR, at 17.
[25] The 463L pallet is the standard air cargo pallet used by the Air
Force and within the defense transportation system.
[26] The SSAC noted, however, that the KC-30's total weight carriage
capability on the main cargo deck was not substantially greater than that
of the KC-767. AR, Tab 55, PAR, at 17.
[27] The KPP No. 7 objective, the only objective under this KPP, provides
that the offeror's "system should be capable of accomplishing all
operational activities identified in Table 5." RFP app. A, Net-Ready Key
Performance Parameter for the KC-X SRD, Feb. 23, 2007, at 3. Table 5 of
the appendix identified a number of information exchange requirements. Id.
at 15-25.
[28] The capability to operate from a 7,000-foot runaway at sea level at
the aircraft's maximum gross weight was a non-KPP/KSA trade space
requirement, see RFP, SRD sect. 3.2.1.1.4.2, which both Boeing and
Northrop Grumman satisfied. AR, Tab 55, PAR, at 21.
[29] An unrefueled ferry range of 9,500 nautical miles starting at maximum
takeoff gross weight and using a maximum range flight profile was
identified as a non-KPP/KSA trade space requirement. RFP, SRD sect.
3.2.1.1.1.4.
[30] The SRD provided that the "KC-X shall be capable of supporting aerial
refueling operations from bare base airfields with confined ramp space."
RFP, SRD sect. 3.2.10.1.1.3.
[31] The SSAC did recognize that Boeing's proposal was more advantageous
than Northrop Grumman's in the survivability area. AR, Tab 55, PAR, at 29.
[32] A "satisfactory confidence" rating was assigned where
[b]ased on the offeror's performance record, the government has
confidence the offeror will successfully perform the required effort.
Normal contractor emphasis should preclude any problems.
A "little confidence" rating was assigned where
[b]ased on the offeror's performance record, substantial doubt exists
that the offeror will successfully perform the required effort.
RFP sect. M.2.4.1.
[33] Much of the information detailing the agency's evaluation under the
IFARA factor is classified.
[34] The SSA concluded that the offerors' proposals were essentially equal
under the remaining four mission capability subfactors. Regarding the
evaluation of Northrop Grumman's proposal under the product support
subfactor, although the Air Force found that Northrop Grumman had failed
to specifically commit to providing planning and support for the "initial
organic D-level [depot-level] maintenance capability" within 2 years
following delivery of the first full-rate production aircraft as required
by the RFP, see RFP, SOO for KSC-X SDD, at 14, the SSA agreed with the
SSAC that this was "merely an administrative oversight." AR, Tab 54,
Source Selection Decision, at 10. (The SSAC termed Northrop Grumman's
failure in this regard to be "an administrative documentation oversight."
AR, Tab 55, PAR, at 34.)
[35] Where, as here, a procurement is conducted on the basis of
competitive proposals, "an unsuccessful offeror, upon written request
received by the agency within 3 days after the date on which the
unsuccessful offeror receives the notification of the contract award,
shall be debriefed and furnished the basis for the selection decision and
award." 10 U.S.C. sect. 2305(b)(5)(A) (2000); Federal Acquisition
Regulation (FAR) sect. 15.506(a)(1).
[36] Although our regulations allow a procuring agency 30 days to provide
relevant documents to the protester, see 4 C.F.R. sect. 21.3(d) (2008),
the Air Force provided many relevant, core documents to Boeing and
Northrop Grumman within days of the filing of the initial protest. The Air
Force, however, continued to produce relevant documents even after the
filing of its agency report and up to the date of the hearing conducted in
this protest, which resulted in Boeing filing a series of supplemental
protests.
[37] Although we have not sustained all of Boeing's protest allegations,
nor do we address them all in this decision, we considered them all, which
required substantial development of the issues during the protest.
[38] Our document production rules are much narrower than other federal
discovery rules, such as the Federal Rules of Civil Procedure (FRCP),
which permits litigants to seek the existence of documents that are
reasonably calculated to lead to the discovery of admissible evidence.
See, e.g., FRCP Rule 26(b)(1). In contrast, our regulations provide for
the production of relevant documents. See 4 C.F.R. sect. 21.3(d).
[39] Although we denied the Air Force's request for documents from Boeing,
we also informed the agency that if, during the further development of the
case, the agency became aware of a specific relevant document, or
documents, that only the protester possesses, the agency was permitted to
request that document or documents. No such request was made.
[40] We will more fully address below certain of the agency's and
intervenor's dismissal requests, such as the arguments concerning the
evaluation of Northrop Grumman's proposal with respect to the fuel offload
versus unrefueled radius range.
[41] To require an offeror to file a protest each time during a
procurement that it is advised of an evaluation judgment with which it
disagrees or believes is inconsistent with the RFP would not be consistent
with the regulatory requirement that such protests can only be filed after
a required debriefing. The objective of this regulation is to avoid the
filing of "defensive protests" out of fear that our Office may dismiss the
protests as untimely and the associated potential to unnecessarily disrupt
procurements.
[42] As noted above, the RFP provided that KPP requirements were more
important than KSA requirements, which were in turn more important than
non-KPP/KSA requirements.
[43] As discussed above, the RFP indicated that KPP thresholds were
minimum, mandatory requirements that must be satisfied and that the
remaining "requirements," including KPP objectives and KSA thresholds and
objectives, were desired functions or characteristics that the firms could
choose to offer.
[44] The PAR combined the (1) and (2) major discriminators into one major
discriminator, although they actually concern separate KPP No. 1
objectives.
[45] Although the Air Force credited Northrop Grumman with exceeding to a
greater degree than Boeing the KPP No. 2 objective related to fuel offload
versus unrefueled range, we find, as discussed below, that this was
inappropriate, given that the RFP provided that no additional credit would
be provided for exceeding KPP objectives. The evaluation record thus shows
that, instead of this being a discriminator, Northrop Grumman and Boeing
should have received equal credit for satisfying this KPP objective.
[46] While we here illustrate this issue in our discussion of the aerial
refueling area, as further illustrated below, the agency's failure to
account for the relative weights given the various SRD requirements or
consider the RFP's request that offerors propose to satisfy as many of the
"trade space" requirements as possible permeates the evaluation of the key
system requirements subfactor.
[47] The Air Force recognizes that Boeing could have proposed an aerial
refueling tanker based upon the larger 777 platform. See, e.g., Air
Force's Memorandum of Law at 84 n.30.
[48] Thus, the Air Force's and Northrop Grumman's interpretation of this
objective as unbounded would render the last sentence of section
M.2.2.1.1.a meaningless, given that that sentence addresses the situation
where unbounded credit will be given for exceeding a KPP threshold.
[49] The Air Force and Northrop Grumman argue that Boeing's interpretation
is unreasonable because this would mean that meeting and exceeding the KPP
threshold relating to fuel offload versus unrefueled radius range would
necessarily approximate one another because both would be bounded by a
single line on a chart, which would render the establishment of an
objective meaningless. We do not agree. As Boeing notes, the establishment
of KPP objectives was expressly for the purpose of limiting the KPP trade
space available to offerors, and we find from our review of the entire
record that Boeing had a reasonable basis for believing that this
limitation--that no credit would be given for exceeding the KPP threshold
amount--was "consistent with real-world tanker operations." See Boeing's
Comments at 16.
[50] Both the Air Force and Northrop Grumman cite our decision in PM
Servs. Co., B-310762, Feb. 4, 2008, 2008 CPD para. 42, for the proposition
that Boeing, having learned how its proposal was being evaluated with
respect to this KPP objective, was required to protest before the next
closing time for receipt of proposals. In that case, we therefore
concluded, unlike here, that the protester was informed during discussions
of the agency's interpretation of a solicitation provision that was
otherwise clear on its face, and that the protester's later post-award
challenge of this provision was an untimely protest of an apparent
solicitation impropriety. Id. at 3.
[51] The Air Force's mid-term briefings to Boeing and Northrop Grumman
stated that "[o]nly SRD KPP Threshold requirements must be met --
strengths may be awarded for greater capability, but not beyond Objective
levels (if an Objective is stated)." See AR, Tab 129, Mid-term Briefing to
Boeing, at 17; Tab 199, Mid-Term Briefing for Northrop Grumman, at 17. The
parties' pre-final proposal revision briefings did not include this
language.
[52] Although the pre-award briefings provided to Boeing identified
"benefits" associated with the firm's offer to exceed the fuel offload
versus unrefueled radius range threshold, Boeing could not know until it
received the source selection decision that the agency was actually
providing additional credit for the degree to which the offerors were
exceeding the fuel offload versus unrefueled radius range.
[53] Even were we to consider the limitation on consideration above the
KPP objective for the fuel offload versus unrefueled range requirement to
be a latent ambiguity, Boeing's protest would still be timely. See Vitro
Servs. Corp., B-233040, Feb. 9, 1989, 89-1 CPD para. 136 at 3 n.1 (protest
filed within 10 days of the date the protester learned of an agency's
interpretation of a latent solicitation ambiguity is timely).
[54] The capability of Northrop Grumman's proposed aircraft to satisfy
this KPP threshold concerns a matter of technical acceptability; stated
differently, if Northrop Grumman could not establish the capability of its
aircraft to refuel all current fixed-wing tanker-compatible fixed wing
aircraft using current Air Force procedures, its proposal could not be
accepted. HT at 625, 649.
[55] In aerial refueling operations, tankers maneuver to a rendezvous
point and establish an orbit pattern at a constant airspeed to await
receiver aircraft. See, e.g., AR, Tab 289, Flight Manual, KC-10A Aircraft,
Flight Crew Tanker Air Refueling Procedures, USAF Series, Technical Order
(T.O.) 1-1C-1-33, Sept. 1, 2002, as revised Jan. 31, 2005, at 2-2, 2-15.
If a receiver aircraft overruns the tanker during the final phase of
rendezvous, the tanker and receiver pilots are directed to adjust to
specified overrun speeds, and after overtaking the receiver aircraft, the
tanker will decelerate to a refueling airspeed. Id. at 2-16. A breakaway
maneuver is an emergency procedure that is done when any tanker or
receiver aircraft crewmember perceives an unsafe condition that requires
immediate separation of the aircraft. See id. at 6-2; see also HT at 619.
In such a situation, the tanker pilot is directed to accelerate in level
flight to achieve separation, or, if required, to accelerate and climb
(during which the tanker pilot is directed to "not allow the airspeed to
decrease below that indicated at the start of climb.") See, e.g., AR, Tab
289, Flight Manual, KC-10A Aircraft, Flight Crew Tanker Air Refueling
Procedures, USAF Series, T.O. 1-1C-1-33, Sept. 1, 2002, as revised Jan.
31, 2005, at 6-2.
[56] In the first EN to Northrop Grumman addressing that firm's aircraft
overrun capability, the Air Force identified [Deleted] KIAS, as the
required overrun speed for the [Deleted]. See AR, Tab 184, EN NPG-MC1-003,
at 2. In the second EN to Northrop Grumman, the agency corrected this to
[Deleted] KIAS, see id., EN NPG-MC1-003a, at 1, which reflects the overrun
speed identified for the [Deleted] in the KC-135 flight manual. See Tab
289, Flight Manual KC-135 (Tanker) Flight Crew Air Refueling Procedures,
Supp. III, T.O. 1-1C-1-3, Jan. 1, 1987, at [Deleted].
[57] Initially, Northrop Grumman informed the Air Force that the agency
should change its current overrun procedures. See AR, Tab 184, Northrop
Grumman Response to EN NPG-MC1-003, at 1-3. Thereafter, Northrop Grumman
asserted that there was nothing in the RFP requirements that established
airspeed limitations for specific aircraft in overrun situations. Id.,
Northrop Grumman Response to EN NPG-MC1-003a, at 1-2. As noted by the Air
Force in the second EN provided to Northrop Grumman on this issue, the
agency's current procedures are established by its flight manuals for the
KC-135 and KC-10 that provide operational airspeed and overrun airspeed
requirements specific for each receiver aircraft type. See id.,
EN NPG-MC1-003a, at 1; e.g., Tab 289, Flight Manual KC-135 (Tanker) Flight
Crew Air Refueling Procedures, Supp. III, T.O. 1-1C-1-3, Jan. 1, 1987, at
[Deleted]; see also HT at 622.
[58] The Air Force and Northrop Grumman contend that Boeing's
contention--that Northrop Grumman's [Deleted] causes [Deleted]--was
untimely raised in Boeing's comments and must be dismissed. Air Force's
Post-Hearing Comments at 18; Northrop Grumman's Post-Hearing Comments at
141. We disagree. In its second supplemental protest (filed within 10 days
of receipt of the first production of documents), Boeing specifically
challenged the Air Force's evaluation of Northrop Grumman's proposal to
[Deleted], arguing that the "Air Force never considered the feasibility of
this extreme measure or its implications on the KC-30's ability to carry
out the refueling mission." See Boeing's Second Supplemental Protest
at 29-31. The arguments concerning the [Deleted] are thus within the scope
of Boeing's timely protest.
[59] As noted above, our Office requested that the Air Force provide
knowledgeable witnesses who could testify with respect to the previously
identified hearing issues.
[60] FAA's regulations provide that the design dive speed of an aircraft
be established so that the design cruise speed is no greater than 0.8
times the design dive speed. See 14 C.F.R. sect. 25.335(b) (2008).
[61] The record indicates that the evaluators did not consider during the
procurement whether the [Deleted] during aerial refueling operations was a
current Air Force procedure and how this may affect Northrop Grumman's
proposed solution to satisfying the overrun speed requirements. Rather,
these issues apparently were only considered in response to Boeing's
protest allegations. See HT at 711.
[62] The flight manuals for the KC-135 and KC-10 both state that the
"[p]rocedures in this manual are mandatory and must be performed in the
prescribed manner except where deviations are required in the interest of
safety of flight." AR, Tab 289, Flight Manual KC-135 (Tanker) Flight Crew
Air Refueling Procedures, Supp. III, T.O. 1-1C-1-3, Jan. 1, 1987, as
revised Sept. 1, 2004, at i; Tab 289, Flight Manual, KC-10A Aircraft,
Flight Crew Tanker Air Refueling Procedures, USAF Series, T.O. 1-1C-1-33,
Sept. 1, 2002, as revised Jan. 31, 2005, at ii.
[63] In its post-hearing comments, Northrop Grumman argues that [Deleted].
See Northrop Grumman's Post-Hearing Comments at 141-43. We provide little
weight to this post-procurement description of Northrop Grumman's proposed
design, given that this argument seems inconsistent with Northrop
Grumman's EN response and is not supported by statements of consultants or
other knowledgeable sources, and it represents information that was not
presented to the agency for its consideration during the procurement.
[64] In this regard, in response to cross examination, the SSET mission
capability team chief testified that, although Northrop Grumman in its EN
response was not "very good at articulating what they were doing at the
end there, okay," the evaluation team did not ask Northrop Grumman to
clarify what it was proposing in its EN response. See HT at 664.
[65] Northrop Grumman provided to the Air Force with its EN response a FAA
Type Certificate for the Airbus A330-200 and A330-300 series aircraft,
which identified the maximum operating limit airspeed as 330 KIAS and the
design diving speed as 365 KIAS. AR, Tab 184, Northrop Grumman Response to
EN NPG-MC1-003a, attach., FAA Type Certificate Data Sheet No. A46NM, Rev.
10, Mar. 19, 2007, at II-J-72. In November 2007, the FAA-type certificate
for the A330 aircraft was revised, but stated the same maximum operating
limit and dive speeds. See Boeing's Airspeed Hearing exh. 13, FAA Type
Certificate No. A46NM, Rev. 11, Nov. 13, 2007, at 12.
[66] We have been presented with no testimony, statements or documentation
from any member of the SSET professing to understand the relationship of
maximum operational airspeed and dive velocity or airspeed, or to
otherwise support the agency's conclusion that the A330's certified design
dive velocity indicated that the aircraft was capable of achieving that
speed as a maximum operational airspeed.
[67] The SSET mission capability factor team chief also testified that
Northrop Grumman's response indicated to the SSET that the KC-30 could
achieve [Deleted] KIAS with both the [Deleted]. HT at 636. In this regard,
Northrop Grumman's EN response contained a "Note" under case two
([Deleted]), which stated that the [Deleted]. AR, Tab 184, Northrop
Grumman Response to EN NPG-MC1-003a, at 6. It is unexplained in what
situation this occurs, given that the [Deleted] is supposed to [Deleted].
Moreover, neither the Air Force nor Northrop Grumman has identified any
evidence in the contemporaneous record of the agency's consideration of
this note.
[68] Although the Air Force argues that it considered whether there was
any schedule or cost risk associated with the proposed changes to Northrop
Grumman's aircraft to satisfy the airspeed requirements, see Air Force's
Post-Hearing Comments, at 18, we have not been directed to documentation
in the record establishing that such an analysis was performed. Instead,
the Air Force relies upon the testimony of the SSET mission capability
factor team chief that the SSET concluded that Northrop Grumman had
provided "associated costs and schedule impact" for the firm's proposed
approach to satisfying the airspeed requirements. See HT at 629. However,
he was unable to point to anything in the record to support his testimony,
except his statement that a structural engineer on the SSET reviewed
Northrop Grumman's EN response and determined that any required changes to
the proposed aircraft could be accomplished within Northrop Grumman's
proposed schedule. HT at 721. The totality of that review by the
structural engineer, however, was apparently captured in an e-mail sent
during the evaluation. See HT at 757-59, 783 (proffer by Air Force
counsel). This e-mail does not establish that the structural engineer
validated the capability of Northrop Grumman's aircraft to satisfy the
overrun airspeed requirements or that changes in the aircraft's maximum
operational airspeed could be achieved within the offeror's proposed
schedule or cost. Rather, the e-mail states that the effect of higher
airspeed on the integrity of KC-30 aircraft structure has not been
analyzed and that "[b]ottom line, these [Deleted] are major concerns that
must be addressed by Analysis for sure and Flight Test if warranted." AR,
Tab 332, E-mail 32002 re: EN NPG-MC1-003a, Jan. 25, 2008.
[69] The Air Force did not contemporaneously express any concern to
Northrop Grumman with respect to its aircraft's ability to achieve
breakaway speeds. HT at 619.
[70] Boeing argues, citing the procedures identified in Northrop Grumman's
final proposal, that [Deleted] Northrop Grumman's [Deleted] could take as
long as [Deleted], which would require the tanker pilot to either
accelerate beyond [Deleted] KIAS before [Deleted] or delay initiating the
breakaway until after [Deleted]; Boeing contends that the Air Force did
not assess these concerns. See Boeing's Post-Hearing Comments at 68,
citing, AR, Tab 187, Northrop Grumman's Final Proposal Revision, vol. II,
Mission Capability/Proposal Risk, Jan. 4, 2008, at II-SF116-16a.
[71] It is unclear from the record when the Air Force changed its
evaluation of this KPP objective.
[72] As set forth above, the agency also identified a weakness for
Northrop Grumman in the aerial refueling area related to the firm's boom
approach. Because the record did not contain any documentation explaining
why the Air Force's evaluated concern with Northrop Grumman's proposed
boom design represents a low risk as to schedule or cost, we also
identified this as an area in which hearing testimony would be required to
"explain why evaluated weaknesses in Northrop Grumman's boom have low
schedule or cost risk." See GAO Confirmation of Hearing, Apr. 29, 2008,
at 3. The Air Force produced its SSET team chief to address this issue,
and, although he clearly articulated the SSET's evaluated concerns with
regard to Northrop Grumman's boom design, his testimony regarding any
schedule and/or cost risk associated with these concerns was conclusory.
See, e.g., HT at 1009-13, 1016-17, 1022. Although the record, including
the SSET team chief's testimony, indicates that some analyses of the
impact of these evaluated concerns may have been performed, little detail
has been provided. In this regard, we have been provided with no other
testimony or statements from SSET members or citation to documentation in
the record that would otherwise support the agency's judgment that there
is little schedule or cost risk associated with these evaluated concerns.
Given our recommendation below that the Air Force obtain and re-evaluate
revised proposals, we think that this is also a matter that the agency
should consider further.
[73] "Organic" maintenance refers to maintenance that the agency does for
itself as opposed to maintenance provided by the contractor. See HT at
1215.
[74] Unlike Northrop Grumman, Boeing committed to providing the required
planning and support services within the specified 2-year timeframe. HT at
1221.
[75] The agency's product support subfactor team chief testified regarding
this requirement:
the idea behind that is to support the government in standing up this
capability, so their approach would have to include the planning and
support, the planning part being those type of actionable steps that
support the type of things they would support us within that time
constraint.
HT at 1215.
[76] The SSET product support subfactor team chief stated that the
pre-final proposal revision briefing slide erroneously did not also refer
to the SDD SOW, in addition to the production SOW, and that both SOWs
would be implicated by Northrop Grumman's failure to commit to providing
these services within the required 2-year timeframe. See HT at 1266-67.
[77] Similarly, the SSET's product support subfactor team chief doggedly
insisted that Northrop Grumman's failure to agree to perform the required
services within the specified time frame was merely an oversight, even
where he admitted under cross examination that "Northrop [Grumman] didn't
forget about this issue," that Northrop Grumman's "[final proposal
revision] was not silent on the issue," and that in fact "Northrop Grumman
did consider the issue; they just decided not to address it in their
[final proposal revision]." See HT at 1274-76.
[78] Northrop Grumman does not explain why it made the "intentional"
choice not to specifically include the 2-year requirement in the
contractual SOW, even though it was repeatedly requested to do so by the
Air Force.
[79] In any event, the SSAC's and SSA's judgment that the firms' proposals
were essentially equal under the product support subfactor is undermined
by their erroneous conclusion that Northrop Grumman's failure to commit to
the 2-year timeframe was an oversight.
[80] Thus, the Air Force essentially concedes that the conclusion in the
source selection decision that Boeing's evaluated MPLCC was more than
Northrop Grumman's was in error. [In preparing the public version of the
protected decision, as the Air Force correctly points out, one of the five
acknowledged errors was actually with respect to repair costs. The
magnitude of these five errors remains unchanged.]
[81] The KC-30 is capable of carrying [Deleted] passengers, while the
KC-767 can carry [Deleted] passengers. AR, Tab 55, PAR, at 18-19.
[82] The Air Force also suggests that the RFP "directed that the MILCON
portion of the MPLCC would be estimated entirely by the Air Force, with
inputs from both Air Mobility Command (AMC) and Air Force Materiel Command
(AFMC)." Air Force's Post-Hearing Comments at 119; see Air Force's
Memorandum of Law at 221. This argument is based upon section M.2.5.2.4 of
the RFP, which informed offerors that the agency's assessment of MPLCC
would include evaluating MILCON costs and further informed offerors:
"Note: Air Mobility Command and Air Force Materiel Command are estimating
MILCON." This section does not, however, inform offerors that the Air
Force would not consider their proposals in preparing this estimate.
[83] The test base was not included in the agency's MILCON cost
evaluation.
[84] The SDD aircraft are provided to the Air Force under a cost
reimbursement line item.
[85] A Monte Carlo simulation is a cost risk analysis model that is
generally used for quantifying the lowest and highest possible costs of
weapons systems, based upon estimated costs of various components. See
TRW, Inc., B-234558, June 21, 1989, 89-1 CPD para. 584 at 3 n.1. Developed
in 1946 by a mathematician who pondered the probabilities associated with
winning a card game of solitaire, a Monte Carlo simulation is used to
approximate the probability outcomes of multiple trials by generating
random numbers. In determining the uncertainty associated with a program's
point estimate, a Monte Carlo simulation randomly generates values for
uncertain variables over and over to simulate a model. Cost Assessment
Guide: Best Practices for Estimating and Managing Program Costs,
GAO-07-1134SP, July 2007, at 154.
[86] We have in a number of decisions explained the relationship between
probable cost adjustments and proposal risk, but in no case have we found
that an agency's adjustment of an offeror's proposed costs of performing a
contract should be based only upon risk, and not upon a reasoned
assessment of the realism of the proposed costs being adjusted. Thus, for
example, we have recognized that an agency is not required to upwardly
adjust an offeror's proposed costs which the agency found realistic, even
where the agency also assessed some risk with regard to those costs. See,
e.g., ITT Indus., Inc., B-294389 et al., Oct. 20, 2004, 2004 CPD para. 222
at 15-16; Vinnell Corp., B-270793, B-270793.2, Apr. 24, 1996, 96-1 CPD
para. 271 at 6. Conversely, an agency may both make cost realism
adjustments and assign proposal risk, where "the cost adjustments are
necessary to reflect the offeror's probable costs of performance based on
its proposal," and that there continued to be proposal risk despite the
cost adjustment. See Raytheon Co., B-291449, Jan. 7, 2003, 2003 CPD
para. 54 at 16 n.12.
[87] Similarly, we do not see any validity to using the overall cost
growth associated with the Boeing's Multi-Mission Maritime Aircraft
contract with the Navy to forecast cost growth associated with Boeing's
SDD non-recurring engineering costs. In addition, Boeing asserts, without
rebuttal, that the cost growth under that contract was due to reasons
unrelated to non-recurring engineering costs. See Boeing's Protest at 85.
[88] The Air Force argues that Boeing is not prejudiced by these errors
because the SSA in [the SSA's] selection decision stated that [the SSA]
would have selected Northrop Grumman's proposal for award "even if
Boeing's proposed cost/price had not been adjusted upward by the
Government and Boeing's cost/price risk rating for SDD had been rated as
LOW." AR, Tab 54, Source Selection Decision, at 19-20. We disagree. As
concluded above, the Air Force erred in the evaluation of technical
proposals and the conduct of discussions and this statement by the SSA
does not address any of those errors. In any event, this statement by the
SSA, which is unsupported by specific analysis, would not seem to reflect
the reasoned consideration of cost or price to the government that a
selection official is required to provide in performing a trade-off
analysis. See, e.g., Shumaker Trucking and Excavating Contractors, Inc.,
B-290732, Sept. 25, 2002, 2002 CPD 169 at 6.
[89] The Air Force's evaluation of the fuel costs associated with the
firms' proposed aircraft has been the subject of much argument and hearing
testimony, and the record indicates that the agency did not do much more
than an assessment that the offerors' own proposed fuel burn rates
(gallons of fuel burned per hour) was reasonable. The record also shows,
however, that even a small increase in the amount of fuel that is burned
per hour by a particular aircraft would have a dramatic impact on the
overall fuel costs (for example, Boeing notes that even a
[Deleted]-percent increase in the amount of fuel per hour that is burned
by the KC-30 would result in a $[Deleted] million increase in Northrop
Grumman's life cycle costs for fuel, see Boeing's Post-Hearing Comments,
at 139). Given our recommendation below that the Air Force reevaluate
proposals and obtain revised proposals, this is another matter that the
agency may wish to review to ascertain whether a more detailed analysis of
the fuel costs is appropriate.
[90] Our Office will not sustain a protest unless the protester
demonstrates a reasonable possibility of prejudice, that is, unless the
protester demonstrates that, but for the agency's actions, it would have
had a substantial chance of receiving the award. See McDonald Bradley,
B-270126, Feb. 8, 1996, 96-1 CPD para. 54 at 3; see Statistica, Inc. v.
Christopher, 103 F.3d 1577, 1581 (Fed.Cir. 1996).