Geostationary Operational Environmental Satellites: Steps Remain
in Incorporating Lessons Learned from Other Satellite Programs
(06-SEP-06, GAO-06-993).
The National Oceanic and Atmospheric Administration (NOAA) plans
to procure the next generation of geostationary operational
environmental satellites, called the Geostationary Operational
Environmental Satellites-R series (GOES-R). This new series is
considered critical to the United States' ability to maintain the
continuity of data required for weather forecasting through the
year 2028. GAO was asked to (1) determine the status of and plans
for the GOES-R series procurement, and (2) identify and evaluate
the actions that the program management team is taking to ensure
that past problems experienced in procuring other satellite
programs are not repeated.
-------------------------Indexing Terms-------------------------
REPORTNUM: GAO-06-993
ACCNO: A60332
TITLE: Geostationary Operational Environmental Satellites: Steps
Remain in Incorporating Lessons Learned from Other Satellite
Programs
DATE: 09/06/2006
SUBJECT: Contract administration
Cost overruns
Earth resources satellites
Geosynchronous satellites
Lessons learned
Procurement planning
Program evaluation
Program management
Satellites
Weather forecasting
NOAA Geostationary Operational
Environmental Satellite
******************************************************************
** This file contains an ASCII representation of the text of a **
** GAO Product. **
** **
** No attempt has been made to display graphic images, although **
** figure captions are reproduced. Tables are included, but **
** may not resemble those in the printed version. **
** **
** Please see the PDF (Portable Document Format) file, when **
** available, for a complete electronic file of the printed **
** document's contents. **
** **
******************************************************************
GAO-06-993
* Results in Brief
* Background
* A Brief History of Prior GOES Series
* Original GOES Satellites
* GOES I-M Series
* GOES-N Series
* Planned GOES-R Series
* GOES-R Program-An Overview
* GOES-R Program Office Structure
* Satellite Programs Often Experience Technical Problems, Cost
* National Polar-orbiting Operational Environmental Satellite
* GOES I-M Series
* Space Based Infrared System High Program
* Advanced Extremely High Frequency Satellite System
* GOES-R Procurement Activities Are Under Way, but System Requ
* The GOES-R Program Office Has Taken Steps to Address Past Le
* NOAA Is Taking Steps to Improve the Reliability of Cost and
* NOAA Is Conducting Preliminary Studies in Order to Avoid Tec
* Efforts to Strengthen Government and Contractor Management a
* NOAA Has Established a Senior Executive Committee to Perform
* Conclusions
* Recommendations for Executive Action
* Agency Comments and Our Evaluation
* Appendix I: Objectives, Scope, and Methodology
* Appendix II: Current Shortfalls in Contractor Performance on
* Appendix III: Comments from the Department of Commerce
* Appendix IV: GAO Contact and Staff Acknowledgments
* GAO Contact
* Staff Acknowledgments
* Order by Mail or Phone
Report to the Subcommittee on Environment, Technology, and Standards,
Committee on Science, House of Representatives
United States Government Accountability Office
GAO
September 2006
GEOSTATIONARY OPERATIONAL ENVIRONMENTAL SATELLITES
Steps Remain in Incorporating Lessons Learned from Other Satellite
Programs
GAO-06-993
Contents
Letter 1
Results in Brief 2
Background 3
GOES-R Procurement Activities Are Under Way, but System Requirements and
Cost Estimates May Change 19
The GOES-R Program Office Has Taken Steps to Address Past Lessons Learned,
but Significant Actions Remain 21
Conclusions 29
Recommendations for Executive Action 29
Agency Comments and Our Evaluation 30
Appendix I Objectives, Scope, and Methodology 32
Appendix II Current Shortfalls in Contractor Performance on Key Instrument
Development 34
Appendix III Comments from the Department of Commerce 37
Appendix IV GAO Contact and Staff Acknowledgments 41
Tables
Table 1: Summary of the Procurement History of GOES 6
Table 2: Summary of Key GOES-R System Improvements 8
Table 3: Expected GOES-R Series Instruments, as of June 2006 9
Table 4: Key Problems Experienced on Selected Major Space Systems 13
Table 5: Status of GOES-R Program Contracts, as of June 2006 19
Table 6: GOES-R Program Life Cycle Cost Estimate, as of June 2006 20
Table 7: GOES-R Program Schedule, as of June 2006 20
Figures
Figure 1: Approximate GOES Geographic Coverage 4
Figure 2: Generic GOES Data Relay Pattern 6
Figure 3: GOES-R Program Office Structure and Staffing 11
Figure 4: Cumulative Cost Variance of the ABI Instrument Contract over an
11-month Period 35
Figure 5: Cumulative Schedule Variance of the ABI Instrument Contract over
an 11-month Period 36
Abbreviations
ABI Advanced Baseline Imager AEHF Advanced Extremely High Frequency
Satellite System DOD Department of Defense GOES-R Geostationary
Operational Environmental Satellites-R series NASA National Aeronautics
and Space Administration NESDIS National Environmental Satellite Data and
Information Service NOAA National Oceanic and Atmospheric Administration
NPOESS National Polar-orbiting Operational Environmental Satellite System
SBIRS-High Space Based Infrared System High Program
This is a work of the U.S. government and is not subject to copyright
protection in the United States. It may be reproduced and distributed in
its entirety without further permission from GAO. However, because this
work may contain copyrighted images or other material, permission from the
copyright holder may be necessary if you wish to reproduce this material
separately.
United States Government Accountability Office
Washington, DC 20548
September 6, 2006 September 6, 2006
The Honorable Vernon J. Ehlers Chairman The Honorable David Wu Ranking
Minority Member Subcommittee on Environment, Technology, and Standards
Committee on Science House of Representatives The Honorable Vernon J.
Ehlers Chairman The Honorable David Wu Ranking Minority Member
Subcommittee on Environment, Technology, and Standards Committee on
Science House of Representatives
Operational geostationary environmental satellites play a critical role in
our nation's weather forecasting. These satellites-which are managed by
the National Oceanic and Atmospheric Administration (NOAA)-provide
critical information on atmospheric, oceanic, climatic, and solar
conditions that help meteorologists observe and predict global and local
weather events. They also provide the best means to identify severe storm
conditions, such as hurricanes and tornadoes, and to track the movement
and intensity of these storms once they develop. Operational geostationary
environmental satellites play a critical role in our nation's weather
forecasting. These satellites-which are managed by the National Oceanic
and Atmospheric Administration (NOAA)-provide critical information on
atmospheric, oceanic, climatic, and solar conditions that help
meteorologists observe and predict global and local weather events. They
also provide the best means to identify severe storm conditions, such as
hurricanes and tornadoes, and to track the movement and intensity of these
storms once they develop.
NOAA, with the aid of the National Aeronautics and Space Administration
(NASA), is planning to procure the next generation of geostationary
satellites, called the Geostationary Operational Environmental
Satellites-R series (GOES-R). The GOES-R series is to replace the current
series of satellites which will likely begin to reach the end of their
useful lives in approximately 2012. This new series is expected to mark
the first major technological advance in GOES instrumentation since 1994.
It is also considered critical to the United States' ability to maintain
the continuity of data required for weather forecasting through the year
2028. NOAA, with the aid of the National Aeronautics and Space
Administration (NASA), is planning to procure the next generation of
geostationary satellites, called the Geostationary Operational
Environmental Satellites-R series (GOES-R). The GOES-R series is to
replace the current series of satellites which will likely begin to reach
the end of their useful lives in approximately 2012. This new series is
expected to mark the first major technological advance in GOES
instrumentation since 1994. It is also considered critical to the United
States' ability to maintain the continuity of data required for weather
forecasting through the year 2028.
This report responds to your request that we review NOAA's management of
the GOES-R program. Specifically, we were asked to (1) determine the
status of and plans for the GOES-R series procurement, and (2) identify
and evaluate the actions that the program management team is taking to
ensure that past problems experienced in procuring other satellite
programs are not repeated. This report responds to your request that we
review NOAA's management of the GOES-R program. Specifically, we were
asked to (1) determine the status of and plans for the GOES-R series
procurement, and (2) identify and evaluate the actions that the program
management team is taking to ensure that past problems experienced in
procuring other satellite programs are not repeated.
To determine GOES-R status and plans, we reviewed program documents on the
planned acquisition schedules, cost estimates, and system requirements,
and interviewed program officials. To assess the program office's actions
to address lessons learned from past satellite programs, we analyzed
lessons learned from past satellite programs-including a prior To
determine GOES-R status and plans, we reviewed program documents on the
planned acquisition schedules, cost estimates, and system requirements,
and interviewed program officials. To assess the program office's actions
to address lessons learned from past satellite programs, we analyzed
lessons learned from past satellite programs-including a prior GOES
satellite series (called GOES I-M), the National Polar-orbiting
Operational Environmental Satellite System (NPOESS), and selected military
satellite programs. We analyzed program management documents, including
data on a critical instrument's development, to determine and evaluate
plans for addressing past lessons. We also interviewed program officials
from NOAA and NASA.
We conducted our work at NOAA and NASA offices in the Washington, D.C.,
metropolitan area between December 2005 and August 2006, in accordance
with generally accepted government auditing standards. Appendix I contains
further details on our objectives, scope, and methodology.
Results in Brief
NOAA is nearing the end of the preliminary design phase of its GOES-R
system, which is officially estimated to cost $6.2 billion and scheduled
to have the first satellite ready for launch in 2012. To date, NOAA has
issued contracts for the preliminary design of the overall GOES-R system
to three vendors and expects to award a contract to one of these vendors
in August 2007 to develop the satellites. In addition, to reduce the risks
associated with developing new instruments, NOAA has issued contracts for
the early development of one critical instrument and for the preliminary
designs of four other instruments. The agency plans to turn these
instrument contracts over to the vendor that is awarded the contract for
the overall GOES-R program. However, according to program officials,
NOAA's plans for the GOES-R procurement could change in the near future.
Recent analyses of the GOES-R program cost-which in May 2006 the program
office estimated could reach $11.4 billion-have led the agency to consider
reducing the scope of requirements for the satellite series. NOAA
officials estimated that a decision on the future scope and direction of
the program could be made by the end of September 2006.
NOAA has taken steps to implement lessons learned from past satellite
programs, but more remains to be done. Prior satellite programs-including
a prior GOES series, a polar-orbiting environmental satellite series, and
various military satellite programs-often experience technical challenges,
cost overruns, and schedule delays. Key lessons from these programs
include the need to (1) establish realistic cost and schedule estimates,
(2) ensure sufficient technical readiness of the system's components prior
to key decisions, (3) provide sufficient management at government and
contractor levels, and (4) perform adequate senior executive oversight to
ensure mission success. NOAA has established plans to address these
lessons by conducting independent cost estimates, performing preliminary
studies of key technologies, placing resident government offices at key
contractor locations, and establishing a senior executive oversight
committee. However, many steps remain to fully address these lessons.
Specifically, NOAA has not yet developed a process to evaluate and
reconcile the independent and government cost estimates. In addition, NOAA
has not yet determined how it will ensure that a sufficient level of
technical maturity will be achieved in time for an upcoming decision
milestone, nor has it determined the appropriate level of resources it
needs to adequately track and oversee the program using earned value
management.1 Until it completes these activities, NOAA faces an increased
risk that the GOES-R program will repeat the increased cost, schedule
delays, and performance shortfalls that have plagued past procurements.
We are making recommendations to the Secretary of Commerce to direct its
NOAA Program Management Council to establish a process for objectively
evaluating and reconciling the government and independent life cycle cost
estimates once the program requirements are finalized; to establish a team
of system engineering experts to perform a comprehensive review of the
Advanced Baseline Imager instrument to determine the level of technical
maturity achieved on the instrument before moving it into production; and
to seek assistance in determining the appropriate levels of resources
needed at the program office to adequately track and oversee the
contractor's earned value management data.
The Department of Commerce provided written comments on a draft of this
report in which it agreed with our recommendations and identified planned
steps for implementing them (see app. III). The department also provided
technical corrections, which we have incorporated in this report as
appropriate.
Background
Since the 1960s, geostationary and polar-orbiting environmental satellites
have been used by the United States to provide meteorological data for
weather observation, research, and forecasting. NOAA's National
Environmental Satellite Data and Information Service (NESDIS) is
responsible for managing the civilian geostationary and polar-orbiting
satellite systems as two separate programs, called GOES and the Polar
Operational Environmental Satellites, respectively.
1Earned value management is a method that compares the value of work
accomplished during a given period with that of the work expected in that
period.
Unlike polar-orbiting satellites, which constantly circle the earth in a
relatively low polar orbit, geostationary satellites can maintain a
constant view of the earth from a high orbit of about 22,300 miles in
space. NOAA operates GOES as a two-satellite system that is primarily
focused on the United States (see fig. 1). These satellites are uniquely
positioned to provide timely environmental data to meteorologists and
their audiences on the earth's atmosphere, its surface, cloud cover, and
the space environment. They also observe the development of hazardous
weather, such as hurricanes and severe thunderstorms, and track their
movement and intensity to reduce or avoid major losses of property and
life. Furthermore, the satellites' ability to provide broad, continuously
updated coverage of atmospheric conditions over land and oceans is
important to NOAA's weather forecasting operations.
Figure 1: Approximate GOES Geographic Coverage
To provide continuous satellite coverage, NOAA acquires several satellites
at a time as part of a series and launches new satellites every few years.
Three satellites-GOES-11, GOES-12, and GOES-13-are currently in orbit.
Both GOES-11 and GOES-12 are operational satellites, while GOES-13 is in
an on-orbit storage mode. It is a backup for the other two satellites
should they experience any degradation in service. The others in the
series, GOES-O and GOES-P, are planned for launch over the next few
years.2 NOAA is also planning a future generation of satellites, known as
the GOES-R series, which are planned for launch beginning in 2012.
Each of the operational geostationary satellites continuously transmits
raw environmental data to NOAA ground stations. The data are processed at
these ground stations and transmitted back to the satellite for broadcast
to primary weather services both in the United States and around the
world, including the global research community. Raw and processed data are
also distributed to users via ground stations through other communication
channels, such as dedicated private communication lines and the Internet.
Figure 2 depicts a generic data relay pattern from the geostationary
satellites to the ground stations and commercial terminals.
2Satellites in a series are identified by letters of the alphabet when
they are on the ground and by numbers once they are in orbit.
Figure 2: Generic GOES Data Relay Pattern
A Brief History of Prior GOES Series
To date, NOAA has procured three series of GOES satellites and is in the
planning stages to acquire a fourth one (see table 1).
Table 1: Summary of the Procurement History of GOES
Source: GAO analysis of NOAA data.
aDuration includes time from contract award to final satellite launch.
bThe procurement of these satellites consisted of four separate contracts
for (1) two early prototype satellites and GOES-1, (2) GOES-2 and -3, (3)
GOES-4 through -6, and (4) GOES-G (failed on launch) and GOES-7.
cNOAA decided not to exercise the option for this satellite.
Original GOES Satellites
In 1970, NOAA initiated its original GOES program based on experimental
geostationary satellites developed by NASA. While these satellites
operated effectively for many years, they had technical limitations. For
example, this series of satellites was "spin-stabilized," meaning that the
satellites slowly spun while in orbit to maintain a stable position with
respect to the earth. As a result, the satellite viewed the earth only
about 5 percent of the time and had to collect data very slowly, capturing
one narrow band of data each time its field-of-view swung past the earth.
A complete set of sounding data took 2 to 3 hours to collect.
GOES I-M Series
In 1985, NOAA and NASA began to procure a new generation of GOES, called
the GOES I-M series, based on a set of requirements developed by NOAA's
National Weather Service, NESDIS, and NASA, among others. GOES I-M
consisted of five satellites, GOES-8 through GOES-12, and was a
significant improvement in technology from the original GOES satellites.
For example, GOES I-M was "body-stabilized," meaning that the satellite
held a fixed position in orbit relative to the earth, thereby allowing for
continuous meteorological observations. Instead of maintaining stability
by spinning, the satellite would preserve its fixed position by
continuously making small adjustments in the rotation of internal momentum
wheels or by firing small thrusters to compensate for drift. These and
other enhancements meant that the GOES I-M satellites would be able to
collect significantly better quality data more quickly than the older
series of satellites.
GOES-N Series
In 1998, NOAA began the procurement of satellites to follow GOES I-M,
called the GOES-N series. This series used existing technologies for the
instruments and added system upgrades, including an improved power
subsystem and enhanced satellite pointing accuracy. Furthermore, the
GOES-N satellites were designed to operate longer than its predecessors.
This series originally consisted of four satellites, GOES-N through
GOES-Q. However, the option for the GOES-Q satellite was cancelled based
on NOAA's assessment that it would not need the final satellite to
continue weather coverage. In particular, the agency found that the GOES
satellites already in operation were lasting longer than expected and that
the first satellite in the next series could be available to back up the
last of the GOES-N satellites. As noted earlier, the first GOES-N series
satellite-GOES-13-was launched in May 2006. The GOES-O and GOES-P
satellites are currently in production and are expected to be launched in
July 2008 and July 2011, respectively.
Planned GOES-R Series
NOAA is currently planning to procure the next series of GOES satellites,
called the GOES-R series. This series will consist of four satellites,
GOES-R through GOES-U, and is intended to provide the first major
technological advance in instrumentation since the first satellite of the
GOES I-M series was launched in 1994.3
GOES-R Program-An Overview
NOAA is planning for the GOES-R program to improve on the technology of
prior GOES series, in terms of both system and instrument improvements.
The system improvements are expected to fulfill more demanding user
requirements and to provide more rapid information updates. Table 2
highlights key system-related improvements GOES-R is expected to make to
the geostationary satellite program.
Table 2: Summary of Key GOES-R System Improvements
Source: GAO analysis of NOAA data.
The instruments on the GOES-R series are expected to significantly
increase the clarity and precision of the observed environmental data.
NOAA plans to acquire five different types of instruments. The program
office considers two of the instruments-the Advanced Baseline Imager and
the Hyperspectral Environmental Suite-to be most critical because they
will provide data for key weather products. Table 3 summarizes the planned
instruments and their expected capabilities.
3The instruments were based on 1980s technology.
Table 3: Expected GOES-R Series Instruments, as of June 2006
Source: GAO analysis of NOAA data.
GOES-R Program Office Structure
The program management structure for the GOES-R program differs from past
GOES programs. Prior to the GOES-R series, NOAA was responsible for
program funding, procurement of the ground elements, and on-orbit
operation of the satellites, while NASA was responsible for the
procurement of the spacecraft, instruments, and launch services. NOAA
officials stated that this approach limited the agency's insight and
management involvement in the procurement of major elements of the system.
Alternatively, under the GOES-R management structure, NOAA has
responsibility for the procurement and operation of the overall
system-including spacecraft, instruments, and launch services. NASA is
responsible for the procurement of the individual instruments until they
are transferred to the overall GOES-R system contractor for completion and
integration onto the spacecraft. Additionally, to take advantage of NASA's
acquisition experience and technical expertise, NOAA located the GOES-R
program office at NASA's Goddard Space Flight Center. It also designated
key program management positions to be filled with NASA personnel (see
fig. 3). These positions include the deputy system program director role
for advanced instrument and technology infusion, the project manager for
the flight portion of the system, and the deputy project manager for the
ground and operations portion of the system. NOAA officials explained that
they changed the management structure for the GOES-R program in order to
streamline oversight and fiduciary responsibilities, but that they still
plan to rely on NASA's expertise in space system acquisitions.
Figure 3: GOES-R Program Office Structure and Staffing
Satellite Programs Often Experience Technical Problems, Cost Overruns, and
Schedule Delays
Satellite programs are often technically complex and risky undertakings,
and as a result, they often experience technical problems, cost overruns,
and schedule delays. We and others have reported on a historical pattern
of repeated missteps in the procurement of major satellite systems,
including the National Polar-orbiting Operational Environmental Satellite
System (NPOESS), the GOES I-M series, the Space Based Infrared System High
Program (SBIRS-High), and the Advanced Extremely High Frequency Satellite
System (AEHF).4 Table 4 lists key problems experienced with these programs
and is followed by a summary of each program.
4GAO, Defense Acquisitions: Space System Acquisition Risks and Keys to
Addressing Them, GAO-06-776R (Washington, D.C.: June 1, 2006);
Polar-orbiting Operational Environmental Satellites: Cost Increases
Trigger Review and Place Program's Direction on Hold, GAO-06-573T
(Washington, D.C.: Mar. 30, 2006); Polar-orbiting Operational
Environmental Satellites: Technical Problems, Cost Increases, and Schedule
Delays Trigger Need for Difficult Trade-off Decisions, GAO-06-249T
(Washington, D.C.: Nov. 16, 2005); Polar-orbiting Environmental
Satellites: Information on Program Cost and Schedule Changes, GAO-04-1054
(Washington, D.C.: Sept. 30, 2004); Defense Acquisitions: Despite
Restructuring, SBIRS High Program Remains at Risk of Cost and Schedule
Overruns, GAO-04-48 (Washington, D.C.: Oct. 31, 2003); Military Space
Operations: Common Problems and Their Effects on Satellite and Related
Acquisitions, GAO-03-825R (Washington, D.C.: June 2, 2003); Defense
Acquisitions: Assessments of Major Weapon Programs, GAO-03-476
(Washington, D.C.: May 15, 2003); Weather Satellites: Action Needed to
Resolve Status of the U.S. Geostationary Satellite Program,
GAO/NSIAD-91-252 (Washington, D.C.: July 24, 1991). Defense Science
Board/Air Force Scientific Advisory Board Joint Task Force, Report on the
Acquisition of National Security Space Programs (May 2003).
Table 4: Key Problems Experienced on Selected Major Space Systems
Source: GAO analysis of NOAA and DOD data.
National Polar-orbiting Operational Environmental Satellite System
NPOESS is being developed to combine two separate polar-orbiting
environmental satellite systems currently operated by NOAA and the
Department of Defense (DOD) into a single state-of-the-art environment
monitoring system. A tri-agency program office-comprised of officials from
DOD, NOAA, and NASA-is responsible for managing this program. Within the
program office, each agency has the lead on certain activities. NOAA has
overall program management responsibility for the converged system and for
satellite operations; DOD has the lead on the acquisition; and NASA has
primary responsibility for facilitating the development and incorporation
of new technologies into the converged system.
Since its inception, the NPOESS program has encountered cost overruns and
schedule delays. Specifically, within a year of the contract award, the
program cost estimate increased by $1.2 billion, from $6.9 billion to $8.1
billion, and the expected availability of the first satellite was delayed
by 20 months. We reported in September 2004 that these cost increases and
schedule delays were caused, in part, by changes in the NPOESS funding
stream.5 Subsequently, in November 2005, we reported that problems in the
development of a critical sensor would likely cause program costs to grow
by at least another $3 billion and the schedule for the first launch would
likely be delayed by almost 3 years.6 The senior executive oversight
committee for NPOESS was expected to make a decision in December 2005 on
the direction of the program-which involved increased costs, delayed
schedules, and reduced functionality. We urged this committee to make a
decision quickly so that the program could proceed. However, in late
November 2005, the NPOESS program's anticipated cost growth triggered a
legislative requirement forcing DOD to reassess its options and to
recertify the program.7 In June 2006, DOD decided to reduce the system's
capabilities and number of satellites from six to four, and announced that
the newly-restructured program was estimated to cost $11.5 billion and the
launch of the first satellite had been delayed by at least 4 years from
the time the contract was awarded.
NPOESS' problems involved a number of factors, including unrealistic cost
and schedule estimates, insufficient technical maturity of critical
sensors at a key development milestone, poor performance at multiple
levels of contractor and government management, insufficient executive
oversight, and excessive award fee payments to the contractor.
Specifically, in 2003, an Air Force cost group performed an independent
cost estimate for NPOESS and found that, based on actual outcomes from
historical programs similar to NPOESS, the program office underestimated
contract costs by almost $1 billion. This group also concluded that the
program office underestimated the required time needed to integrate the
sensors onto the spacecraft by almost 80 percent. Despite the differences
in planned cost and schedule, the program office moved forward with its
own estimates-and, in turn, established unrealistic budgets that led, in
part, to the eventual restructuring of the program.
5 GAO-04-1054 .
6 GAO-06-573T and GAO-06-249T .
710 U.S.C. section 2433, as amended by Pub. Law No. 109-163, Div. A,
section 802 (Jan. 6, 2006).
Further, an independent review team charged with assessing the NPOESS
program found that the program management office did not sufficiently
validate the subcontractors' design work on various sensors. As a result,
the sensors were approved to move into production before they reached a
sufficient level of technical maturity. This resulted in unexpected
technical problems during sensor production.
We also reported that the development issues on one critical sensor were
attributed, in part, to the subcontractor's inadequate project
management.8 Specifically, after a series of technical problems, internal
review teams sent by the prime contractor and the program office found
that the sensor's subcontractor had deviated from a number of contract,
management, and policy directives set out by the main office and that both
management and process engineering were inadequate. Neither the contractor
nor the program office recognized the underlying problems in time to fix
them. Further, an independent review team reported that the program
management office did not have the technical system engineering support it
needed to effectively manage the contractor. In addition, the program
office and contractor set aside less than 10 percent of their budgets in
management reserve-an amount which was insufficient to effectively deal
with these technical problems. With just 2 years into the contract, the
prime contractor had spent or allocated over 90 percent of its reserves.
The involvement of the NPOESS executive leadership committee was also
inconsistent and indecisive-it wavered from frequent heavy involvement to
occasional meetings with few resulting decisions. In the 32-month period
from May 2003 through December 2005, the committee met formally six times.
Despite mounting evidence of the seriousness of the critical sensor
problems, the committee did not effectively challenge the program
manager's optimistic assessments, and from May 2003 through December 2004,
convened only twice to consider the program's status.9
8 GAO-06-249T .
9Department of Commerce Office of Inspector General, Poor Management
Oversight and Ineffective Incentives Leave NPOESS Program Well Over Budget
and Behind Schedule, OIG-17794-6-0001 (May 8, 2006).
In May 2006, the Department of Commerce's (Commerce) Inspector General
reported that the NPOESS award fee structure was not an effective system
for promoting high-quality performance by the contractor. Despite the
significant delays and cost overruns on the program, the contractor
received about 84 percent of the available fee pool for the first six
billing periods.
GOES I-M Series
In its development of the GOES I-M series, NOAA experienced severe
technical challenges, massive cost overruns, and risky schedule delays.
The overall development cost of the program was over three times greater
than planned, escalating from $640 million to approximately $2 billion.
Additionally, the launch of the first satellite of this series, which had
been planned for July 1989, did not occur until April 1994. This nearly
5-year schedule delay left NOAA in danger of losing geostationary
satellite coverage, although no gap in coverage occurred. We reported that
these problems were caused by a number of factors, including insufficient
technical readiness of the satellite design prior to contract award,
unrealistic cost and schedule estimates, and inadequate management by NOAA
and NASA.10
Specifically, NOAA and NASA did not require any engineering analyses to be
completed prior to the award of the GOES I-M contract. As a result, both
agencies were unable to anticipate the level of complexity of NOAA's
requirements (related to the satellite's pointing accuracy) or the
contractor's approach to meeting those requirements. This unanticipated
design complexity led to additional analyses, redesigns, and remanufacture
of parts, which resulted in increased costs and schedule delays.
Additionally, the lack of adequate understanding of the system prior to
contract award also prevented program officials from establishing
realistic cost and schedule estimates for the program.
The inadequate management of the GOES I-M program-by both the government
and contractor-played a significant part in its cost increases and program
delays. Specifically, NASA and NOAA made the decision to forgo preliminary
studies of the system because of fiscal constraints and pressure to launch
the first satellite as quickly as possible. This decision was compounded
by NASA's limited technical support in the areas of optics, satellite
control systems, and thermal engineering. Additionally, both the prime
contractor and major subcontractor had little experience in directing the
design of complex weather instruments. The subcontractor had also noted
that it was not prepared for GOES I-M. For example, the instruments were
expected to meet manufacturing and testing standards that the
subcontractor had never experienced before. We recommended Congress
consider directing NASA and NOAA to report on their progress in resolving
these problems and the timeframe and cost for achieving proposed
solutions. Further, we recommended that funds for the production and
testing of the satellites be withheld until a favorable solution was
identified and reported to Congress.
10 GAO/NSIAD-91-252 .
Space Based Infrared System High Program
SBIRS-High satellites are being developed to replace DOD's older missile
warning satellites. In addition to missile warning and missile defense
missions, the satellites are also expected to perform technical
intelligence and battlespace characterization missions. After the program
was initiated in 1994, it faced cost, scheduling, and technology problems.
SBIRS-High had experienced schedule slips of at least 6 years and cost
increases that have triggered legislative requirements to reassess and
recertify the program several times-most recently in 2005. While DOD's
total program cost estimate was initially about $3.9 billion, it is now
$9.9 billion-nearly a 150 percent unit cost increase. DOD is currently
reexamining this program, potential alternatives, and cost estimates.
Our reviews have attributed past problems on the SBIRS-High program to
serious hardware and software design problems, insufficient oversight of
contractors, and technology challenges.11 Further, an independent review
team chartered by DOD reported that a root cause of these problems was
that system requirements were not well understood by DOD when the program
began. Specifically, the requirements-setting process was often ad hoc,
many decisions on requirements were deferred to the contractor, and the
program was too immature to enter system design and development. As a
result, there was too much instability on the program after the contract
award-leading DOD to undertake four major replanning efforts. We made
multiple recommendations to improve this program, including commissioning
an independent task force to assess the development schedule, the
stability of the program design, and software development practices, and
to provide guidance for addressing the program's underlying problems. In
addition, we recommended that DOD establish a mechanism for ensuring that
the knowledge gained from the assessment was used to determine whether
further programmatic changes were needed to strengthen oversight, adjust
cost and schedule estimates, and address requirements changes.
11 GAO-04-48 and GAO-03-476 .
Advanced Extremely High Frequency Satellite System
AEHF is a satellite system intended to be DOD's next generation of
high-speed, protected communication satellites and to replace the existing
Milstar system. In 2003, we reported that cost estimates developed by the
Air Force for this program increased from $4.4 billion in January 1999 to
$5.6 billion in June 2001 for five satellites.12 Moreover, DOD would not
meet its accelerated target date for launching the first satellite in
December 2004. To minimize costs, DOD then decided to purchase three
satellites with options to purchase the fourth and fifth-which brought the
program cost to $4.7 billion. Despite this action, AEHF costs grew to
about $6.1 billion-an increase of more than 15 percent over the baseline
estimate, which triggered legislative requirements to assess and certify
the program. Schedule slippages for launching this communication system
have now stretched to over 3 years.
A number of factors contributed to cost and schedule overruns and
performance shortfalls. First, in the early phases of the AEHF program,
DOD substantially and frequently altered requirements-resulting in major
design modifications that increased costs by millions of dollars. For
instance, a new requirement for additional anti-jamming protection led to
a cost increase of $100 million and an added set of requirements for
training, support, and maintainability that cost an additional $90
million. Second, based on a satellite constellation gap caused by the
failure of a Milstar satellite, DOD accepted a high-risk schedule that
turned out to be overly optimistic and highly compressed-leaving little
room for error and depending on a chain of events taking place at certain
times. Third, AEHF allocated 4 percent of its budget to management
reserve-which was an inadequate amount to cover unforeseen problems for
the duration of the program. Between December 2002 and June 2005, the
contractor had depleted about 86 percent of its reserves with 5 years
remaining on the contract. Lastly, at the time DOD decided to accelerate
the program, it did not have the funding needed to support the activities
or the manpower needed to design and build the satellites more quickly.
The lack of funding also contributed to schedule delays, which in turn,
caused more cost increases. We made a number of recommendations to improve
this program and others, including implementing processes and policies
that stabilize requirements and addressing shortfalls in staff with
science and engineering backgrounds. These recommendations were made to
assure that DOD had an investment strategy in place that would better
match resources to requirements.
12 GAO-03-825R and GAO-03-476 .
GOES-R Procurement Activities Are Under Way, but System Requirements and Cost
Estimates May Change
NOAA is nearing the end of the preliminary design phase on its GOES-R
program and plans to award a contract for the system's development in
August 2007; however, because of concerns with potential cost growth,
NOAA's plans for the GOES-R procurement could change in the near future.
To date, NOAA has issued contracts for the preliminary design of the
overall GOES-R system to three vendors and expects to award a contract to
one of these vendors to develop the system in August 2007. In addition, to
reduce the risks associated with developing new instruments, NASA has
issued contracts for the early development of one critical instrument and
for the preliminary designs of four other instruments. The agency plans to
award these contracts and then turn them over to the contractor
responsible for the overall GOES-R program. However, this approach is
under review and NOAA may wait until the instruments are fully developed
before turning them over to the system contractor. Table 5 provides a
summary of the status of contracts for the GOES-R program.
Table 5: Status of GOES-R Program Contracts, as of June 2006
Source: NOAA.
According to program documentation provided to the Office of Management
and Budget in 2005, the current life cycle cost estimate for GOES-R is
approximately $6.2 billion (see table 6). However, program officials
reported that this estimate is over 2 years old and is under review.
Table 6: GOES-R Program Life Cycle Cost Estimate, as of June 2006
Source: NOAA.
NOAA is tentatively planning to launch the first GOES-R series satellite
in September 2012. The development of the schedule for launching the
satellites was driven by a requirement that the satellites be available to
back up the last remaining GOES satellites (GOES-O and GOES-P) should
anything go wrong during the planned launches of these satellites. Table 7
provides a summary of the planned launch schedule for the GOES-R series.
Table 7: GOES-R Program Schedule, as of June 2006
Source: NOAA.
aGOES-O and GOES-P are not part of the GOES-R series program. Their launch
dates are provided because of their relevance to the GOES-R series
satellite schedules.
bBecause GOES satellites have been operating longer than expected, NOAA is
considering moving the planned launch of the GOES-P satellite to July
2011.
Commerce is scheduled to make a major acquisition decision before the end
of this year. Commerce will decide whether or not the GOES-R series should
proceed into the development and production phase in December 2006.
Program officials reported that the final request for proposal on the
GOES-R contract would be released upon completion of this decision
milestone.
However, NOAA's plans for the GOES-R procurement could change in the near
future because of concerns with potential cost growth. Given its
experiences with cost growth on the NPOESS acquisition, NOAA recently
asked program officials to recalculate the total cost of the estimated
$6.2 billion GOES-R program. In May 2006, program officials estimated that
the life cycle cost could reach $11.4 billion. The agency then requested
that the program identify options for reducing the scope of requirements
for the satellite series. Program officials reported that there are over
10 viable options under consideration, including options for removing one
or more of the planned instruments. The program office is also
reevaluating its planned acquisition schedule based on the potential
program options. Specifically, program officials stated that if there is a
decision to make a major change in system requirements, they will likely
extend the preliminary design phase, delay the decision to proceed into
the development and production phase, and delay the contract award date.
NOAA officials estimated that a decision on the future scope and direction
of the program could be made by the end of September 2006.
The GOES-R Program Office Has Taken Steps to Address Past Lessons Learned, but
Significant Actions Remain
NOAA has taken steps to apply lessons learned from problems encountered on
other satellite programs to the GOES-R procurement. Key lessons include
(1) establishing realistic cost and schedule estimates, (2) ensuring
sufficient technical readiness of the system's components prior to key
decisions, (3) providing sufficient management at government and
contractor levels, and (4) performing adequate senior executive oversight
to ensure mission success. NOAA has established plans designed to mitigate
the problems faced in past acquisitions; however, many activities remain
to fully address these lessons. Until it completes these activities, NOAA
faces an increased risk that the GOES-R program will repeat the increased
cost, schedule delays, and performance shortfalls that have plagued past
procurements.
NOAA Is Taking Steps to Improve the Reliability of Cost and Schedule Estimates,
but Key Steps Remain in Reconciling Cost Estimates
We and others have reported that space system acquisitions are strongly
biased to produce unrealistically low cost and schedule estimates in the
acquisition process. For example, we testified last July on the continued
large cost increases and schedule delays being encountered on military
space acquisition programs-including NPOESS, SBIRS-High, and AEHF.13 We
noted that during program formulation, the competition to win funding is
intense and has led program sponsors to minimize their program cost
estimates.14 Furthermore, a task force chartered by DOD to review the
acquisition of military space programs found that independent cost
estimates and government program assessments have proven ineffective in
countering this tendency.15 NOAA programs face similar unrealistic
estimates. For example, the total development cost of the GOES I-M
acquisition was over three times greater than planned, escalating from
$640 million to $2 billion. The delivery of the first satellite was
delayed by 5 years.
NOAA has several efforts under way to improve the reliability of its cost
and schedule estimates for the GOES-R program. NOAA's Chief Financial
Officer has contracted with a cost-estimating firm to complete an
independent cost estimate, while the GOES-R program office has hired a
support contractor to assist with its internal program cost estimating.
The program office is re-assessing its estimates based on preliminary
information from the three vendors contracted to develop preliminary
designs for the overall GOES-R system. Once the program office and
independent cost estimates are completed, program officials intend to
compare them and to develop a revised programmatic cost estimate that will
be used in its decision on whether to proceed into system development and
production. In addition, NOAA has planned for an independent review
team-consisting of former senior industry and government space acquisition
experts-to provide an assessment of the program office and independent
cost estimates for this decision milestone. To improve its schedule
reliability, the program office is currently conducting a schedule risk
analysis in order to estimate the amount of adequate reserve funds and
schedule margin needed to deal with unexpected problems and setbacks.
Finally, the NOAA Observing System Council16 submitted a prioritized list
of GOES-R system requirements to the Commerce Undersecretary for approval.
This list is expected to allow the program office to act quickly in
deleting lower priority requirements in the event of severe technical
challenges or shifting funding streams.
13GAO, Space Acquisitions: Stronger Development Practices and Investment
Planning Needed to Address Continuing Problems, GAO-05-891T (Washington,
D.C.: July 12, 2005).
14 GAO-05-891T .
15Defense Science Board/Air Force Scientific Advisory Board Joint Task
Force, Report on the Acquisition of National Security Space Programs (May
2003).
While NOAA acknowledges the need to establish realistic cost and schedule
estimates, several hurdles remain. As discussed earlier, the agency is
considering reducing the requirements for the GOES-R program to mitigate
the increased cost estimates for the program. Therefore, the agency's
efforts to date to establish realistic cost estimates cannot be fully
effective in addressing this lesson until this uncertainty is resolved.
NOAA suspended the work being performed by its independent cost estimator
until a decision is made on the scope of the program. Further, the agency
has not yet developed a process to evaluate and reconcile the independent
and program office cost estimates once final program decisions are made.
Without this process, the agency may lack the objectivity necessary to
counter the optimism of program sponsors and is more likely to move
forward with an unreliable estimate. Until it completes this activity,
NOAA faces an increased risk that the GOES-R program will repeat the cost
increases and schedule delays that have plagued past procurements.
NOAA Is Conducting Preliminary Studies in Order to Avoid Technical Problems in
Later Acquisition Phases, but Steps Remain in Determining Components' Technical
Maturity
Space programs often experience unforeseen technical problems in the
development of critical components as a result of having insufficient
knowledge of the components and their supporting technologies prior to key
decision points. One key decision point is when an agency decides on
whether the component is sufficiently ready to proceed from a preliminary
study phase into a development phase; this decision point results in the
award of the development contract. Another key decision point occurs
during the development phase when an agency decides whether the component
is ready to proceed from design into production (also called the critical
design review). Without sufficient technical readiness at these
milestones, agencies could proceed into development contracts on
components that are not well understood and enter into the production
phase of development with technologies that are not yet mature. For
example:
16NOAA's Observing System Council is the principal advisory council for
NOAA's earth observation and data management activities. It includes
members from each NOAA line office, other relevant councils, and program
offices. The Assistant Administrator for Satellite and Information
Services and the Assistant Administrator for Weather Services serve as the
co-chairs of the council.
o On the GOES I-M series, NOAA and NASA did not require
engineering analyses prior to awarding the development contracts
in order to accelerate the schedule and launch the first
satellite. The lack of these studies resulted in unexpected
technical issues in later acquisition phases-including the
inability of the original instrument designs to withstand the
temperature variations in the geostationary orbit.
o Both the NPOESS and SBIRS-High programs committed funds for
system development before the design was proven and before the
technologies had properly matured. For instance, at the critical
design review milestone for a key NPOESS sensor, the program
office decided that the sensor was ready to proceed into
production even though an engineering model had not been
constructed. This sensor has since faced severe technical
challenges that directly led to program-wide cost and schedule
overruns.
To address the lesson learned from the GOES I-M experience, in
1997, NOAA began preliminary studies on technologies that could be
used on the GOES-R instruments. These studies target existing
technologies and assessed how they could be expanded for GOES-R.
The program office is also conducting detailed trade-off studies
on the integrated system to improve its ability to make decisions
that balance performance, affordability, risk, and schedule. For
instance, the program office is analyzing the potential
architectures for the GOES-R constellation of satellites-the
quantity and configuration of satellites, including how the
instruments will be distributed over these satellites. These
studies are expected to allow for a more mature definition of the
system specifications.
NOAA has also developed plans to have an independent review team
assess project status on an annual basis once the overall system
contract has been awarded. In particular, this team will review
technical, programmatic, and management areas; identify any
outstanding risks; and recommend corrective actions. This measure
is designed to ensure that sufficient technical readiness has been
reached prior to the critical design review milestone. The program
office's ongoing studies and plans are expected to provide greater
insight into the technical requirements for key system components
and to mitigate the risk of unforeseen problems in later
acquisition phases.
However, the progress currently being made on the only instrument
currently under development-the Advanced Baseline Imager-has
experienced technical problems and could be an indication of more
problems to come in the future. These problems relate to, among
other things, the design complexity of the instrument's detectors
and electronics. As a result, the contractor is experiencing
negative cost and schedule performance trends. As of May 2006, the
contractor incurred a total cost overrun of almost $6 million with
the instrument's development only 28 percent complete. In
addition, from June 2005 to May 2006, it was unable to complete
approximately $3.3 million worth of work. Unless risk mitigation
actions are aggressively pursued to reverse these trends, we
project the cost overrun at completion to be about $23 million.
(See app. II for further detail on the Advanced Baseline Imager's
cost and schedule performance.)
While NOAA expects to make a decision on whether to move the
instrument into production (a milestone called the critical design
review) in January 2007, the contractor's current performance
raises questions as to whether the instrument designs will be
sufficiently mature by that time. Further, the agency does not
have a process to validate the level of technical maturity
achieved on this instrument or to determine whether the contractor
has implemented sound management and process engineering to ensure
that the appropriate level of technical readiness can be achieved
prior to the decision milestone. Until it does so, NOAA risks
making a poor decision based on inaccurate or insufficient
information-which could lead to unforeseen technical problems in
the development of this instrument.
Efforts to Strengthen Government and Contractor Management are Under
Way, but Significant Work on Program Controls Remains
In the past, we have reported on poor performance in the
management of satellite acquisitions.17 The key drivers of poor
management included inadequate systems engineering and earned
value management18 capabilities, unsuitable allocation of contract
award fees, inadequate levels of management reserve, and
inefficient decision-making and reporting structure within the
program office. The NPOESS program office lacked adequate program
control capabilities in systems engineering and earned value
management to effectively manage the contractor's cost, schedule,
and technical performance. Furthermore, Commerce's Inspector
General reported that NOAA awarded the NPOESS contractor excessive
award fees for a program plagued with severe technical problems
and a consistent failure to meet cost and schedule targets.19
Additionally, on SBIRS-High, the program management office had
fewer systems engineers than other historical space programs. As a
result, the program did not have enough engineers to handle the
workload of ensuring that system requirements properly flowed down
into the designs of the system's components. Further, the NPOESS
and AEHF programs had less than 5 percent of funds allocated to
management reserve at the start of the system's development and
spent or allocated over 85 percent of that reserve within 3 years
of beginning development. On GOES I-M, NOAA found that it did not
have the ability to make quick decisions on problems because the
program office was managed by another agency.
NOAA has taken numerous steps to restructure its management
approach on the GOES-R procurement in an effort to improve
performance and to avoid past mistakes. These steps include:
o The program office revised its staffing profile to provide for
government staff to be located on-site at prime contractor and key
subcontractor locations.
o The program office plans to increase the number of resident
systems engineers from 31 to 54 to provide adequate government
oversight of the contractor's system engineering, including
verification and validation of engineering designs at key decision
points (such as the critical design review milestone).
o The program office has better defined the role and
responsibilities of the program scientist, the individual who is
expected to maintain an independent voice with regard to
scientific matters and advise the program manager on related
technical issues and risks.
o The program office also intends to add three resident
specialists in earned value management to monitor contractor cost
and schedule performance.
o NOAA has work under way to develop the GOES-R contract award
fee structure and the award fee review board that is consistent
with our recent findings, the Commerce Inspector General's
findings, and other best practices, such as designating a
non-program executive as the fee-determining official to ensure
objectivity in the allocation of award fees.
o NOAA and NASA have implemented a more integrated management
approach that is designed to draw on NASA's expertise in satellite
acquisitions and increase NOAA's involvement on all major
components of the acquisition.
o The program office reported that it intended to establish a
management reserve of 25 percent consistent with the
recommendations of the Defense Science Board Report on Acquisition
of National Security Space Programs.20
While these steps should provide more robust government oversight
and independent analysis capabilities, more work remains to be
done to fully address this lesson. Specifically, the program
office has not determined the appropriate level of resources it
needs to adequately track and oversee the program and the planned
addition of three earned value management specialists may not be
enough as acquisition activities increase. By contrast, after its
recent problems and in response to the independent review team
findings, NPOESS program officials plan to add 10 program staff
dedicated to earned value, cost, and schedule analysis. An
insufficient level of established capabilities in earned value
management places the GOES-R program office at risk of making poor
decisions based on inaccurate and potentially misleading
information. Finally, while NOAA officials believe that assuming
sole responsibility for the acquisition of GOES-R will improve
their ability to manage the program effectively, this change also
elevates NOAA's risk for mission success. Specifically, NOAA is
taking on its first major system acquisition and an increased risk
due to its lack of experience. Until it fully addresses the lesson
of ensuring an appropriate level of resources to oversee its
contractor, NOAA faces an increased risk that the GOES-R program
will repeat the management and contractor performance shortfalls
that have plagued past procurements.
NOAA Has Established a Senior Executive Committee to Perform
Critical Oversight of the GOES-R Program
We and others have reported on NOAA's significant deficiencies in
its senior executive oversight of NPOESS.21 The lack of timely
decisions and regular involvement of senior executive management
was a critical factor in the program's rapid cost and schedule
growth. The senior executive committee was provided with monthly
status reports that consistently described in explicit detail the
growing costs and delays attributable to the development of a key
instrument. Despite mounting evidence of the seriousness of the
instrument's problems, this committee convened only twice between
May 2003 and December 2004 to consider the program's status.
NOAA formed its program management council in response to the lack
of adequate senior executive oversight on NPOESS. In particular,
this council is expected to provide regular reviews and
assessments of selected NOAA programs and projects-the first of
which is the GOES-R program. The council is headed by the NOAA
Deputy Undersecretary and includes senior officials from Commerce
and NASA. The council is expected to hold meetings to discuss
GOES-R program status on a monthly basis and to approve the
program's entry into subsequent acquisition phases at key decision
milestones-including contract award and critical design reviews,
among others. Since its establishment in January 2006, the council
has met regularly and has established a mechanism for tracking
action items to closure.
The establishment of the NOAA Program Management Council is a
positive action that should support the agency's senior-level
governance of the GOES-R program. In moving forward, it is
important that this council continue to meet on a regular basis
and exercise diligence in questioning the data presented to it and
making difficult decisions. In particular, it will be essential
that the results of all preliminary studies and independent
assessments on technical maturity of the system and its components
be reviewed by this council so that an informed decision can be
made about the level of technical complexity it is taking on when
proceeding past these key decision milestones. In light of the
recent uncertainty regarding the future scope and cost of the
GOES-R program, the council's governance will be critical in
making those difficult decisions in a timely manner.
Conclusions
Procurement activities are under way for the next series of
geostationary environmental satellites, called the GOES-R
series-which is scheduled to launch its first satellite in
September 2012. With the GOES-R system development contract
planned for award in August 2007, NOAA is positioning itself to
improve the acquisition of this system by incorporating the
lessons learned from other satellite procurements, including the
need to establish realistic cost estimates, ensure sufficient
government and contractor management, and obtain effective
executive oversight. However, further steps remain to fully
address selected lessons. Specifically, NOAA has not yet developed
a process to evaluate and reconcile the independent and government
cost estimates. In addition, NOAA has not yet determined how it
will ensure that a sufficient level of technical maturity will be
achieved in time for an upcoming decision milestone, or determined
the appropriate level of resources it needs to adequately track
and oversee the program using earned value management. Until it
completes these activities, NOAA faces an increased risk that the
GOES-R program will repeat the increased cost, schedule delays,
and performance shortfalls that have plagued past procurements.
Recent concerns about the potential for cost growth on the GOES-R
procurement have led the agency to consider reducing the scope of
requirements for the satellite series. A decision on the future
scope and direction of the program could by made by the end of
September 2006. Once the decision is made, it will be important to
move quickly to implement the decision in the agency budgets and
contracts.
Recommendations for Executive Action
To improve NOAA's ability to effectively manage the procurement of
the GOES-R system, we recommend that the Secretary of Commerce
direct its NOAA Program Management Council to take the following
three actions:
o Once the scope of the program has been finalized, establish a
process for objectively evaluating and reconciling the government
and independent life cycle cost estimates.
o Perform a comprehensive review of the Advanced Baseline Imager,
using system engineering experts, to determine the level of
technical maturity achieved on the instrument, to assess whether
the contractor has implemented sound management and process
engineering, and to assert that the technology is sufficiently
mature before moving the instrument into production.
o Seek assistance from an independent review team to determine
the appropriate level of resources needed at the program office to
adequately track and oversee the contractor's earned value
management. Among other things, the program office should be able
to perform a comprehensive integrated baseline review after system
development contract award, provide surveillance of contractor
earned value management systems, and perform project scheduling
analyses and cost estimates.
Agency Comments and Our Evaluation
We received written comments on a draft of this report from the
Department of Commerce (see app. III). In the department's
response, the Deputy Secretary of Commerce agreed with our
recommendations and identified plans for implementing them.
Specifically, the department noted that it plans to establish a
process for reconciling government and independent cost estimates
and to evaluate the process and results with an independent team
of recognized senior experts in the satellite acquisition field.
The department also noted that an independent review team is
planning to perform assessments of the technical maturity of the
Advanced Baseline Imager and the extent to which the program
management structure and reporting process will provide adequate
oversight of the GOES-R system acquisition. Additionally, the
department expressed concern regarding our use of a cost estimate
that they considered to be premature and misleading. During the
course of our review, NOAA provided us with a cost estimate that
was later determined by agency officials to be inaccurate and was
subsequently corrected. We have incorporated the revised cost
estimate of $11.4 billion for the overall GOES-R program to ensure
that all cost estimates reported at this time are accurate.
The department provided additional technical corrections, which we
have incorporated as appropriate.
As agreed with your offices, unless you publicly announce the
contents of this report earlier, we plan no further distribution
until 30 days from the report date. At that time, we will send
copies of this report to interested congressional committees, the
Secretary of Commerce, the Administrator of NASA, the Director of
the Office of Management and Budget, and other interested parties.
In addition, this report will be available at no charge on our Web
site at http:// www.gao.gov .
If you have any questions on matters discussed in this report,
please contact me at (202) 512-9286 or by e-mail at
[email protected] . Contact points for our Offices of Congressional
Relations and Public Affairs may be found on the last page of this
report. GAO staff who made major contributions to this report are
listed in appendix IV.
David A. Powner Director, Information Technology Management Issues
Appendix I: Objectives, Scope, and Methodology
Our objectives were to (1) determine the status of and plans for
the Geostationary Operational Environmental Satellites-R series
(GOES-R) procurement and (2) identify and evaluate the actions
that the project management team is taking to ensure that past
problems experienced in procuring other satellite programs are not
repeated. To accomplish these objectives, we focused our review on
the National Oceanic and Atmospheric Administration's (NOAA)
GOES-R program office, the organization responsible for the
overall GOES-R program.
To determine the status of and plans for the GOES-R series
procurement, we reviewed various program office plans and
management reports such as acquisition schedules, cost estimates,
and planned system requirements. Furthermore, we conducted
interviews with NOAA and National Aeronautics and Space
Administration (NASA) officials to determine key dates for future
GOES-R acquisitions efforts and milestones, and potential changes
in program scope, cost, and schedule.
To identify the steps the GOES-R project management team is taking
to ensure that past problems experienced in procuring other
satellite series are not repeated, we analyzed our past body of
work on major space system acquisitions, including the Advanced
Extremely High Frequency satellites, the GOES I-M satellites, the
National Polar-orbiting Operational Environmental Satellite
System, and the Space Based Infrared System High program in order
to identify key lessons. We also analyzed findings from other
government reports on satellite procurements, such as by the
Defense Science Board-Air Force Scientific Advisory Board Joint
Task Force and the Department of Commerce's Office of Inspector
General. We assessed relevant management documents, such as cost
reports and program risk plans. Our evaluation included the
application of earned value analysis techniques1 to data from
contractor cost performance reports over an 11-month period (from
June 2005 to May 2006). We also conducted interviews with agency
officials to identify and to evaluate the adequacy of the actions
taken to address these lessons.
We obtained comments on a draft of this report from officials at
the Department of Commerce and incorporated these comments as
appropriate. We performed our work at NOAA and NASA offices in the
Washington, D.C., metropolitan area between December 2005 and
August 2006 in accordance with generally accepted government
auditing standards.
Appendix II: Current Shortfalls in Contractor Performance on Key
Instrument Development
The development of one of the critical GOES-R instruments, the
Advanced Baseline Imager (ABI), is experiencing technical
challenges and, as a result, the contractor is missing cost and
schedule targets. Despite the uncertainty regarding the future
scope of the GOES-R program, it is expected that the requirements
for this instrument will not change.
Contractor-provided data from June 2005 to May 2006 indicates that
ABI's cost performance is experiencing a trend of negative
variances. Figure 4 shows the 11-month cumulative cost variance
for the ABI contract. As of May 2006, the contractor has incurred
a total cost overrun of almost $6 million with ABI development
only 28 percent complete. This information is useful because
trends tend to continue and can be difficult to reverse unless
management attention is focused on key risk areas and risk
mitigation actions are aggressively pursued. Studies have shown
that, once work is 15 percent complete, the performance indicators
are indicative of the final outcome.
Based on contractor performance from June 2005 to May 2006, we
estimated that the current ABI instrument contract-which is worth
approximately $360 million-will overrun its budget by between $17
million and $47 million. Our projection of the most likely cost
overrun will be about $23 million. The contractor, in contrast,
estimates about a $7 million overrun at completion of the ABI
contract. Given that the contractor has 72 percent of work
remaining and has already accumulated a cost overrun of $5.9
million, the likelihood that the contractor will meet its
estimated projection is small.
Figure 4: Cumulative Cost Variance of the ABI Instrument Contract
over an 11-month Period
Our analysis also indicates that the contractor has been unable to
meet its planned schedule targets. Figure 5 shows the 11-month
cumulative schedule variance for the ABI contract. From June 2005
to May 2006, the contractor was unable to complete approximately
$3.3 million worth of scheduled work. The contractor reported that
its incorporation of revised subcontractor budgets resulted in the
fluctuations in schedule performance data prior to March 2006. The
current inability to meet contract schedule performance could be a
predictor of future rising costs, as more spending is often
necessary to resolve schedule overruns.
Figure 5: Cumulative Schedule Variance of the ABI Instrument
Contract over an 11-month Period
According to contractor-provided documents, the cost and schedule
overruns were primarily caused by design complexity issues
experienced in the development of the instrument's detectors and
the electronics design for the cryocooler1 and the unplanned time
and manpower expended to resolve these issues. Other significant
cost and schedule drivers include software issues on the scanner
and supplier quality issues on some parts.
Appendix III: Comments from the Department of Commerce
Appendix IV: GAO Contact and Staff Acknowledgments
GAO Contact
David A. Powner, (202) 512-9286 or [email protected] .
Staff Acknowledgments
In addition to the contact named above, Carol Cha, Neil Doherty,
Nancy Glover, Kush Malhotra, Colleen Phillips, and Karen Richey
made key contributions to this report.
GAO�s Mission
The Government Accountability Office, the audit, evaluation and
investigative arm of Congress, exists to support Congress in
meeting its constitutional responsibilities and to help improve
the performance and accountability of the federal government for
the American people. GAO examines the use of public funds;
evaluates federal programs and policies; and provides analyses,
recommendations, and other assistance to help Congress make
informed oversight, policy, and funding decisions. GAO's
commitment to good government is reflected in its core values of
accountability, integrity, and reliability.
Obtaining Copies of GAO Reports and Testimony
The fastest and easiest way to obtain copies of GAO documents at
no cost is through GAO's Web site ( www.gao.gov ). Each weekday,
GAO posts newly released reports, testimony, and correspondence on
its Web site. To have GAO e-mail you a list of newly posted
products every afternoon, go to www.gao.gov and select "Subscribe
to Updates."
Order by Mail or Phone
The first copy of each printed report is free. Additional copies
are $2 each. A check or money order should be made out to the
Superintendent of Documents. GAO also accepts VISA and Mastercard.
Orders for 100 or more copies mailed to a single address are
discounted 25 percent. Orders should be sent to:
U.S. Government Accountability Office 441 G Street NW, Room LM
Washington, D.C. 20548
To order by Phone: Voice: (202) 512-6000 TDD: (202) 512-2537 Fax:
(202) 512-6061
To Report Fraud, Waste, and Abuse in Federal Programs
Contact:
Web site: www.gao.gov/fraudnet/fraudnet.htm E-mail:
[email protected] Automated answering system: (800) 424-5454 or
(202) 512-7470
Congressional Relations
Gloria Jarmon, Managing Director, [email protected] (202) 512-4400
U.S. Government Accountability Office, 441 G Street NW, Room 7125
Washington, D.C. 20548
Public Affairs
Paul Anderson, Managing Director, [email protected] (202)
512-4800 U.S. Government Accountability Office, 441 G Street NW,
Room 7149 Washington, D.C. 20548
17 GAO-06-573T , GAO-06-249T , GAO/NSIAD-91-252 , Defense Acquisitions:
DOD Has Paid Billions in Award and Incentive Fees Regardless of
Acquisition Outcomes, GAO-06-66 (Washington, D.C.: Dec. 19, 2005), and
Weather Satellites: Cost Growth and Development Delays Jeopardize U.S.
Forecasting Ability, GAO/NSIAD-89-169 (Washington, D.C.: June 30, 1989).
18Earned value management is a method, used by DOD for several decades, to
track a contractor's progress in meeting project deliverables. It compares
the value of work accomplished during a given period with that of the work
expected in that period. Differences from expectations are measured in
both cost and schedule variances.
19Department of Commerce Office of Inspector General, Poor Management
Oversight and Ineffective Incentives Leave NPOESS Program Well Over Budget
and Behind Schedule, OIG-17794-6-0001 (May 8, 2006).
20Defense Science Board/Air Force Scientific Advisory Board Joint Task
Force, Report on the Acquisition of National Security Space Programs (May
2003).
21GAO, Polar-orbiting Environmental Satellites: Cost Increases Trigger
Review and Place Program's Direction on Hold, GAO-06-573T (Washington,
D.C.: March 30, 2006); Department of Commerce Office of Inspector General,
Poor Management Oversight and Ineffective Incentives Leave NPOESS Program
Well Over Budget and Behind Schedule, OIG-17794-6-0001 (May 8, 2006).
1The earned value concept is applied as a means of placing a dollar value
on project status. It is a technique that compares budget versus actual
costs versus project status in dollar amounts. For our analysis, we used
standard earned value formulas to calculate cost and schedule variance and
forecast the range of cost overrun at contract completion.
1The cryocooler is a key component of the ABI instrument. It is intended
to cool down components of the instrument.
(310827)
www.gao.gov/cgi-bin/getrpt? GAO-06-993 .
To view the full product, including the scope
and methodology, click on the link above.
For more information, contact Dave Powner at (202) 512-9286 or
[email protected].
Highlights of GAO-06-993 , a report to the Subcommittee on Environment,
Technology, and Standards, Committee on Science, House of Representatives
September 2006
GEOSTATIONARY OPERATIONAL ENVIRONMENTAL SATELLITES
Steps Remain in Incorporating Lessons Learned from Other Satellite
Programs
The National Oceanic and Atmospheric Administration (NOAA) plans to
procure the next generation of geostationary operational environmental
satellites, called the Geostationary Operational Environmental
Satellites-R series (GOES-R). This new series is considered critical to
the United States' ability to maintain the continuity of data required for
weather forecasting through the year 2028. GAO was asked to (1) determine
the status of and plans for the GOES-R series procurement, and (2)
identify and evaluate the actions that the program management team is
taking to ensure that past problems experienced in procuring other
satellite programs are not repeated.
What GAO Recommends
We are making recommendations to the Secretary of Commerce to direct its
NOAA Program Management Council to establish a process for reconciling the
government and independent cost estimates; perform a comprehensive review
of a key instrument prior to moving it into production; and to evaluate
the appropriate levels of resources needed at the program office to
oversee the contractor's performance in meeting cost and schedule targets.
In written comments, the Department of Commerce agreed with the
recommendations and identified plans for implementing them.
NOAA is nearing the end of the preliminary design phase of its GOES-R
system-which was estimated to cost $6.2 billion and scheduled to have the
first satellite ready for launch in 2012. It expects to award a contract
in August 2007 to develop this system. However, according to program
officials, NOAA's plans for the GOES-R procurement could change in the
near future. Recent analyses of the GOES-R program cost-which in May 2006
the program office estimated could reach $11.4 billion-have led the agency
to consider reducing the scope of requirements for the satellite series.
NOAA officials estimated that a decision on the future scope and direction
of the program could be made by the end of September 2006.
NOAA has taken steps to implement lessons learned from past satellite
programs, but more remains to be done. Prior satellite programs-including
a prior GOES series, a polar-orbiting environmental satellite series, and
various military satellite programs-often experienced technical
challenges, cost overruns, and schedule delays. Key lessons from these
programs include the need to (1) establish realistic cost and schedule
estimates, (2) ensure sufficient technical readiness of the system's
components prior to key decisions, (3) provide sufficient management at
government and contractor levels, and (4) perform adequate senior
executive oversight to ensure mission success. NOAA has established plans
to address these lessons by conducting independent cost estimates,
performing preliminary studies of key technologies, placing resident
government offices at key contractor locations, and establishing a senior
executive oversight committee. However, many steps remain to fully address
these lessons (see table). Until it completes these activities, NOAA faces
an increased risk that the GOES-R program will repeat the increased cost,
schedule delays, and performance shortfalls that have plagued past
procurements.
Key Lessons Learned and the Activities Taken or Remaining to Fully Address
Them
Source: GAO analysis.
*** End of document. ***