FROM DOC_DB.DOCUMENT D, DOC_DB.REPORTS R, DOC_DB.BACKGROUND B
*
ERROR at line 2:
ORA-00942: table or view does not exist
-------------------------Indexing Terms-------------------------
REPORTNUM: GAO-06-792
TITLE: WEATHER FORECASTING: National Weather Service Is Planning
to Improve Service and Gain Efficiency, but Impacts of Potential
Changes Are Not Yet Known
DATE: 07/14/2006
-----------------------------------------------------------------
******************************************************************
** This file contains an ASCII representation of the text of a **
** GAO Product. **
** **
** No attempt has been made to display graphic images, although **
** figure captions are reproduced. Tables are included, but **
** may not resemble those in the printed version. **
** **
** Please see the PDF (Portable Document Format) file, when **
** available, for a complete electronic file of the printed **
** document's contents. **
** **
******************************************************************
GAO-06-792
* Results in Brief
* Background
* NWS Office Structure: An Overview
* NWS Relies on Key Systems and Technologies to Fulfill Its Mi
* Next Generation Weather Radar (NEXRAD)
* Automated Surface Observing System (ASOS)
* Operational Environmental Satellites
* Advanced Weather Interactive Processing System (AWIPS)
* Numerical Models
* Supercomputers
* Previous Reports Focused on NWS Modernization Systems Risks
* NWS Established Performance Goals and Tracks Progress agains
* NWS Is Positioning Itself to Provide Better Service through
* NWS Has Plans for Upgrading Its Systems, Models, and Computa
* NEXRAD
* ASOS
* AWIPS
* Numerical Models
* Supercomputers
* NWS Appropriately Links Its System and Technical Upgrades to
* NWS's Training Is Expected to Result in Forecast Service Imp
* Changing Concept of Operations Could Affect Nationwide Offic
* NWS Is Evaluating Changes to Its Current Operations
* Impacts of New Concept of Operations Have Yet to Be Determin
* Conclusions
* Recommendations for Executive Action
* Agency Comments
* GAO Contact
* Staff Acknowledgments
* GAO's Mission
* Obtaining Copies of GAO Reports and Testimony
* Order by Mail or Phone
* To Report Fraud, Waste, and Abuse in Federal Programs
* Congressional Relations
* Public Affairs
Report to the Subcommittee on Environment, Technology, and Standards,
Committee on Science, House of Representatives
United States Government Accountability Office
GAO
July 2006
WEATHER FORECASTING
National Weather Service Is Planning to Improve Service and Gain
Efficiency, but Impacts of Potential Changes Are Not Yet Known
Weather Forecasting Weather Forecasting Weather Forecasting Weather
Forecasting Weather Forecasting Weather Forecasting Weather Forecasting
Weather Forecasting Weather Forecasting Weather Forecasting Weather
Forecasting Weather Forecasting Weather Forecasting Weather Forecasting
Weather Forecasting Weather Forecasting Weather Forecasting Weather
Forecasting Weather Forecasting Weather Forecasting Weather Forecasting
Weather Forecasting Weather Forecasting Weather Forecasting Weather
Forecasting Weather Forecasting Weather Forecasting Weather Forecasting
Weather Forecasting Weather Forecasting Weather Forecasting Weather
Forecasting Weather Forecasting Weather Forecasting Weather Forecasting
Weather Forecasting Weather Forecasting Weather Forecasting Weather
Forecasting Weather Forecasting Weather Forecasting Weather Forecasting
GAO-06-792
Contents
Letter 1
Results in Brief 2
Background 3
NWS Is Positioning Itself to Provide Better Service through Upgrades to
Its Systems and Technologies 20
NWS's Training Is Expected to Result in Forecast Service Improvements, but
the Training Selection Process Lacks Sufficient Oversight 27
Changing Concept of Operations Could Affect Nationwide Office
Configuration, but Impact on Forecast Services, Staffing, and Budget Is
Not Yet Known 28
Conclusions 31
Recommendations for Executive Action 31
Agency Comments 31
Appendix I Objectives, Scope, and Methodology 33
Appendix II NWS Performance Goals for Fiscal Years 2005 to 2011 34
Appendix III NWS Previously Used A Stringent Process to Ensure Service Was
Not Degraded 36
Appendix IV Comments from the Department of Commerce 37
Appendix V GAO Contact and Staff Acknowledgments 40
Tables
Table 1: NWS's Performance Measures, Goals, and Actual Performance for
Fiscal Years (FY) 2005, 2006, and 2011 18
Table 2: Ongoing and Planned NEXRAD Improvements (as of May 31, 2006) 22
Table 3: Ongoing and Planned ASOS Improvements (as of May 31, 2006) 22
Table 4: Ongoing and Planned AWIPS Improvements 23
Table 5: System Upgrades Are Linked to Expected Performance Improvements
26
Figures
Figure 1: NWS's 122 Weather Forecast Offices 5
Figure 2: Overview of Key Systems and Technologies Supporting NWS
Forecasts 6
Figure 3: NEXRAD Radar Tower 8
Figure 4: An ASOS System 9
Figure 5: ASOS Sensors 10
Figure 6: Approximate GOES Geographic Coverage 12
Figure 7: Configuration of Operational Polar Satellites 13
Figure 8: An AWIPS Workstation 14
Figure 9: Weather Model Output Shown on an AWIPS Workstation 15
Abbreviations
ASOS Automated Surface Observing System AWIPS Advanced Weather Interactive
Processing System DMSP Defense Meteorological Satellite Program DOD
Department of Defense FAA Federal Aviation Administration GOES
Geostationary Operational Environmental Satellites NEXRAD Next Generation
Weather Radar NPOESS National Polar-orbiting Operational Environmental
Satellite System NWS National Weather Service NOAA National Oceanic and
Atmospheric Administration OMB Office of Management and Budget POES
Polar-orbiting Operational Environmental Satellites
This is a work of the U.S. government and is not subject to copyright
protection in the United States. It may be reproduced and distributed in
its entirety without further permission from GAO. However, because this
work may contain copyrighted images or other material, permission from the
copyright holder may be necessary if you wish to reproduce this material
separately.
United States Government Accountability Office
Washington, DC 20548
July 14, 2006
The Honorable Vernon J. Ehlers Chairman The Honorable David Wu Ranking
Minority Member Subcommittee on Environment, Technology, and Standards
Committee on Science House of Representatives
The National Weather Service's (NWS) ability to forecast the weather
affects the life and property of every American. The agency's basic
mission is to provide storm and flood warnings and weather forecasts for
the United States, its territories, and adjacent oceans and waters-in
order to protect life and property and to enhance the national economy.
NWS operations also support other agencies by providing aviation and
marine-related weather forecasts and warnings.
To carry out its mission, NWS uses a variety of systems, technologies, and
manual processes to collect, process, and disseminate weather data to and
among its network of field offices and regional and national centers. In
the 1980s and 1990s, NWS undertook a nationwide modernization program to
upgrade its systems and technologies and to consolidate its field office
structure. Today, with the modernization completed, NWS continues to seek
ways to upgrade its systems with a goal of further improving its
forecasting abilities. The agency is also considering changing how its
nationwide office structure works in order to enhance its efficiency.
Because of your interest in these plans for continued improvements, you
asked us to (1) evaluate NWS's efforts to achieve improvements in the
delivery of its services through the upgrades to its systems, models, and
computational abilities; (2) assess agency plans to achieve improvements
in the delivery of its services through the training and professional
development of its employees; and (3) evaluate agency plans to revise its
nationwide office configuration and the implications of these plans on
local forecasting services, staffing, and budgets.
To address these objectives, we reviewed NWS plans for system
enhancements, technology improvements, and professional training, and
assessed the extent to which these plans were tied to the agency's service
improvement goals. We also interviewed officials from NWS and the National
Oceanic and Atmospheric Administration to obtain clarification on agency
plans and goals. To determine the status and potential impact of any plans
to revise the national office configuration, we assessed NWS reports on
ways to enhance its operations and interviewed key officials involved in
these reports.
We conducted our work at NWS headquarters in the Washington, D.C.,
metropolitan area and at NWS offices in Denver, Tampa, and Miami. We
performed our work from October 2005 to June 2006, in accordance with
generally accepted government auditing standards. Additional details on
our objectives, scope, and methodology are provided in appendix I.
Results in Brief
NWS is positioning itself to provide better service through over $315
million in planned upgrades to its systems and technologies through 2011.
In annual plans, the agency links expected improvements in its service
performance measures with the technologies and systems expected to improve
them. For example, NWS expects to reduce the average error in its
forecasts of hurricane paths by approximately 20 nautical miles between
2005 and 2011 through a combination of upgrades to observation systems,
better hurricane forecast models, enhancements to the computer
infrastructure, and research that will be transferred to forecast
operations. Also, NWS expects to increase its lead time on tornado
warnings from 13 to 15 minutes by the end of fiscal year 2008 after the
agency completes an upgrade to its radar system and realizes benefits from
software improvements to its forecaster workstations.
NWS also provides training courses for its employees to help improve its
forecast services, but the agency's process for selecting training lacks
sufficient oversight. Program officials propose and justify training needs
on the basis of up to eight different criteria-including whether a course
is expected to improve NWS forecasting performance measures, support
customer outreach, or increase scientific awareness. Many of these course
justifications appropriately demonstrate support for improved forecasting
performance. For example, training on how to more effectively use
forecaster workstations is expected to help improve tornado, flash flood,
and hurricane warnings. However, in justifying training courses, program
officials routinely link courses to NWS forecasting performance measures.
For example, in 2006, almost all training needs were linked to
expectations for improved forecast performance-including training on
cardiopulmonary resuscitation, spill prevention, and systems security. The
training selection process did not validate or question how these courses
could help improve weather forecasts. The overuse of this justification
undermines the distinctions among different training courses and the
credibility of the course selection process. Until it establishes a
training selection process that uses reliable justifications, NWS risks
selecting courses that do not most effectively support its training goals.
To improve its efficiency, NWS plans to develop a prototype of a new
concept of operations-an effort that could affect its national office
configuration, including the location and functions of its offices
nationwide. However, many details about the impact of any proposed changes
on forecast services, staffing, and budget have yet to be determined.
Further, NWS has not yet determined key activities, timelines, or measures
for evaluating the prototype of the new office operational structure. As a
result, it is not evident that NWS will collect the information it needs
on the impact and benefits of any office restructuring in order to make
sound and cost-effective decisions.
We are making recommendations to the Secretary of Commerce to direct NWS
to strengthen its training selection process; to establish key activities,
timelines, and measures for evaluating the prototype of the new concept of
operations; and to ensure that the plans for evaluating the new concept of
operations address the impact of any changes on budget, staffing, and
services.
The Department of Commerce provided written comments on a draft of this
report in which it agreed with our recommendations and identified planned
steps for implementing them (see app. IV). The department also provided
technical corrections, which we have incorporated in this report as
appropriate.
Background
The mission of NWS-an agency within the Department of Commerce's National
Oceanic and Atmospheric Administration (NOAA)-is to provide weather,
water, and climate forecasts and warnings for the United States, its
territories, and its adjacent waters and oceans, in order to protect life
and property and to enhance the national economy. NWS is the official
source of aviation- and marine-related weather forecasts and warnings, as
well as warnings about life-threatening weather situations.
In the 1980s and 1990s, NWS undertook a nationwide modernization program
to develop new systems and technologies and to consolidate its field
office structure. The goals of the modernization program were to achieve
more uniform weather services across the nation, improve forecasts,
provide more reliable detection and prediction of severe weather and
flooding, permit more cost-effective operations, and achieve higher
productivity. The weather observing systems (including radars, satellites,
and ground-based sensors) and data processing systems that currently
support NWS operations were developed and deployed under the modernization
program. During this period, NWS consolidated over 250 large and small
weather service offices into the office structure currently in use.
NWS Office Structure: An Overview
The coordinated activities of weather facilities throughout the United
States allow NWS to deliver a broad spectrum of climate, weather, water,
and space weather services. These facilities include weather forecast
offices, river forecast centers, national centers, and aviation center
weather service units. The functions of these facilities are described
below.
o 122 weather forecast offices are responsible for providing a
wide variety of weather, water, and climate services for their
local county warning areas, including advisories, warnings, and
forecasts (see fig. 1 for the current location of weather forecast
offices).
o 13 river forecast centers provide river, stream, and reservoir
information to a wide variety of government and commercial users
as well as to local weather forecast offices for use in flood
forecasts and warnings.
o 9 national centers constitute the National Centers for
Environmental Prediction, which provide nationwide computer model
output and manual forecast information to all NWS field offices
and to a wide variety of government and commercial users. These
centers include the Environmental Modeling Center, Storm
Prediction Center, Tropical Prediction Center, Climate Prediction
Center, Aviation Weather Center, and Space Environment Center,
among others.
o 21 aviation center weather service units, which are co-located
with key Federal Aviation Administration (FAA) air traffic control
centers across the nation, provide meteorological support to air
traffic controllers.
Figure 1: NWS's 122 Weather Forecast Offices
NWS Relies on Key Systems and Technologies to Fulfill Its Mission
To fulfill its mission, NWS relies on a national infrastructure of systems
and technologies to gather and process data from the land, sea, and air.
NWS collects data from many sources, including ground-based Automated
Surface Observing Systems (ASOS), Next Generation Weather Radars (NEXRAD),
and operational environmental satellites. These data are integrated by
advanced data processing workstations-called Advanced Weather Interactive
Processing Systems (AWIPS)-used by meteorologists to issue local forecasts
and warnings. The data are also fed into sophisticated computer models
running on high-speed supercomputers, which are then used to help develop
forecasts and warnings. Figure 2 depicts the integration of the various
systems and technologies and is followed by a description of each.
Figure 2: Overview of Key Systems and Technologies Supporting NWS
Forecasts
Next Generation Weather Radar (NEXRAD)
NEXRAD is a Doppler radar system1 that detects, tracks, and determines the
intensity of storms and other areas of precipitation, determines wind
velocities in and around detected storm events, and generates data and
imagery to help forecasters distinguish hazards such as severe
thunderstorms and tornadoes. It also provides information about heavy
precipitation that leads to warnings about flash floods and heavy snow.
The NEXRAD network provides data to other government and commercial users
and to the general public via the Internet.
The NEXRAD network is made up of 158 operational radars and 8
nonoperational radars that are used for training and testing. Of these,
NWS operates 120 radars, the Air Force operates 26 radars, and the FAA
operates 12 radars. These radars are located throughout the continental
United States and in 17 locations outside the continental United States.
Figure 3 shows a NEXRAD radar tower.
1Doppler radar is used to determine the speed and direction of rain or
snow particles, cloud droplets, or dust moving toward or away from the
radar. The radar accomplishes this by sending out a pulse using a stable
frequency and then measuring the changing frequencies as the distance
between the radar and the object changes.
Figure 3: NEXRAD Radar Tower
Automated Surface Observing System (ASOS)
ASOS is a system of sensors, computers, display units, and communications
equipment that automates the ground-based observation and dissemination of
weather information nationwide. This system collects data on temperature
and dew point, visibility, wind direction and speed, pressure, cloud
height and amount, and types and amounts of precipitation. ASOS supports
weather forecast activities and aviation operations, as well as the needs
of research communities that study weather, water, and climate. Figure 4
is a picture of the system, while figure 5 depicts a configuration of ASOS
sensors and describes their functions.
There are currently 1,002 ASOS units deployed across the United States,
with NWS, FAA, and the Department of Defense (DOD) operating 313, 571, and
118 units, respectively.
Figure 4: An ASOS System
Figure 5: ASOS Sensors
Operational Environmental Satellites
Although NWS does not own or operate satellites, geostationary and
polar-orbiting environmental satellite programs are key sources of data
for its operations. NOAA manages the Geostationary Operational
Environmental Satellite (GOES) system and the Polar-orbiting Operational
Environmental Satellite (POES) system. In addition, DOD operates a
different polar satellite program called the Defense Meteorological
Satellite Program (DMSP). These satellite systems continuously collect
environmental data about the Earth's atmosphere, surface, cloud cover, and
electromagnetic environment. These data are used by meteorologists to
develop weather forecasts and other services, and are critical to the
early and reliable prediction of severe storms, such as tornadoes and
hurricanes.
Geostationary satellites orbit above the Earth's surface at the same speed
as the Earth rotates, so that each satellite remains over the same
location on Earth. NOAA operates GOES as a two-satellite system that is
primarily focused on the United States (see fig. 6). To provide continuous
satellite coverage, NOAA acquires several satellites at a time as part of
a series and launches new satellites every few years.2 Three satellites,
GOES-10, GOES-11, and GOES-12, are currently in orbit. Both GOES-10 and
GOES-12 are operational satellites, while GOES-11 is in an on-orbit
storage mode. It is a backup for the other two satellites should they
experience any degradation in service. The first in the next series of
satellites, GOES-13, was launched in May 2006, and the others in the
series, GOES-O and GOES-P, are planned for launch over the next few
years.3 In addition, NOAA is planning a future generation of satellites,
known as the GOES-R series, which are planned for launch beginning in
2014.
2GOES has historically been a joint program between NOAA and the National
Aeronautics and Space Administration (NASA), with NOAA funding and
managing the program and NASA providing engineering and launch
capabilities.
3Satellites in a series are identified by letters of the alphabet when
they are on the ground and by numbers once they are in orbit.
Figure 6: Approximate GOES Geographic Coverage
Unlike the GOES satellites, which maintain a fixed position above the
earth, polar satellites constantly circle the Earth in an almost
north-south orbit, providing global coverage of conditions that affect the
weather and climate. Each satellite makes about 14 orbits a day. As the
Earth rotates beneath it, each satellite views the entire Earth's surface
twice a day. Currently, there are four operational polar-orbiting
satellites-two are POES satellites and two are DMSP satellites. These
satellites are positioned so that they can observe the Earth in early
morning, morning, and afternoon polar orbits. Together, they ensure that
for any region of the Earth, the data are generally no more than 6 hours
old. Figure 7 illustrates the current configuration of operational polar
satellites.
Figure 7: Configuration of Operational Polar Satellites
NOAA and DOD plan to continue to launch remaining satellites in the POES
and DMSP programs, with final launches scheduled for 2007 and 2011,
respectively. In addition, NOAA, DOD, and the National Aeronautics and
Space Administration are planning to replace the POES and DMSP systems
with a state-of-the-art environment monitoring satellite system called the
National Polar-orbiting Operational Environmental Satellite System
(NPOESS). In recent years, we reported on a variety of issues affecting
this major system acquisition.4
4GAO, Polar-orbiting Operational Environmental Satellites: Cost Increases
Trigger Review and Place Program's Direction on Hold, GAO-06-573T
(Washington, D.C.: Mar. 30, 2006); Polar-orbiting Operational
Environmental Satellites: Technical Problems, Cost Increases, and Schedule
Delays Trigger Need for Difficult Tradeoff Decisions, GAO-06-249T
Washington, D.C.: Nov. 16, 2005); Polar-orbiting Environmental Satellites:
Information on Program Cost and Schedule Changes, GAO-04-1054 (Washington,
D.C.: Sept. 30, 2004); Polar-orbiting Environmental Satellites: Project
Risks Could Affect Weather Data Needed by Civilian and Military Users,
GAO-03-987T (Washington, D.C.: July 15, 2003); and Polar-orbiting
Environmental Satellites: Status, Plans, and Future Data Management
Challenges, GAO-02-684T (Washington, D.C.: July 24, 2002).
Advanced Weather Interactive Processing System (AWIPS)
AWIPS is a computer system that integrates and displays all
hydrometeorological data at NWS field offices. This system integrates data
from NEXRAD, ASOS, GOES, and other sources to produce rich graphical
displays to aid forecaster analysis and decision making. AWIPS is used to
disseminate weather information to the national centers, weather offices,
the media, and other federal, state, and local government agencies. NWS
deployed hardware and software for this system to weather forecast
offices, river forecast centers, and national centers throughout the
United States between 1996 and 1999. As a software-intensive system, AWIPS
regularly receives software upgrades called "builds." The most recent
build, called Operational Build 6, is currently being deployed. NWS
officials estimated that the nationwide deployment of this build should be
completed by July 2006. Figure 8 shows a standard AWIPS workstation.
Figure 8: An AWIPS Workstation
Numerical Models
Numerical models are advanced software programs that assimilate data from
satellites and ground-based observing systems and provide short- and
long-term weather pattern predictions. Meteorologists typically use a
combination of models and their own experience to develop local forecasts
and warnings. Numerical weather models are also a critical source for
forecasting weather up to 7 days in advance and forecasting long-term
climate changes. One of NWS's National Centers for Environmental
Prediction, the Environmental Modeling Center, is the primary developer of
these models within NWS and is responsible for making new and improved
models available to regional forecasters via the AWIPS system. Figure 9
depicts model output as shown on an AWIPS workstation.
Figure 9: Weather Model Output Shown on an AWIPS Workstation
Supercomputers
NWS leases high-performance supercomputers to execute numerical
calculations supporting weather prediction and climate modeling. In 2002,
NWS awarded a $227 million contract to lease high-performance
supercomputers to run its environmental models from 2002 through September
2011. Included in this contract are an operational supercomputer used to
run numerical weather models, an identical backup supercomputer located at
a different site, and a research and development supercomputer on which
researchers can test out new analyses and models. The supercomputer lease
contract allows NWS to exercise options to upgrade the processing
capabilities of the operational supercomputer.
Previous Reports Focused on NWS Modernization Systems Risks
During the 1990s, we issued a series of reports on NWS modernization
systems and made recommendations to improve them.5 For example, early in
the AWIPS acquisition, we reported that the respective roles and
responsibilities of the contractor and government were not clear and that
a structured system development environment had not been established. We
made recommendations to correct these shortfalls before the system design
was approved. We also reported that the ASOS system was not meeting
specifications or user needs, and recommended that NWS define and
prioritize system corrections and enhancements. On NEXRAD, we reported
that selected units were falling short of availability requirements and
recommended that NWS analyze and monitor system availability on a
site-specific basis and correct any shortfalls. Because of such concerns,
we identified NWS modernization as a high-risk information technology
initiative in 1995, 1997, and 1999.6
NWS took a number of actions to address our recommendations and to resolve
system risks. For example, NWS enhanced its AWIPS system development
processes, prioritized its ASOS enhancements, and improved the
availability of its NEXRAD systems. In 2001, because of NWS's progress in
addressing key concerns and in deploying and using the AWIPS system-the
final component of its modernization program-we removed the modernization
from our high-risk list.
5See, for example, GAO, Weather Forecasting: Improvements Needed in
Laboratory Software Development Process, GAO/AIMD-95-24 (Washington, D.C.:
Dec. 14, 1994); Weather Forecasting: Unmet Needs and Unknown Costs Warrant
Reassessment of Observing System Plans, GAO/AIMD-95-81 (Washington, D.C.:
Apr. 21, 1995); Weather Forecasting: Radar Availability Requirement Not
Being Met, GAO/AIMD-95-132 (Washington, D.C.: May 31, 1995); Weather
Forecasting: Radars Far Superior to Predecessors, but Location and
Availability Questions Remain, GAO/T-AIMD-96-2 (Washington, D.C.: Oct. 17,
1995); Weather Forecasting: New Processing System Faces Uncertainties and
Risks, GAO/T-AIMD-96-47 (Washington, D.C.: Feb. 29, 1996); Weather
Forecasting: Recommendations to Address New Weather Processing System
Development Risks, GAO/AIMD-96-74 (May 13, 1996); and Weather Satellites:
Planning for the Geostationary Satellite Program Needs More Attention,
GAO/AIMD-97-37 (Washington, D.C.: Mar. 13, 1997).
6GAO, High-Risk Series: An Overview, GAO/HR-95-1 (Washington, D.C.:
February 1995); High-Risk Series: Information Management and Technology,
GAO/HR-97-9 (Washington, D.C.: February 1997); High-Risk Series: An
Update, GAO/HR-99-1 (Washington, D.C.: January 1999); High-Risk Series: An
Update, GAO-01-263 (Washington, D.C.: January 2001).
NWS Established Performance Goals and Tracks Progress against These Goals
In accordance with federal legislation requiring federal managers to focus
more directly on program results, NWS established short- and long-term
performance goals and regularly tracks its actual performance in meeting
these goals.7 Specifically, NWS established 14 different performance
measures-such as lead time for flash floods and false-alarm rates for
tornado warnings. It also established 5-year goals for improving its
performance in each of the 14 performance measures through 2011. For
example, the agency plans to increase its lead time on tornado warnings
from 13 minutes in 2005 to 15 minutes in 2011. Table 1 identifies NWS's 14
performance measures, selected goals, and performance against those goals,
when available. Appendix II provides additional information on NWS's
performance goals.
7The Government Performance and Results Act of 1993 (Pub. L. 103-62) was
intended to improve federal program effectiveness, accountability, and
service delivery by requiring federal agencies to develop strategic plans
with long-term, outcome-oriented goals and objectives; annual performance
goals linked to the long-term goals; and annual reports on actual results.
Table 1: NWS's Performance Measures, Goals, and Actual Performance for
Fiscal Years (FY) 2005, 2006, and 2011
FY05 FY06 FY11
Actual
Performance Actual Goal to On
measure Description Goal (final) met? Goal date target? Goal
Tornado warning The 13 13 Yes 13 13a Yes 15
lead time difference
(minutes) between the
time a
warning is
issued and
the time of
the first
report of a
tornado in a
given county
Tornado warning The 73 75 Yes 76 82a Yes 76
accuracy (percent) percentage of
time a
tornado
actually
occurred in
an area
covered by a
tornado
warning
Tornado warning The 73 77 No 75 76a No 74
false-alarm rate percentage of
(percent) time a
tornado
warning was
issued but no
tornado event
was reported
Flash flood The 48 54 Yes 48 63a Yes 49
warning lead time difference
(minutes) between the
time a
warning is
issued and
the time of
the first
report of a
flash flood
in a given
county
Flash flood The 89 88 No 89 93a Yes 90
warning accuracy percentage of
(percent) time a flash
flood
actually
occurred in
an area
covered by a
flash flood
warning
Marine wind speed A measure of 57 57 Yes 58 56b No 59
forecast accuracy the accuracy
(percent) of wind speed
forecasts
Marine wave height A measure of 67 67 Yes 68 71b Yes 69
forecast accuracy the accuracy
(percent) of wave
forecasts
Aviation forecast The 46 46 Yes 47 45b No 59
Instrument Flight percentage of
Rule time
ceiling/visibility Instrument
accuracy (percent) Flight Rule
conditionse
are predicted
and occur
Aviation forecast The 68 63 Yes 65 61b Yes 50
Instrument Flight percentage of
Rule time
ceiling/visibility Instrument
false-alarm rate Flight Rule
(percent) conditionse
are predicted
but do not
occur
Winter storm The average 15 17 Yes 15 16c Yes 17
warning lead time time from the
(hours) issuance of a
warning to
the time of
the first
report of a
winter storm
in a given
county
Winter storm The 90 91 Yes 90 91c Yes 92
warning accuracy percentage of
(percent) verified
winter storm
events that
were covered
by winter
storm
warnings
Precipitation A score based 27 29 Yes 28 39d Yes 30
forecast day 1 on the
threat (score) agency's
accuracy in
forecasting
precipitation
U.S. seasonal A score based 18 19 Yes 18 24d Yes 20
temperature on the
forecast skill agency's
(score) accuracy in
forecasting
temperature
Hurricane track A measure of 128 101 Yes 111 N/Af N/Af 106
forecast error at the
48 hours (nautical difference
miles) between the
projected
locations of
the center of
storms and
the actual
location in
nautical
miles for the
Atlantic
Basin
Source: GAO analysis of NOAA and NWS reports.
aMetric measured between October 2005 and January 2006.
bMetric measured between October 2005 and February 2006.
cMetric measured between October 2005 and December 2005.
dMetric measured between October 2005 and March 2006.
eInstrument Flight Rule conditions exist when ceilings and visibilities
are less than 1,000 feet and/or 3 miles, respectively, and ceilings and
visibilities are greater than, or equal to, 500 feet and/or 1 mile,
respectively.
fData for this metric are not available until the beginning of the next
calendar year because of the timing of the hurricane season.
NWS periodically adjusts its performance goals as its assumptions change.
After reviewing actual results from previous fiscal years and its
assumptions about the future, in January 2006, NWS adjusted eight of its
5-year performance goals to make more realistic predictions for
performance for the next several years. Specifically, NWS made six
performance goals less stringent and two goals more stringent. The six
goals that were made less stringent-and the reasons for the changes-are
the following:
o Tornado warning lead time: NWS changed its 2011 goal from 17 to
15 minutes of warning because of delays in deploying new
technologies on NEXRAD radars and a lack of access to FAA radar
data.
o Tornado warning false-alarm rate: NWS changed its 2011 goal
from a 70 to 74 percent false-alarm rate for the same reasons
listed above.
o Flash flood warning accuracy: NWS changed its 2011 goal from 91
to 90 percent accuracy after delays on two different systems in
2004, 2005, and 2006.
o Marine wind speed accuracy: NWS changed its 2011 goal from 67
to 59 percent accuracy after experiencing the delay of marine
models and datasets, a deficiency of shallow water wave guidance,
and a reduction in funds for training.
o Marine wave height accuracy: NWS changed its 2011 goal from 77
to 69 percent accuracy for the same reasons identified above for
marine wind speed accuracy.
o Aviation instrument flight rule ceiling/visibility: NWS changed
its goal from 48 to 47 percent accuracy in 2006 because of a
system delay and a reduction in funds for training. Goals for 2007
through 2011 remained the same.
Additionally, the following two goals were made more stringent:
o Aviation instrument flight rule ceiling/visibility false-alarm
rate: NWS reduced its expected false-alarm rate from 68 percent to
65 percent for 2006 because of better than anticipated results
from the AWIPS aviation forecast preparation system and an
aviation learning training course. Goals for the remaining years
in the 5-year plan, 2007 to 2011, remained the same.
o Hurricane track forecasts: NWS changed its 2011 hurricane track
forecast goal from 123 to 106 nautical miles after trends in
observed data from 1987 to 2004 showed that this measure was
improving more quickly than expected.
NWS is positioning itself to provide better service through system
and technology upgrades. Over the next few years, the agency plans
to upgrade and improve its systems, predictive weather models, and
computational abilities, and it appropriately links these upgrades
to its performance goals. For example, planned improvements in
NEXRAD technology are expected to help improve the lead times for
tornado warnings, while AWIPS software enhancements are expected
to help improve the accuracy of marine weather forecasts. The
agency anticipates continued steady improvement in its forecast
accuracy as it obtains better observation data, as computational
resources are increased, and as scientists are better able to
implement advanced modeling and data assimilation techniques.
Over the next few years, NWS has plans to spend over $315 million
to upgrade its systems, models, and computational abilities.8 Some
planned upgrades are to maintain the weather system infrastructure
(either to replace obsolete and difficult-to-maintain parts or to
refresh aging hardware and workstations), while others are to take
advantage of new technologies. Often, the infrastructure upgrades
allow NWS to take advantage of newer technologies. For example,
the replacement of an aging and proprietary NEXRAD subsystem is
expected to allow the agency to implement enhancements in image
resolution. Key planned upgrades for each of NWS's major systems
and technologies are listed below.
NWS has initiated two major NEXRAD improvements. It is currently
replacing an outdated subsystem-the radar data acquisition
subsystem-with current hardware that is compliant with open system
standards. This new hardware is expected to enable important
software upgrades. In addition, NWS plans to add a new technology
called dual polarization to this subsystem, which will provide
more accurate rainfall estimates and differentiate various forms
of precipitation. Table 2 shows the details of these two projects.
NWS Is Positioning Itself to Provide Better Service through Upgrades to Its
Systems and Technologies
NWS Has Plans for Upgrading Its Systems, Models, and Computational Abilities
NEXRAD
8This cost estimate includes the expected cost of key system upgrades, as
well as estimated annual costs for improvements to AWIPS software and
numerical models through the year 2011. It does not include the expected
costs of supercomputer upgrades because NWS does not estimate what portion
of its $26 million annual supercomputer budget is attributable to
upgrades.
Table 2: Ongoing and Planned NEXRAD Improvements (as of May 31, 2006)
Estimated Estimated
Current acquisition completion
Improvement Description status cost date
Radar data A subsystem that In process; $43.8 Estimated
acquisition transmits and receives 107 of 158 million (NWS to be
subsystem radar signals, controls sites have portion is completed
replacement the radar antenna, been $22.6 in late
processes the received installed. million) 2006.
signal, and sends the
processed data to the
radar product generator;
replacement of this
subsystem will enable
software upgrades
including an enhancement
that will allow
operators to view more
detailed weather
features.
Dual A technology upgrade to Acquisition $38 million Expected
polarization allow enhanced target process is (NWS portion contract
technology identification. under way. is $25 award at
upgrade million) the end of
2006.
Deployment
is expected
to begin in
fiscal year
2009 and
end in
fiscal year
2011.
Source: NEXRAD Program Office.
ASOS
NWS has seven ongoing and planned improvements for its ASOS system (see
table 3). Many of these improvements are to replace aging parts and are
expected to make the system more reliable and maintainable. Key subsystem
replacements-including the all-weather precipitation accumulation
gauge-are also expected to result in more accurate measurements.
Table 3: Ongoing and Planned ASOS Improvements (as of May 31, 2006)
Estimated or Estimated
actual or actual
acquisition completion
Improvement Description Current status cost date
Processor Provides a more 962 $6.61 June 30,
upgrade robust processor installations million (NWS 2006
with increased completed out of portion is
capacity, speed, 1002 total $2.89
and memory. planned sites million)
(312
installations
completed out of
313 NWS sites)
All-weather Replaces existing 323 of 331 $7.10 June 30,
precipitation heated tipping installed (303 million 2006
accumulation bucket rain gauge of 311 NWS)
gauge with a gauge that
measures
precipitation by
weight, resulting
in more accurate
measurements.
Dewpoint sensor Replace existing 958 of 1002 $9.20 June 30,
sensor's chilled installed (303 million (NWS 2006
mirror technology of 311 NWS) portion is
with a humidity $3.14
sensitive million)
capacitor.
Ice-free wind Replaces the 231 of 1000 $7.53 November
sensor existing cup and installed (60 of million (NWS 30, 2006
vane anemometer 311 NWS) portion is
with a new $2.90
ultrasonic million)
sensor.
Enhanced Replaces sensor Field $10.14 March 31,
precipitation that only reports demonstration million (NWS 2009
identifier rain and snow testing to begin portion is
with one that is July 2006 $3.55
to report rain, million)
snow, drizzle,
hail, and ice
pellets.
Ceilometer Replaces senor Evaluation of $33 million September
(cloud height) that measures commercial (NWS portion 30, 2011
cloud heights up sensors almost is $12
to 12,000 feet complete; million)
with one that is solicitation for
expected to system
measure cloud development
heights up to expected to
40,000 feet. begin by end of
May 2006.
Sunshine Adds a new sensor Program on hold $1.77 September
duration sensor to measure solar pending million 30, 2011
radiation. ceilometer (this
production; will upgrade
be developed affects only
after the NWS systems)
ceilometer;
planned to
restart by 2010.
Source: ASOS program office.
AWIPS
Selected AWIPS system components have become obsolete, and NWS is
replacing these components. In 2001, NWS began to migrate the existing
Unix-based systems to a Linux system to reduce its dependence on any
particular hardware platform. NWS expects this project, combined with
upgraded information technology, to delay the need for a major information
technology replacement. Table 4 shows planned improvements for the AWIPS
system.
Table 4: Ongoing and Planned AWIPS Improvements
Current Estimated Timeline/estimated
Improvement Description status cost completion date
Linux migration An effort to In $17.92 2002 to 2007
replace legacy progress million
hardware and to
port approximately
4 million source
lines of code of
AWIPS software from
the original
proprietary
Hewlett-Packard
Unix operating
system to the open
source Linux
operating system.
Architecture An effort to refine In $900,000 2004 to 2006
analysis AWIPS hardware and progress
communications
architecture in
support of the
Linux migration and
to build an
advanced Linux
prototype system.
Information An initiative to In $3.22 2004 to 2006
technology replace obsolete progress million
security routers and
firewalls
throughout the
system.
Hardware refresh An initiative to In $53.21 2006 to 2015
keep the AWIPS progress million
hardware baseline
fresh and
maintainable
through a
continuous
technology refresh.
NWS plans to
refresh hardware
components every 4
to 5 years after
the Linux migration
is completed.
Software An initiative to In $23 2006 to 2010
re-architecture reengineer the progress million
AWIPS software
suite to a standard
service-oriented
architecture.
Software Includes efforts to In About $10 Continuous
upgrades enhance advanced progress million
precipitation per year
algorithms for
estimating
rainfall; continue
enhancement of
advanced decision
assistance tools;
implement a
distributed
hydraulic model;
and enhance
forecasting and
evaluation of seas
and lakes to
provide a
prediction
capability tool for
marine forecasters.
Source: NWS.
Numerical Models
NWS plans to continue to improve its modeling capabilities by (1) better
assimilating data from improved observation systems such as ASOS, NEXRAD,
and environmental satellites; (2) developing and implementing an advanced
global forecasting model (called the Weather Research and Forecast model)
to allow forecasters to look at a larger domain area; (3) implementing a
hurricane weather research forecast model; and (4) improving ensemble
modeling, which involves running a single model multiple times with slight
variations on a variable to get a probability that a given forecast is
likely to occur. NWS expects to spend approximately $12.7 million in
fiscal year 2006 to improve its weather and real-time ocean models.
Supercomputers
NWS is planning to exercise an option within its existing supercomputer
lease to upgrade its computing capabilities to allow more advanced
numerical weather and climate prediction modeling.
NWS Appropriately Links Its System and Technical Upgrades to Expected Service
Improvements
In accordance with federal legislation and policy, NWS's planned upgrades
to its systems and technologies are expected to result in improved
service. The Government Performance and Results Act calls for federal
managers to develop strategic performance goals and to focus program
activities on obtaining results.9 Also, the Office of Management and
Budget (OMB) requires agencies to justify major investments by showing how
they support performance goals.10 NOAA and NWS implement the act and OMB
guidance by requiring project officials to describe how planned system and
technology upgrades are linked to the agency's programmatic priorities and
performance measures. Further, in its annual performance plans, NOAA
reports on expected NWS service improvements and identifies the
technologies and systems that are expected to help improve them.
9Pub. L. 103-62, 107 Stat. 285 (1993).
NWS service improvements are often expected through a combination of
system and technology improvements. For example, NWS expects to reduce its
average error in forecasting a hurricane's path by approximately 20
nautical miles between 2005 and 2011 through a combination of upgrades to
observation systems, better hurricane forecast models, enhancements to the
computer infrastructure, and research that will be transferred to NWS
forecast operations. Also, NWS expects tornado warning lead times to
increase from 13 to 15 minutes by the end of fiscal year 2008 after NWS
completes retrofits to the NEXRAD systems, realizes the benefits of AWIPS
software enhancements, and implements new training techniques. Table 5
provides a summary of how system upgrades are expected to result in
service improvements.
10OMB requires agencies to annually submit documentation, called an
exhibit 300, justifying major information technology initiatives or
improvements.
Table 5: System Upgrades Are Linked to Expected Performance Improvements
Expected results of ongoing Primary performance measures
System and planned system upgrades affected
NEXRAD Replacement of the data Tornado warnings lead time
acquisition subsystem is Tornado warnings accuracy
expected to allow future Tornado warnings false-alarm
software and hardware rate Flash flood warning
enhancements. These lead time Flash flood
enhancements are expected to warning accuracy Winter
improve forecasting storm warnings lead time
performance. Winter storm warnings
accuracy
ASOS Processor and sensor Flash flood warning lead
replacements are expected to time Flash flood warning
allow more reliable and accuracy Aviation forecast
maintainable systems. Selected ceiling/visibility accuracy
system improvements-including Aviation forecast
the deployment of an ceiling/visibility
all-weather precipitation false-alarm rate
gauge, an enhanced
precipitation identifier, and
a new ceilometer-are expected
to directly improve
forecasting performance.
AWIPS Infrastructure upgrades Tornado warnings lead time
(including a software Tornado warnings accuracy
migration and hardware Tornado warnings false-alarm
refreshment) are expected to rate Flash flood warning
allow major software lead time Flash flood
enhancements that will result warning accuracy Marine wind
in more accurate and timely speed forecasts accuracy
forecasts. Marine wave height forecasts
accuracy Aviation forecast
ceiling/visibility accuracy
Aviation forecast
ceiling/visibility
false-alarm rate Winter
storm warnings lead time
Supercomputers Increased computational Winter storm warnings lead
capabilities are expected to time Winter storm warnings
allow advanced modeling and accuracy Precipitation
data assimilation-and to forecast day 1 threat score
result in improved forecast U.S. seasonal temperature
accuracy. forecast skill Hurricane
track forecasts at 48 hours
Models Modeling improvements, enabled Flash flood warning lead
by increased supercomputer time Flash flood warning
capacity, are expected to accuracy Marine wind speed
result in more accurate and forecasts accuracy Marine
timely forecasts. wave height forecasts
accuracy Aviation forecast
ceiling/visibility accuracy
Aviation forecast
ceiling/visibility
false-alarm rate Winter
storm warnings lead time
Winter storm warnings
accuracy Precipitation
forecast day 1 threat score
U.S. seasonal temperature
forecast skill Hurricane
track forecasts at 48 hours
Source: GAO analysis of NWS data.
NWS's Training Is Expected to Result in Forecast Service Improvements, but the
Training Selection Process Lacks Sufficient Oversight
NWS provides employee training courses that are expected to help improve
forecast service performance, but the agency's process for selecting this
training lacks sufficient oversight. Each year, NWS identifies its
training needs and develops this training in order to enhance its
services. NWS develops an annual training and education plan identifying
planned training, how this training supports key criteria, and associated
costs for the upcoming year. To develop the annual plan, program area
teams, with representatives from NWS headquarters and field offices,
prioritize and submit training recommendations. Each submission identifies
how the training will support up to eight different criteria-including the
course's effect on NWS forecasting performance measures, NOAA strategic
goals, ensuring operational continuity, and providing customer outreach.
These submissions are screened by a training and education team, and
depending on available resources, selected for development (if not
pre-existing) and implementation. The planned training courses are then
delivered through a variety of means, including courses at the NWS
training center, online training, and training at local forecast offices.
In its 2006 training process, 25 program area teams identified 134
training needs, such as training on how to more effectively use AWIPS,
training on an advanced weather simulator, and training on maintaining
ASOS systems. Given an expected funding level of $6.1 million, the
training and education team then selected 68 of these training needs for
implementation. NWS later identified another 5 training needs and
allocated an additional $1.25 million to its training budget. In total,
NWS funded 73 of 139 training courses.
The majority of planned training courses demonstrate a clear link to
expected forecasting service improvements. For example, NWS developed a
weather event simulator to help forecasters improve their tornado warning
lead times. In addition, AWIPS-related training courses are expected to
help improve each of the agency's 14 forecasting performance measures by
teaching forecasters advanced techniques in using the integrated data
processing workstations.
However, NWS's process for selecting which training courses to implement
lacks sufficient oversight. In justifying training courses, program
officials routinely link proposed courses to NWS forecast performance
measures. Specifically, in 2006, 131 of the 134 original training needs
were linked to expectations for improved forecasting performance-including
training on cardiopulmonary resuscitation, spill prevention, leadership,
systems security, and equal employment opportunity/diversity. The training
selection process did not validate or question that these courses would
improve tornado warning lead times or hurricane warning accuracy. Although
these courses are important and likely justifiable on other bases, the
overuse of this justification undermines the distinctions among training
courses and the credibility of the course selection process. Additionally,
because the training selection process does not clearly distinguish among
courses, it is difficult to determine whether sufficient funds are
dedicated to the courses that are expected improve performance.
NWS training officials acknowledged that some of the course justifications
seem questionable and that more needs to be done to strengthen the
training selection process to ensure oversight of the justification and
prioritization process. They noted that the training division plans to
improve the training selection process over the next few years by adding a
more systematic worker-focused assessment of training needs, better
prioritizing strategic and organizational needs, and initiating
post-implementation reviews. However, until NWS establishes a training
selection process that uses reliable justification and results in
understandable decisions, NWS risks selecting courses that do not most
effectively support its training goals.
Changing Concept of Operations Could Affect Nationwide Office Configuration, but
Impact on Forecast Services, Staffing, and Budget Is Not Yet Known
NWS plans to develop a prototype of a new concept of operations-an effort
that could affect its national office configuration,11 including the
location and functions of its offices nationwide. However, NWS has yet to
determine many details about the impact of any proposed changes on NWS
forecast services, staffing, and budget. Further, NWS has not yet
identified key activities, timelines, or measures for evaluating the
concept of operations prototype. As a result, it is not evident that NWS
will collect the information it needs on the impact and benefits of any
office restructuring in order to make sound and cost-effective decisions.
11Because there is no precise definition of the term "office
configuration," we have defined it as NWS's current number of offices, the
location of the offices, hours worked at each of the offices, and the
services and functions provided at each of the offices.
NWS Is Evaluating Changes to Its Current Operations
According to agency officials, over the last several years, NWS's
corporate board12 noted that the constrained budget, high labor costs,
difficulty in training and developing its employees, and a lack of
flexibility in how the agency was operating were making it more difficult
for the agency to continue to perform its mission. In August 2005, the
board chartered a working group to evaluate the roles, responsibilities,
and functions of weather offices nationwide and to make a proposal for a
new concept of operations. The group was given a set of guiding
principles, including that the proposed concept should (1) be cost
effective, (2) ensure that there would be no degradation of service, (3)
ensure that weather services nationwide were equitable, and (4) not reduce
the number of forecast offices nationwide. In addition, the working group
was instructed not to address grade structure, staffing levels, office
sizes, or overall organizational chart structure.
The group gathered input from various agency stakeholders and other
partners within NOAA and considered multiple alternatives. They dismissed
all but one of the alternative concepts because they were not consistent
with the guiding principles. In its December 2005 proposal, the working
group proposed a "clustered peer" office plan designed to redistribute
some functions among various offices, particularly when there is a
high-intensity weather event. An agency official explained that each
weather forecast office currently has a fixed geographic area for which it
provides forecasts. If a severe weather event occurs, forecast offices ask
their staff to work overtime so that there are enough personnel available
to do both the normal forecasting work and the watches and warnings
required by the severe event. If a local office becomes unable to provide
forecast and warning functions, an adjacent office will temporarily assume
those duties by calling in extra personnel to handle the workload of both
offices.
Alternatively, under a clustered peer office structure, several offices
with the same type of weather and warning responsibilities, climate, and
customers would be grouped in a cluster. Offices within a cluster would
share the workload associated with routine services, such as 7-day
forecasts. During a high-impact weather event-such as a severe storm,
flood, or wildfire-the offices would redistribute the workload to allow
the impacted office to focus solely on the event, while the other offices
in the cluster would pick up the impacted office's routine services. In
this way, peer offices could help supplement staffing needs and the
workload across multiple offices could be more efficiently balanced.
12NWS's Corporate Board is chaired by the Director of the National Weather
Service, and made up of senior officials responsible for different aspects
of the agency's mission, including the Chief Information Officer and the
Directors of the Office of Climate, Water, and Weather Services; the
National Centers for Environmental Prediction; and the NWS Regions. It
meets at least twice annually to discuss the NWS budget and other
strategic issues. It also holds special meetings, as needed, to focus on
NWS issues such as postevent assessments of major weather services, such
as an assessment of weather services during Hurricane Charley in 2004.
After receiving this proposal, the NWS corporate board chartered another
team to develop a prototype of the clustered peer idea to evaluate the
benefits of this approach. The team plans to recommend the scope of the
prototype and select several weather offices for the prototype
demonstration by the end of September 2006. It also plans to conduct the
prototype demonstration in fiscal years 2007 and 2008. Initial prototype
results are due in fiscal year 2009.
Impacts of New Concept of Operations Have Yet to Be Determined
Many details about the impact of the changes on NWS forecast services,
staffing, and budget have yet to be determined. Sound decision making on
moving forward with a new concept of operations will require data on the
relative costs, benefits, and impacts of such a change, but at this time
the implications of NWS's revised concept of operations on staffing,
budget, and forecasting services are unknown.
The charter for the team developing the prototype for the new concept of
operations calls for it to identify metrics for evaluating the prototype
and to define mechanisms for obtaining customer feedback. However, the
team has not yet established a plan or timeline for developing these
metrics or mechanisms. Further, it is not yet evident that these metrics
will include the relative costs, benefits, or impacts of this change or
which customers will be offered the opportunity to provide feedback. This
is not consistent with the last time NWS undertook a major change to its
concept of operations-during its modernization in the mid-1990s. During
that effort, the agency developed a detailed process for identifying
impacts and ensuring that there would be no degradation of service (see
app. III for a summary of this prior process).
Until it establishes plans, timelines, and metrics for evaluating its
prototype of a revised concept of operations, NWS is not able to ensure
that it is on track to gather the information it needs to fully evaluate
the merits of the revised concept of operations and to make sound and
informed decisions on a new office configuration.
Conclusions
NWS is appropriately positioning itself to improve its forecasting
services by upgrading its systems and technologies and by developing
training to enhance the performance of its professional staff. Over the
next few years, NWS expects to improve all of its 14 performance
measures-ranging from seasonal temperature forecasts, to severe weather
warnings, to specialized aviation and marine weather warnings. However, it
is not clear that NWS is consistently choosing the best training courses
to improve its performance because the training selection process does not
rigorously review the training justifications.
Recognizing that high labor costs, difficulty in training and developing
its employees, and a constrained budget environment make it difficult to
fulfill its mission, NWS is evaluating changes to its office structure and
operations in order to achieve greater productivity and efficiency. It
plans to develop a prototype of a new concept of operations that entails
sharing responsibilities among a cluster of offices. Because it is early
in the prototype process, the implications of these plans on staffing,
budget, and forecasting services are unknown at this time. However, NWS
does not yet have detailed plans, timelines, or measures for assessing the
prototype. As a result, NWS risks not gathering the information it needs
to make an informed decision in moving forward with a new office
operational structure.
Recommendations for Executive Action
To improve NWS's ability to achieve planned service improvements, we
recommend that the Secretary of Commerce direct the Assistant
Administrator for Weather Services to take the following three actions:
o require training officials to validate the accuracy of training
justifications;
o establish key activities, timelines, and measures for
evaluating the "clustered peer" office structure prototype before
beginning the prototype; and
o ensure that plans for evaluating the prototype address the
impact of any changes on budget, staffing, and services.
We received written comments on a draft of this report from the
Department of Commerce (see app. IV). In the department's
response, the Deputy Secretary of Commerce agreed with our
recommendations and identified plans for implementing them.
Specifically, the department noted that it plans to revise its
training process to ensure limited training resources continue to
target improvements in NWS performance. The department also noted
that the concept of operations working team is developing a plan
for the prototype and stated that this plan will include the items
we recommended.
The department also provided technical corrections, which we have
incorporated as appropriate.
We are sending copies of this report to the Secretary of Commerce,
the Director of the Office of Management and Budget, and other
interested congressional committees. Copies will be made available
to others on request. In addition, this report will be available
at no charge on our Web site at www.gao.gov .
If you have any questions about this report, please contact me at
(202) 512-9286 or by e-mail at [email protected] . Contact points
for our Offices of Congressional Relations and Public Affairs may
be found on the last page of this report. GAO staff who made major
contributions to this report are listed in appendix V.
David A. Powner Director, Information Technology Management Issues
Our objectives were (1) to evaluate the National Weather Service's
(NWS) efforts to achieve improvements in the delivery of its
services through upgrades to its systems, models, and
computational abilities; (2) to assess the agency's plans to
achieve improvements in the delivery of its services through the
training and professional development of its employees; and (3) to
evaluate the agency's plans for revising its nationwide office
configuration and the implications of these plans on local
forecasting services, staffing, and budgets.
To evaluate NWS's efforts to achieve service improvements through
system and technology upgrades, we reviewed the agency's system
development plans and discussed system-specific plans with NWS
program officials. We assessed system-specific documentation
justifying system upgrades to evaluate whether these upgrades were
linked to anticipated improvements in performance goals. We also
evaluated NWS performance goals and identified the extent to which
anticipated service improvements were tied to system and
technology upgrades. We interviewed National Oceanic and
Atmospheric Administration (NOAA) and NWS officials to obtain
clarification on agency plans and goals.
To assess NWS's plans for achieving service improvements through
the training and professional development of its employees, we
reviewed NWS policies and plans for training and professional
development. We reviewed the agency's service performance goals
and assessed the link between those goals and planned and expected
training and professional development activities. We also
interviewed NWS officials responsible for training and
professional development activities.
To evaluate the status and potential impact of any plans to revise
the national office configuration, we assessed studies of options
for changing the NWS concept of operations. We also reviewed the
charter for the prototype and interviewed key NWS officials to
determine the possible effect of these plans on local forecasting
services, staffing, and budgets and to identify plans for
determining the implications of changing to a new concept of
operations.
We performed our work at NWS headquarters in the Washington, D.C.,
metropolitan area, and at geographically diverse NOAA and NWS
weather forecast offices in Denver and in Tampa, and at the NWS
National Hurricane Center in Miami. We performed our work from
October 2005 to June 2006 in accordance with generally accepted
government auditing standards.
Source: GAO analysis of NOAA and NWS reports.
aMetric measured between October 2005 and January 2006.
bMetric measured between October 2005 and February 2006.
cMetric measured between October 2005 and December 2005.
dMetric measured between October 2005 and March 2006.
eInstrument Flight Rules take effect when ceilings and
visibilities are less than 1,000 feet and/or 3 miles,
respectively, and ceilings and visibilities are greater than, or
equal to, 500 feet and/or 1 mile, respectively.
fData for this metric are not available until the beginning of the
next calendar year because of the timing of the hurricane season.
In the 1980s, NWS began a nationwide modernization program to
upgrade weather observing systems such as satellites and radars,
to design and develop advanced computer workstations for
forecasters, and to reorganize its field office structure. The
goals of the modernization were to achieve more uniform weather
services across the nation, improve forecasting, provide more
reliable detection and prediction of severe weather and flooding,
achieve higher productivity, and permit more cost-effective
operations through staff and office reductions.
NWS's plans for revising its office structure were governed by the
Weather Service Modernization Act,1 which required that, prior to
closing a field office, the Secretary of Commerce certify that
there was no degradation of service. NWS developed a plan for
complying with the law. To identify community concerns regarding
modernization changes and to study the potential for degradation
of service, the Department of Commerce published a notice in the
Federal Register requesting comments on service areas where it was
believed that services could be degraded by planned modernization
changes. The department also contracted for an independent
assessment by the National Research Council on whether weather
services would be degraded by the proposed changes. As part of
this assessment, the contractor developed criteria to identify
whether service would be degraded in certain areas of concern. The
department then applied these criteria to areas of concern to
determine whether services would be degraded or not. Before
closing any office, the Secretary of Commerce certified that
services would not be degraded.
David A. Powner, (202) 512-9286 or [email protected] .
In addition to the contact named above, William Carrigg, Barbara
Collier, Neil Doherty, Kathleen S. Lovett, Colleen Phillips, Karen
Talley, and Jessica Waselkow made key contributions to this
report.
The Government Accountability Office, the audit, evaluation and
investigative arm of Congress, exists to support Congress in
meeting its constitutional responsibilities and to help improve
the performance and accountability of the federal government for
the American people. GAO examines the use of public funds;
evaluates federal programs and policies; and provides analyses,
recommendations, and other assistance to help Congress make
informed oversight, policy, and funding decisions. GAO's
commitment to good government is reflected in its core values of
accountability, integrity, and reliability.
The fastest and easiest way to obtain copies of GAO documents at
no cost is through GAO's Web site ( www.gao.gov ). Each weekday,
GAO posts newly released reports, testimony, and correspondence on
its Web site. To have GAO e-mail you a list of newly posted
products every afternoon, go to www.gao.gov and select "Subscribe
to Updates."
The first copy of each printed report is free. Additional copies
are $2 each. A check or money order should be made out to the
Superintendent of Documents. GAO also accepts VISA and Mastercard.
Orders for 100 or more copies mailed to a single address are
discounted 25 percent. Orders should be sent to:
U.S. Government Accountability Office 441 G Street NW, Room LM
Washington, D.C. 20548
To order by Phone: Voice: (202) 512-6000 TDD: (202) 512-2537 Fax:
(202) 512-6061
Contact:
Web site: www.gao.gov/fraudnet/fraudnet.htm E-mail:
[email protected] Automated answering system: (800) 424-5454 or
(202) 512-7470
Gloria Jarmon, Managing Director, [email protected] (202) 512-4400
U.S. Government Accountability Office, 441 G Street NW, Room 7125
Washington, D.C. 20548
Paul Anderson, Managing Director, [email protected] (202)
512-4800 U.S. Government Accountability Office, 441 G Street NW,
Room 7149 Washington, D.C. 20548
Agency Comments
Appendix I: Objectives, Scope, and Methodology Appendix I: Objectives,
Scope, and Methodology
Appendix II: NWS Performance Goals for Fiscal Years 2005 to 2011 Appendix
II: NWS Performance Goals for Fiscal Years 2005 to 2011
FY05 FY06 FY07 FY08 FY09 FY10 FY11
Actual
Performance Final to
measure Description Goal actual Goal date Goal Goal Goal Goal Goal
Tornado warning The 13 13 13 13a 14 15 15 15 15
lead time difference
(minutes) between the
time a
warning is
issued and
the time of
the first
report of a
tornado in a
given county
Tornado warning The 73 75 76 82a 76 76 76 76 76
accuracy (percent) percentage of
time a
tornado
actually
occurred in
an area
covered by a
tornado
warning
Tornado warning The 73 77 75 76a 74 74 74 74 74
false-alarm rate percentage of
(percent) time a
tornado
warning was
issued but no
tornado event
was reported
Flash flood The 48 54 48 63a 49 49 49 49 49
warning lead time difference
(minutes) between the
time a
warning is
issued and
the time of
the first
report of a
flash flood
in a given
county
Flash flood The 89 88 89 93a 90 90 90 90 90
warning accuracy percentage of
(percent) time a flash
flood
actually
occurred in
an area
covered by a
flash flood
warning
Marine wind speed A measure of 57 57 58 56b 58 58 59 59 59
forecast accuracy the accuracy
(percent) of wind speed
forecasts
Marine wave height A measure of 67 67 68 71b 68 68 69 69 69
forecasts accuracy the accuracy
(percent) of wave
forecasts
Aviation forecast The 46 46 47 45b 48 51 52 53 59
Instrument Flight percentage of
Rule time
ceiling/visibility Instrument
accuracy (percent) Flight Rule
conditionse
are predicted
and occur
Aviation forecast The 68 63 65 61b 64 58 57 56 50
Instrument Flight percentage of
Rule time
ceiling/visibility Instrument
false-alarm rate Flight Rule
(percent) conditionse
are predicted
but do not
occur
Winter storm The average 15 17 15 16c 15 15 16 17 17
warning lead time time from the
(hours) issuance of a
warning to
the time of
the first
report of a
winter storm
in a given
county
Winter storm The 90 91 90 91c 90 90 91 92 92
warning accuracy percentage of
(percent) verified
winter storm
events that
were covered
by winter
storm
warnings
Precipitation A score based 27 29 28 39d 29 29 29 30 30
forecast day 1 on the
threat (score) agency's
accuracy in
forecasting
precipitation
U.S. seasonal A score based 18 19 18 24d 19 19 19 20 20
temperature on the
forecast skill agency's
(score) accuracy in
forecasting
temperature
Hurricane track A measure of 128 101 111 N/Af 110 109 108 107 106
forecasts at 48 the
hours (nautical difference
miles) between the
projected
locations of
the center of
storms and
the actual
locations in
nautical
miles for the
Atlantic
Basin
Appendix III: NWS Previously Used A Stringent Process to Ensure Service
Was Not Degraded Appendix III: NWS Previously Used A Stringent Process to
Ensure Service Was Not Degraded
1Pub. L. 102-567 S: 706(b), 106 Stat. 4303, 4306 (1992).
Appendix IV: Comments from the Department of Commerce Appendix IV:
Comments from the Department of Commerce
Appendix V: A Appendix V: GAO Contact and Staff Acknowledgments
GAO Contact
Staff Acknowledgments
(310817)
GAO's Mission
Obtaining Copies of GAO Reports and Testimony
Order by Mail or Phone
To Report Fraud, Waste, and Abuse in Federal Programs
Congressional Relations
Public Affairs
www.gao.gov/cgi-bin/getrpt? GAO-06-792 .
To view the full product, including the scope
and methodology, click on the link above.
For more information, contact David Powner at (202) 512-9286 or
[email protected].
Highlights of GAO-06-792 , a report to the Subcommittee on Environment,
Technology, and Standards, Committee on Science, House of Representatives
July 2006
WEATHER FORECASTING
National Weather Service Is Planning to Improve Service and Gain
Efficiency, but Impacts of Potential Changes Are Not Yet Known
To provide accurate and timely weather forecasts, the National Weather
Service (NWS) uses systems, technologies, and manual processes to collect,
process, and disseminate weather data to its nationwide network of field
offices and centers. After completing a major modernization program in the
1990s, NWS is seeking to upgrade its systems with the goal of improving
its forecasting abilities, and it is considering changing how its
nationwide office structure operates in order to enhance efficiency. GAO
was asked to (1) evaluate NWS's efforts to achieve improvements in the
delivery of its services through system and technology upgrades, (2)
assess agency plans to achieve service improvements through training its
employees, and (3) evaluate agency plans to revise its nationwide office
configuration and the implications of these plans on local forecasting
services, staffing, and budgets.
What GAO Recommends
GAO is making recommendations to the Secretary of Commerce to direct NWS
to strengthen its training selection process, and to establish key
activities, timelines, and measures for evaluating the prototype of a new
concept of operations before beginning the prototype. In written comments,
the Department of Commerce agreed with the recommendations and identified
plans for implementing them.
NWSis positioning itself to provide better service through over
$315millionin planned upgrades to its systems and technologies. In annual
plans, the agency links expected improvements in its service performance
measures with the technologies and systems expected to improve them. For
example, NWS expects to reduce the average error in its forecasts of
hurricane paths by approximately 20 nautical miles between 2005 and 2011
through a combination of upgrades to observation systems, better hurricane
forecast models, enhancements to the computer infrastructure, and research
that will be transferred to forecast operations. Also, NWS expects to
increase tornado warning lead times from 13 to 15 minutes by the end of
fiscal year 2008 after the agency completes an upgrade to its radar system
and realizes benefits from software improvements to its forecaster
workstations.
NWS also provides training courses for its employees to help improve its
forecasting services, but the agency's process for selecting training
lacks sufficient oversight. Program officials propose and justify training
needs on the basis of up to eight different criteria-including whether a
course is expected to improve NWS forecasting performance measures,
support customer outreach, or increase scientific awareness. Many of these
course justifications appropriately demonstrate support for improved
forecasting performance. For example, training on how to more effectively
use forecaster workstations is expected to help improve tornado and
hurricane warnings. However, in justifying training courses, program
officials routinely link courses to NWS forecasting performance measures.
For example, in 2006, almost all training needs were linked to
expectations for improved performance-including training on
cardiopulmonary resuscitation, spill prevention, and systems security. The
training selection process did not validate or question how these courses
could help improve weather forecasts. Overuse of this justification
undermines the distinctions among different training courses and the
credibility of the course selection process. Additionally, because the
training selection process does not clearly distinguish among courses, it
is difficult to determine whether sufficient funds are dedicated to the
courses that are expected to improve performance.
To improve its efficiency, NWS plans to develop a prototype of a new
concept of operations, an effort that could affect its national office
configuration, including the location and functions of its offices
nationwide. However, many details about the impact of any proposed changes
on NWS forecast services, staffing, and budget have yet to be determined.
Further, the agency has not yet determined key activities, timelines, or
measures for evaluating the prototype of the new office operational
structure. As a result, it is not evident that NWS will collect the
information it needs on the impact and benefits of any office
restructuring in order to make sound and cost-effective decisions.
*** End of document. ***