[Federal Register Volume 77, Number 234 (Wednesday, December 5, 2012)]
[Notices]
[Pages 72435-72438]
From the Federal Register Online via the Government Printing Office [www.gpo.gov]
[FR Doc No: 2012-29362]
-----------------------------------------------------------------------
DEPARTMENT OF TRANSPORTATION
Pipeline and Hazardous Materials Safety Administration
[Docket No. PHMSA-2012-0279]
Pipeline Safety: Using Meaningful Metrics in Conducting Integrity
Management Program Evaluations
AGENCY: Pipeline and Hazardous Materials Safety Administration (PHMSA),
DOT.
ACTION: Notice; Issuance of Advisory Bulletin.
-----------------------------------------------------------------------
SUMMARY: PHMSA is issuing an Advisory Bulletin to remind operators of
gas transmission and hazardous liquid pipeline facilities of their
responsibilities, under Federal integrity management regulations, to
perform evaluations of their integrity management programs using
meaningful performance metrics.
FOR FURTHER INFORMATION CONTACT: Alan Mayberry by phone at 202-366-5124
or by email at alan.mayberry@dot.gov. All materials in this docket may
be accessed electronically at http://www.regulations.gov. General
information about the PHMSA Office of Pipeline Safety (OPS) can be
obtained by accessing OPS's Internet home page at http://www.phmsa.dot.gov/pipeline.
SUPPLEMENTARY INFORMATION:
I. Background
PHMSA's integrity management regulations require operators to
establish processes to evaluate the effectiveness of their integrity
management programs. Program evaluation is one of the key required
program elements as established in the integrity management rules. For
hazardous liquid pipelines, Sec. Sec. 195.452(f)(7) and 195.452(k)
require methods to measure program effectiveness:
Sec. 195.452(f) What are the elements of an integrity management
program? An integrity management program begins with the initial
framework. An operator must continually change the program to reflect
operating experience, conclusions drawn from results of the integrity
assessments, other maintenance and surveillance data, and evaluation of
consequences of a failure on the high consequence area. An operator
must include, at minimum, each of the following elements in its written
integrity management program:
* * * * *
(7) Methods to measure the program's effectiveness (see paragraph
(k) of this section);
Sec. 195.452(k) What methods to measure program effectiveness must
be used? An operator's program must include methods to measure whether
the program is effective in assessing and evaluating the integrity of
each pipeline segment and in protecting the high consequence areas.
(See Appendix C of this part for guidance on methods that can be used
to evaluate a program's effectiveness.)
Appendix C provides more specific guidance on establishing
performance measures, including the need to select measures based on
the understanding and analysis of integrity threats to each pipeline
segment. Appendix C also describes three general types of metrics that
an integrity management program should have:
Activity Measures that monitor the surveillance and
preventive activities that are in place to control risk. These measures
indicate how well an operator is implementing the elements of its
integrity management program.
[[Page 72436]]
Deterioration Measures that monitor operational and
maintenance trends to indicate if the program is successful or
weakening, or if the desired outcome is being achieved or not, despite
the risk control activities in place.
Failure Measures that reflect whether the program is
effective in achieving the objective of improving integrity. These are
typically lagging indicators that measure the number of releases, the
volume spilled, percent recovered, etc.
Section 13 ``Program Evaluation'' of API Standard 1160, Managing
Integrity for Hazardous Liquid Pipelines also provides additional
guidance on the program evaluation process in which these measures are
used to improve performance.
For gas transmission pipelines, Sec. Sec. 192.911(i) and 192.945
define the requirements for establishing performance metrics and
evaluating integrity management program performance.
Sec. 192.911 What are the elements of an integrity management
program?
An operator's initial integrity management program begins with a
framework (see Sec. 192.907) and evolves into a more detailed and
comprehensive integrity management program as information is gained and
incorporated into the program. An operator must make continual
improvements to its program. The initial program framework and
subsequent program must, at minimum, contain the following elements.
(When indicated, refer to ASME/ANSI B31.8S incorporated by reference,
see Sec. 192.7) for more detailed information on the listed element.)
* * * * *
(i) A performance plan as outlined in ASME/ANSI B31.8S, section 9
that includes performance measures meeting the requirements of Sec.
192.945.
Sec. 192.945 What methods must an operator use to measure program
effectiveness?
(a) General. An operator must include in its integrity management
program methods to measure whether the program is effective in
assessing and evaluating the integrity of each covered pipeline segment
and in protecting the high consequence areas. These measures must
include the four overall performance measures specified in ASME/ANSI
B31.8S (incorporated by reference, see Sec. 192.7 of this part),
section 9.4, and the specific measures for each identified threat
specified in ASME/ANSI B31.8S, Appendix A. An operator must submit the
four overall performance measures as part of the annual report required
by Sec. 191.17 of this subchapter.
(b) External Corrosion Direct Assessment (ECDA). In addition to the
general requirements for performance measures in paragraph (a) of this
section, an operator using direct assessment to assess an external
corrosion threat must define and monitor measures to determine the
effectiveness of the ECDA process. These measures must meet the
requirements of Sec. 192.925.
The gas transmission requirements invoke ASME B31.8S-2004, Managing
System Integrity of Gas Pipelines. Section 9 of this standard provides
guidance on the selection of performance measures. It describes three
categories of measures that are directly analogous to those noted above
in Appendix C of Part 195. These are:
Process or Activity Measures used to evaluate preventive
and mitigation activities. These determine how well an operator is
implementing the various elements of its integrity management program.
Operational Measures, which include operational and
maintenance trends that measure how well the system is responding to
the integrity management program.
Direct Integrity Measures, which include leaks, ruptures,
injuries, and fatalities.
Furthermore, the hazardous liquid and gas transmission integrity
management rules also require that operators retain adequate records to
support integrity management program decisions and activities. These
include the information that supports the selection of performance
metrics, the performance metric data and trends, and the decisions that
are based in whole or in part on these metrics. Specifically, the
hazardous liquid integrity management program requirements are:
Sec. 195.452(l) What records must be kept? (1) An operator must
maintain for review during an inspection:
* * * * *
(ii) Documents to support the decisions and analyses, including any
modifications, justifications, variances, deviations and determinations
made, and actions taken, to implement and evaluate each element of the
integrity management program listed in paragraph (f) of this section.
(2) See Appendix C of this part for examples of records an operator
would be required to keep.
Appendix C further states:
Sec. 195.452 Appendix C. VI. Examples of types of records an
operator must maintain.
* * * * *
(22) methods used to measure the program's effectiveness.
The comparable gas transmission integrity management program
requirements are:
Sec. 192.947 What records must be kept?
An operator must maintain, for the useful life of the pipeline,
records that demonstrate compliance with the requirements of this
subpart. At minimum, an operator must maintain the following records
for review during an inspection.
* * * * *
(d) Documents to support any decision, analysis, and process
developed and used to implement and evaluate each element of the
baseline assessment plan and integrity management program. Documents
include those developed and used in support of any identification,
calculation, amendment, modification, justification, deviation and
determination made, and any action taken, to implement and evaluate any
of the program elements;
PHMSA's inspection protocols currently address the need to examine
operator compliance with these requirements.
In its report on the September 9, 2010, gas pipeline accident in
San Bruno, California, the National Transportation Safety Board (NTSB)
identified concerns with Pacific Gas and Electric Company's (PG&E)
self-assessments of its integrity management program. NTSB concluded
that the company's self-assessments were ``superficial and resulted in
no improvements to the integrity management program.'' As a result,
NTSB recommended that PG&E:
Assess every aspect of your integrity management program, paying
particular attention to the areas identified in this investigation, and
implement a revised program that includes, at a minimum,
* * * * *
(4) an improved self-assessment that adequately measures whether
the program is effectively assessing and evaluating the integrity of
each covered pipeline segment. (Recommendation P-11-29)
In this same investigation, NTSB raised some concerns with PHMSA's
oversight of performance-based safety programs such as integrity
management. NTSB concluded that greater focus is needed on how
performance-based safety systems are implemented, executed and
evaluated, and whether problem areas are being detected and corrected.
Critical to this overall process is the selection of meaningful metrics
by operators that allow them to quantify,
[[Page 72437]]
understand, and improve their own performance.
Following its investigation, NTSB issued two related
recommendations for enhancing PHMSA's oversight of operator programs to
assess the effectiveness of PHMSA's programs using performance metrics.
These recommendations are:
Revise your integrity management inspection protocol to:
(1) incorporate a review of meaningful metrics;
(2) require auditors to verify that the operator has a procedure in
place for ensuring the completeness and accuracy of underlying
information;
(3) require auditors to review all integrity management performance
measures reported to the Pipeline and Hazardous Materials Safety
Administration and compare the leak, failure, and incident measures to
the operator's risk model; and
(4) require setting performance goals for pipeline operators at
each audit and follow up on those goals at subsequent audits.
(Recommendation P-11-18)
(1) Develop and implement standards for integrity management and
other performance-based safety programs that require operators of all
types of pipeline systems to regularly assess the effectiveness of
their programs using clear and meaningful metrics and to identify and
then correct deficiencies; and (2) make those metrics available in a
centralized database. (Recommendation P-11-19)
These recommendations reinforce the importance of a rigorous
evaluation of a company's integrity management program in improving
performance. Through this Advisory Bulletin, PHMSA is reminding
operators of the importance of these regulation-required program
elements. Operators should review their current programs for evaluating
integrity management program effectiveness and the performance metrics
used in these programs to be sure they provide a current and accurate
representation of integrity management program performance. Further,
operators should ensure that program improvements and corrective
actions identified by these evaluations are implemented in a timely
manner.
As a result of NTSB's recommendations, PHMSA is initiating efforts
to strengthen its protocols and oversight of these key integrity
management program elements. Beginning immediately, PHMSA's inspections
will emphasize reviewing operator methods for integrity management
program evaluation as required by Sec. 192.945 and Sec. 195.452(k)
for gas transmission and hazardous liquid pipelines, respectively.
PHMSA will evaluate specific metrics operators use to assess program
effectiveness and how those metrics are used in a process of continuous
improvement. PHMSA will also confirm that operators are maintaining
adequate records of their program effectiveness evaluations and their
performance metrics data, as well as the activities and decisions
associated with all required integrity management program elements. Our
inspectors will check to confirm that information and data gaps are
aggressively being addressed and that assumptions are appropriately
based on location-specific data.
II. Advisory Bulletin (ADB-20l2-10)
To: Owners and Operators of Hazardous Liquid and Gas Transmission
Pipeline Systems
Subject: Using Meaningful Metrics in Conducting Integrity
Management Program Evaluations
Advisory: To further enhance PHMSA's safety efforts and as an
initial step in addressing NTSB Recommendations P-11-18 and P-11-19,
PHMSA is issuing this Advisory Bulletin concerning operator integrity
management program evaluation using meaningful metrics.
A critical program element of an operator's integrity management
program is the systematic, rigorous evaluation of the program's
effectiveness using clear and meaningful metrics. When executed
diligently, this self-evaluation process will lead to more robust and
effective integrity management programs and improve overall safety
performance. This process is critical to achieving a mature integrity
management program and a culture of continuous improvement. Program
evaluation is a required integrity management program element as
established in Sec. Sec. 192.911(i) and 195.452(k) for gas
transmission and hazardous liquid pipelines, respectively. In light of
NTSB's findings following the San Bruno gas transmission incident,
PHMSA is reminding operators about the importance of these
requirements.
Operators are advised to critically review their processes and
methods for evaluating integrity management program performance and
take action to strengthen these processes where warranted. An effective
operator performance evaluation process is expected to have the
following characteristics:
A well-defined description of the scope, objectives, and
frequency of program evaluations.
The use of periodic self-assessments, internal or external
audits, management reviews, performance metrics analysis, benchmarking
against other operators, or other self-critical evaluations to assess
program effectiveness.
Clear performance goals and objectives to measure the
effectiveness of key integrity activities.
Clear assignment of responsibility for implementing
required actions.
Review and follow-up of program evaluation results,
findings, and recommendations, etc., by appropriate company managers.
Operators are also advised that a clear and meaningful set of
performance metrics is essential to program effectiveness. An effective
program for measuring integrity management program effectiveness should
have the following characteristics:
A description of the type of performance measures to be
used, along with the data sources, data validation and quality
assurance activities, the frequency of data collection, and any
normalization factors.
A means to update the performance measures (if needed) to
assure they are providing useful information about the effectiveness of
integrity management program activities.
The use of performance metrics data to check and calibrate
the operator's risk analysis tools to assure these best represent the
performance of the operator's specific assets.
The performance metrics that are required to be reported to PHMSA
annually, such as the number of miles of pipeline assessed, number of
anomalies found requiring repair or mitigation, etc., are a small
subset of the overall suite of metrics used by an operator to evaluate
its program. A much larger set of operator-specific metrics to be used
internally is needed to effectively evaluate an integrity management
program performance. Metrics should be developed for each of the
following:
Overall program effectiveness indicated by the number of
releases, number of injuries or fatalities, volume released, etc.
Specific threats that include both leading and lagging
indicators for the important integrity threats on an operator's
systems. These include:
[cir] Activity Measures that monitor the surveillance and
preventive activities that are in place to control risk.
[cir] Deterioration Measures that monitor operational and
maintenance trends to indicate if the program is successful or
weakening despite the risk control activities in place. (Also
[[Page 72438]]
identified as Operational Measures in ASME B31.8S.)
[cir] Failure Measures that reflect whether the program is
effective in achieving the objective of improving integrity. (Also
identified as Direct Integrity Measures in ASME B31.8S)
Metrics that measure and provide insights into how well an
operator's processes associated with the various integrity management
program elements are performing. Examples of such processes would
include integrity assessment, risk analysis, the identification of
preventive and mitigative measures, etc.
While operator-level rollups of metrics are useful for small
operators, a robust program for large operators should also include
metrics at a more granular level. The metrics should enable operators
to drill down to understand the performance of specific systems or
segments within systems. This is particularly important for the threat-
specific metrics mentioned previously.
Finally, as required by Sec. Sec. 195.452(l) and 192.947,
operators must keep records supporting the decisions, analyses, and
processes developed and used in their evaluation of integrity
management program effectiveness. These records should include those
justifying the selection of performance metrics, the performance metric
data and trends, and how these metrics are used to improve the
integrity management program. Operators should also be diligently
working to eliminate information and data gaps throughout their entire
integrity management program.
Issued in Washington, DC, on November 29, 2012.
Jeffrey D. Wiese,
Associate Administrator for Pipeline Safety.
[FR Doc. 2012-29362 Filed 12-4-12; 8:45 am]
BILLING CODE 4910-60-P