Benchmarking Activities and Fees

GADS Services of the North American Electric Reliability Corporation (NERC) is pleased to provide electric generating unit benchmarking services to the electric industry.  

Electric unit benchmarking is provided by a team of two experts: G. Michael (Mike) Curley of the NERC GADS Services staff and Robert R. (Bob) Richwine, Reliability Management Consultant. Biographies and work experiences of both Team members are shown at the end of this document.  

The following descriptions present what services we can provide power generators and the cost for each service.

Introduction

Our benchmarking team begins the process by identifying peer groups to single units or groups of units operated by electric power owners. Whenever we benchmark a generating plant’s performance, it is vital that we start by selecting a peer group that have as close a similarity in design and operating characteristics as possible. Certainly, we would never compare a fossil steam unit against a group that included nuclear, hydro or combined cycle units.  However, many benchmarking programs have assumed that for fossil steam units, fuel type and size ranges are the proper select criteria. We have found from our extensive benchmarking studies that fuel types and especially the arbitrary size ranges (100-199MW, 200-299MW, etc.) are relatively much less statistically significant than other design and operational characteristics such as criticality, duty cycle, vintage,  pressurized/balanced draft, etc. Because each individual unit is unique, our process ensures that the optimal peer group is selected; balancing the need for similarity in design and operations with the need for a large enough sample size for statistical validity. Without this objective analysis to find the optimal peer select criteria any conclusions drawn from the comparisons could very well be invalid and misleading.  

By teaming with NERC, your company will gain the assurance that the results are objective and repeatable, with NERC having performed this service for over 12 years for US companies as well as for many international utilities. For many companies this has helped to ensure that the results are accepted throughout the organization and by their regulatory bodies.  

It has been asked by others “but does it really make a difference?” To answer this we can look at a previous study that was done for the New England Power Pool in 1990 (the data is obviously out of date but should give an indication). As the study was undertaken we found that the most statistically significant factor was “criticality”. Then within the supercritical group the next most important factor was “vintage”.  

Clearly from this result it would be highly inappropriate to include recent vintage supercritical units in any peer population if the candidate unit was an early vintage supercritical unit, since it would be compared against units that had clearly benefited from the “learning curve” of the early vintage units.  

In another study completed much recently, a fossil steam plant was analyzed to find its statistically appropriate peer groups. Here we can see that if we were to set a goal as the best quartile performers in our peer group, we would be setting unrealistically high expectations using the old criteria, compared to what we might set using the more appropriate peer group with new criteria.  

Clearly, these plants must have design and/or operational characteristics that create a more difficult challenge for their plant management to achieve the highest levels of reliability; a difficulty that might not be recognized through a benchmarking process that did not begin with a rigorous peer selection criteria analysis. Furthermore, the goals we might set using an inappropriate peer group may not be cost effective and we may end up spending more money than is justified to achieve these goals. Other plants studied have had just the opposite result. In those we may be setting our goals too low since the new peer group performs better than the one using the traditional criteria.  

These examples are not isolated results. Rather we have found similar instances in virtually every analysis we have conducted including benchmarking studies of Nuclear, Hydro and other technologies. We firmly believe that if a benchmarking project is undertaken at all, it should start with an objective analysis to determine optimal peer unit characteristics.

For a more complete description of the benchmarking procedure, please refer to the NERC Generating Availability Trend Evaluation (GATE) Working Group paper Predicting Generating Unit Reliability and World Energy Council (WEC) Case Study of the Month (CASOM).  (See CASOM studies August 2002 and September 2003)

Fee Structure for Performance Benchmarking:  

1. Setup Fee:  For all benchmarking work, there is a $5,000 setup fee that covers the cost of preparing computer programs, phone consultation, and work forms used in the benchmarking process. It covers consultation for reviewing the fleet of units to be benchmarked and to divide the units into groups, if asked by the client. Although it is normally best to determine the optimal peer select criteria for each individual unit, in some cases a group of units operated by one company may be designed and operated in a similar manner so that an “average” for a group can be used in order to reduce costs.
   

2. Peer Criteria Selection Procedure:  Our first step in benchmarking is find the optimal balance between the need for as close a match as possible in the design and operating characteristics of the client’s unit (or group of similar units) and the need for there to be enough units in the peer group for statistical validity. For each unit (or group of similar units) the fee is $1200.
   

3. Performance Graphs and Tables:  Once the optimal select criteria is found, a set of tables and graphs are created for the performances indices selected. We normally use EFOR, SOF and EAF as the key indicators. However, we can provide additional graphs for any three indices wanted. Cost for the three-set is $600. If the client wants additional indices graphed, the cost is $200 per index. An example of these graphs is shown below.
   
   
   
   

4. Variability Graphs and Tables: We also offer a set of graphs and tables that show the make up of the mean values. The variability graphs (shown below) provide a year-by-year and mean value for each unit in the peer group. This allows the client to see the variation of a single index over the period of investigation. In the example below, the data was averaged over a five-year period. Therefore there are 5 annual and one 5-year average point.

The cost of each graph and its accompanying table is $200.

In this cumulative frequency graph for EAF the five-year average is rank ordered but the individual years for each unit are also shown in order to gain a perspective of EAF variability.  

Fee Structure for Cost Benchmarking

Cost data normally comes as a cost per unit from various venders. The “per unit” data is a process of dividing the cost at a plant level by the MW capacity of the plant. This is an acceptable way of looking at costs if the units at the site are identical in all aspects of operation, size, fuels and others. That is not always the case. If a plant has a newer fossil unit and two old fossil units plus a gas turbine, the true cost per MW is not the same and should be treated at the same.

If cost benchmarking is required, it is best to more fully discuss the way the cost data would be used (on a plant or unit basis) before the cost for the work will be provided. This is a safe way for both GADS Services and the customer to know exactly what will be provided and how the cost data will be evaluated before work starts.

Setup Fee: If cost data is wanted, there is a $5,000 setup fee that covers the cost of preparing computer programs, purchase of cost information, phone consultation, and work forms used in the benchmarking process. It covers consultation for reviewing the fleet of units to be benchmarked and to divide the units into groups, if asked by the client.

Once the cost data is completed, graphs and tables can be produced. An example of the resulting graph is shown below:

Some of the standard cost graphs and tables available include:   

• Plant Fuel Cost Distribution ($/MWh)

• Plant Non-fuel O&M Cost Distribution ($/MWh)

• Plant Non-fuel O&M Cost Distribution ($/kW)

• Plant Employee per MW Distribution

• Plant MWh per Employee Distribution

The cost for each set of five graphs for each peer group is $1000.   

If additional graphs or tables are wanted, you can discuss the cost with a member of the Benchmarking Team.

Travel and Expenses

It is not required but recommended that the client and a member(s) of the NERC-GADS Benchmarking team meet to discuss the scope and approach before the work begins. It is also recommended that a meeting be scheduled to present the final results and discuss the findings. Traveling members of the NERC team would be compensated for their travel expenses plus $1200 per day for their time.  

Final Report

NERC-GADS will provide a formal report on all results that will include:

1. Introduction
2. Overview of the process
3. Specific Results for each unit/group analyzed including a list of units in each peer group
4. Peer select Criterion for each of the client’s units or group of similar units
5. Copy of Power Point slides discussing the NERC-GADS benchmarking process
6. Conference reports and research documents on NERC-GADS benchmarking work
7. CD-ROM and paper copy of the complete final report

Biography of the Benchmarking Team

G. Michael (Mike) Curley  
Manager of GADS Services at the North American Electric Reliability Corporation (NERC) located in Princeton, New Jersey.  

Shortly after graduating from college, Mike started working for the electric power industry, specializing in analyzing equipment failures and suggesting possible solutions to increase equipment and power unit productivity. He has worked for NERC since 1983 as a consultant in processing, analyzing and preparing both topical and specialized studies from the data collected by Generating Availability Data System (GADS). He has been involved in a number of NERC, Edison Electric Institute (EEI), American Society of Mechanical Engineers (ASME) and Electric Power Research Institute (EPRI) committees, and other industry and professional work groups.

Mike is a member of the World Energy Council’s (WEC) Performance of Generating Plant (PGP) Committee. He is the chairman of two WEC-PGP working groups. Mike is a Fellow grade in the ASME. He is the author and co-author of more than a dozen technical papers.  

Robert (Bob) R. Richwine
Reliability Management Consultant.  As an independent consultant since 2002, Bob has consulted for numerous companies, including AES Energy (plant performance improvement), the International Atomic Energy Agency (inventory management), Tennessee Valley Authority (benchmarking), Mirant (reliability models and long term non-recurring cost estimating) and The National Grid Company of Ireland (benchmarking).

From 1976 to 1993 Bob was consulting reliability engineer at Southern Company Services in Birmingham, Alabama.  After joining the firm Bob organized the company’s Reliability Engineering group and progressed in increasing levels of responsibility until his departure for Southern Energy, Inc. in 1993.  During these 17 years Bob was involved in the development and implementation of numerous programs and projects that have seen the southern electric system’s coal-fired plant’s availability increase from 68% in 1976 to over 92% by the late 1980’s.  Among these projects were the development and implementation of the company’s Availability Improvement Program; Reliability, Availability, Maintainability (RAM) analysis; availability/reliability projections for existing/proposed power plants. Bob received The Southern Company Chairman’s Excellence Award in 1995 and Southern Company Services President’s Award in 1992.

In 1993 Bob moved to Southern Energy, Inc, Southern Company’s unregulated arm, to head their Consulting Business unit. During his 9 years there he also provided Reliability Engineering support to SEI’s (later Mirant) project development, trading and marketing and operations departments. In 2002 Bob left to pursue independent consulting.  

Bob is a member of the World Energy Council’s (WEC) Performance of Generating Plant (PGP) Committee and is the chairman of its Working Group on Workshops and Communications. In that capacity he publishes a monthly case study on various aspects of Reliability Engineering on the WEC website.  Bob is also the author or co-author of over 30 technical publications on the subjects of Availability Improvement, Reliability Engineering, and cost/performance relationships for power plants.