Research Papers: Gas Turbines: Controls, Diagnostics, and Instrumentation

Application of Cost Matrices and Cost Curves to Enhance Diagnostic Health Management Metrics for Gas Turbine Engines

[+] Author and Article Information
Craig R. Davison

Gas Turbine Laboratory, Institute for Aerospace Research, National Research Council Canada, Ottawa, ON K1A 0R6, Canada

Chris Drummond

Institute for Information Technology, National Research Council Canada, Ottawa, ON K1A 0R6, Canada

J. Eng. Gas Turbines Power 132(4), 041604 (Jan 27, 2010) (8 pages) doi:10.1115/1.3159384 History: Received March 25, 2009; Revised April 06, 2009; Published January 27, 2010; Online January 27, 2010

Statistically based metrics, incorporating operating costs, for gas turbine engine diagnostic systems are required to evaluate competing products fairly and to establish a convincing business case. Diagnostic algorithm validation often includes engine testing with implanted faults. The implantation rate is rarely, if ever, representative of the true fault occurrence rate and the sample size is very small. Costs related to diagnostic outcomes have a significant effect on the utility of a given algorithm and need to be incorporated into the assessment. Techniques for assessing diagnostics are drawn from the literature and modified for application to gas turbine applications. The techniques are modified with computational experiments and the application demonstrated through examples. New techniques are compared to the traditional methods and the advantages presented. A technique is presented to convert a confusion matrix with a non-representative fault distribution to one representative of the expected distribution. The small sample size associated with fault implantation studies requires a confidence interval on the results to provide valid comparisons and a method for calculating confidence intervals, including on zero entries, is presented. Receiver operating characteristic (ROC) curves evaluate diagnostic system performance across a range of threshold settings. This allows an algorithm’s ability to be assessed over a range of possible usage. Cost curves are analogous to ROC curves but offer several advantages. The techniques for applying cost curves to diagnostic algorithms are presented and their advantages over ROC curves are outlined. This paper provides techniques for more informed comparison of diagnostic algorithms, possibly preventing incorrect assessment due to small sample sizes.

Copyright © 2010 by Her Majesty the Queen in Right of Canada
Your Session has timed out. Please sign back in to continue.



Grahic Jump Location
Figure 1

Accuracy of 95% confidence intervals with sample size of 20

Grahic Jump Location
Figure 2

Accuracy of 95% confidence intervals with sample size of 200

Grahic Jump Location
Figure 3

Confidence intervals on MSC for Laplace corrected (λ=0.25) and original confusion matrices produced with increasing number of implanted faults

Grahic Jump Location
Figure 4

MSC confidence interval for Laplace corrected (λ=0.25) and uncorrected confusion matrices normalized by confidence interval for the full data set

Grahic Jump Location
Figure 5

Sample ROC curve

Grahic Jump Location
Figure 6

Sample cost curve for single threshold setting showing simple classifiers and 90% confidence intervals

Grahic Jump Location
Figure 7

Cost curves for algorithms 1 and 2

Grahic Jump Location
Figure 8

Cost curves showing bootstrap and analytical 90% confidence intervals for original sample size of 1000



Some tools below are only available to our subscribers or users with an online account.

Related Content

Customize your page view by dragging and repositioning the boxes below.

Related Journal Articles
Related eBook Content
Topic Collections

Sorry! You do not have access to this content. For assistance or to subscribe, please contact us:

  • TELEPHONE: 1-800-843-2763 (Toll-free in the USA)
  • EMAIL: asmedigitalcollection@asme.org
Sign In