Method to use a receiver operator characteristics curve for model comparison in machine condition monitoring

• US 20050144537A1
• Filed: 10/28/2004
• Published: 06/30/2005
• Est. Priority Date: 11/12/2003
• Status: Active Grant

• Current Assignee: Siemens Corp. (Siemens AG)
• Original Assignee: Siemens Corporate Research Incorporated (Siemens AG)
• Inventors: Neubauer, Claus, Yuan, Chao, Cataltepe, Zehra
• Granted Patent: US 7,552,035 B2
• Time in Patent Office: Days
• US Class Current: 714/47
• CPC Class Codes: G05B 23/0243 model based detection method, e.g. first-principles knowledge model

Abstract

A method to use a receiver operator characteristics curve for model comparison in machine condition monitoring. The method and systems of using this method may be used to evaluate different monitoring models. These models may be used to monitor a variety of different systems such as power plant systems or magnetic resonance imaging systems. The methods use training data and designate one or more points in the data as a false negative, thereby permitting a receiver operator characteristics analysis to be performed. Multiple receiver operator characteristics analyses may be performed either on different models or on different points within a single model, thereby permitting the receiver operator characteristics analyses to be used to select a beneficial model for monitoring a particular system.

Claims

1. A method of evaluating a monitoring model comprising:

obtaining a first set of test data, the set of first test data including a first plurality of negative data points;

designating at least one of the first plurality of negative data points as a first positive data point; and

performing a first receiver operator characteristics analysis on the set of test data to obtain a first receiver operator characteristics analysis curve for the test data.

2. The method of claim 1, wherein the step of designating at least one of the first plurality of negative data points as a positive data point is performed using a first determinator selected from a trained technician, a statistical analysis of the data, a review of prior training data, or a combination thereof.

3. The method of claim 1, further comprising the steps of:

obtaining a second set of test data is obtained, the second set of test data including a second plurality of negative data points;

designating at least one of the second plurality of negative data points as a positive data point;

performing a receiver operator characteristics analysis on the second set of test data to obtain a second receiver operator characteristics analysis curve for the test data; and

comparing the first receiver operator characteristics analysis curve with the second receiver operator characteristics analysis curve to select a monitoring model.

4. The method of claim 3, wherein the step of designating at least one of the first plurality of negative data points as a positive data point is performed using a first determinator selected from a trained technician, a statistical analysis of the data, a review of prior training data, or a combination thereof;

further wherein the step of designating at least one of the second plurality of negative data points as a positive data point is performed using a second determinator selected from a trained technician, a statistical analysis of the data, a review of prior training data, or a combination thereof;

wherein the first determinator is different than the second determinator.

5. The method of claim 1, further comprising the steps of:

designating another of at least one of the first plurality of negative data points as a second positive data point;

performing a second receiver operator characteristics analysis on the set of test data to obtain a second receiver operator characteristics analysis curve for the test data; and

.

comparing the first receiver operator characteristics analysis curve with the second receiver operator characteristics analysis curve to select a monitoring model.

6. A machine-readable storage having stored thereon, a computer program having a plurality of code sections, said code sections executable by a machine for causing the machine to perform the steps of:

obtaining a first set of test data, the set of first test data including a first plurality of negative data points;

designating at least one of the first plurality of negative data points as a first positive data point; and

performing a first receiver operator characteristics analysis on the set of test data to obtain a first receiver operator characteristics analysis curve for the test data.

7. The machine-readable storage of claim 6, wherein the step of designating at least one of the first plurality of negative data points as a positive data point is performed using a determinator selected from a trained technician, a statistical analysis of the data, a review of prior training data, or a combination thereof.

8. The machine-readable storage of claim 6, further comprising code sections executable by a machine for causing the machine to perform the steps of:

obtaining a second set of test data is obtained, the second set of test data including a second plurality of negative data points;

designating at least one of the second plurality of negative data points as a positive data point;

performing a receiver operator characteristics analysis on the second set of test data to obtain a second receiver operator characteristics analysis curve for the test data; and

comparing the first receiver operator characteristics analysis curve with the second receiver operator characteristics analysis curve to select a monitoring model.

9. The machine-readable storage of claim 8, wherein the step of designating at least one of the first plurality of negative data points as a positive data point is performed using a first determinator selected from a trained technician, a statistical analysis of the data, a review of prior training data, or a combination thereof;

further wherein the step of designating at least one of the second plurality of negative data points as a-positive data point is performed using a second determinator selected from a trained technician, a statistical analysis of the data, a review of prior training data, or a combination thereof;

wherein the first determinator is different than the second determinator.

10. The machine-readable storage of claim 6, further comprising code sections executable by a machine for causing the machine to perform the steps of:

designating another of at least one of the first plurality of negative data points as a second positive data point;

performing a second receiver operator characteristics analysis on the set of test data to obtain a second receiver operator characteristics analysis curve for the test data; and

.

comparing the first receiver operator characteristics analysis curve with the second receiver operator characteristics analysis curve to select a monitoring model.

11. A system for evaluating a monitoring model comprising:

means for obtaining a first set of test data, the set of first test data including a first plurality of negative data points;

a first determinator for designating at least one of the first plurality of negative data points as a first positive data point; and

means for performing a first receiver operator characteristics analysis on the set of test data to obtain a first receiver operator characteristics analysis curve for the test data.

12. The system of claim 11, wherein the first determinator is selected from a trained technician, a statistical analysis of the data, a review of prior training data, or a combination thereof.

13. The system of claim 11, further comprising:

means for obtaining a second set of test data is obtained, the second set of test data including a second plurality of negative data points;

a second determinator for designating at least one of the second plurality of negative data points as a positive data point;

means for performing a receiver operator characteristics analysis on the second set of test data to obtain a second receiver operator characteristics analysis curve for the test data; and

means for comparing the first receiver operator characteristics analysis curve with the second receiver operator characteristics analysis curve to select a monitoring model.

14. The system of claim 13, wherein the first determinator is selected from a trained technician, a statistical analysis of the data, a review of prior training data, or a combination thereof;

further wherein the second determinator is selected from a trained technician, a statistical analysis of the data, a review of prior training data, or a combination thereof;

wherein the first determinator is different than the second determinator.

15. The system of claim 11, further comprising:

a determinator for designating another of at least one of the first plurality of negative data points as a second positive data point;

means for performing a second receiver operator characteristics analysis on the set of test data to obtain a second receiver operator characteristics analysis curve for the test data; and

.

means for comparing the first receiver operator characteristics analysis curve with the second receiver operator characteristics analysis curve to select a monitoring model.