Computer system and method for creating a supervised failure model

• US 10,635,095 B2
• Filed: 04/24/2018
• Issued: 04/28/2020
• Est. Priority Date: 04/24/2018
• Status: Active Grant

• Current Assignee: Uptake Technologies, Inc.
• Original Assignee: Uptake Technologies, Inc.
• Inventors: Herzog, James, Augustine, Benedict, Burns, Brian, Hall, Eric, Li, Tuo
• Primary Examiner(s): Cheung, Calvin
• Application Number:
  US15/961,715
• Publication Number:
  US 20190324430A1
• Time in Patent Office: 735 Days
• CPC Class Codes: G05B 23/024 Quantitative history assessment, e.g. mathematical relationships between available data; Functions therefor; Principal component analysis [PCA]; Partial least square [PLS]; Statistical classifiers, e.g. Bayesian networks, linear regression or correlation analysis; Neural networks
• CPC Class Codes: G05B 23/0254 based on a quantitative model, e.g. mathematical relationships between inputs and outputs; functions: observer, Kalman filter, residual calculation, Neural Networks
• CPC Class Codes: G05B 23/0283 Predictive maintenance, e.g. involving the monitoring of a system and, based on the monitoring results, taking decisions on the maintenance schedule of the monitored system; Estimating remaining useful life [RUL] preventive maintenance, i.e. planning maintenance according to the available resources without monitoring the system G06Q10/06
• CPC Class Codes: G06Q 50/40 Business processes related to the transportation industry shipping G06Q10/083
• CPC Class Codes: G07C 5/006 Indicating maintenance
• CPC Class Codes: G07C 5/008 communicating information to a remotely located station transmission systems for measured values G08C
• CPC Class Codes: G07C 5/0808 Diagnosing performance data testing of vehicles G01M17/00; testing of electrical installation on vehicles G01R31/005
• CPC Class Codes: G08G 1/20 Monitoring the location of vehicles belonging to a group, e.g. fleet of vehicles, countable or determined number of vehicles
• CPC Class Codes: G08G 1/207 with respect to certain areas, e.g. forbidden or allowed areas with possible alerting when inside or outside boundaries

Abstract

The example systems, methods, and devices disclosed herein generally relate to generating create a supervised failure model for assets in the given fleet that is configured to receive operating data as inputs and output a prediction as to the occurrence of a given failure type at the asset. In some instances, a data analytics platform may create and use an unsupervised failure model for a subset of the assets, use the respective unsupervised failure models to detect a set of anomalies that are each suggestive of a prior failure occurrence, from the set of anomalies, identify a subset of anomalies that are each suggest of a prior failure occurrence of the given failure type, and create the supervised failure model using failure data for the identified subset of anomalies.

Claims

1. A computing system comprising:

a network interface configured to facilitate communication with a plurality of assets in a given fleet;

at least one processor;

a non-transitory computer-readable medium; and

program instructions stored on the non-transitory computer-readable medium that are executable by the at least one processor to cause the computing system to create a supervised failure model for assets in the given fleet that is configured to receive operating data as inputs and output a prediction as to an occurrence of a given failure type at an asset of the given fleet, wherein program instructions that are executable to cause the computing system to create the supervised failure model comprise program instructions stored on the non-transitory computer-readable medium that are executable by the at least one processor to cause the computing system to;

identify a first subset of assets in the given fleet that have known prior failure occurrences;

for each asset in the first subset, obtain a respective set of cleaned historical operating data for the asset and then use the respective set of cleaned historical operating data for the asset to create a respective unsupervised failure model for the asset;

use the respective unsupervised failure model for each asset in the first subset to determine a set of deviation bounds for the first subset of assets;

for each remaining asset in the given fleet, obtain a respective set of historical operating data for the asset and then use the respective set of historical operating data for the asset to create a respective unsupervised failure model for the asset;

based on the respective unsupervised failure model for each remaining asset in the given fleet and the set of deviation bounds for the first subset of assets, classify each remaining asset in the given fleet into either a second subset of assets that have suspected prior failure occurrences or a third subset of assets that do not have known or suspected prior failure occurrences;

use the respective unsupervised failure model for each asset in the third subset to determine a set of deviation bounds for the third subset of assets;

based on the respective sets of deviation bounds for the first subset of assets and the third subset of assets, define a set of anomaly thresholds for the given fleet;

apply the set of anomaly thresholds to historical operating data for each of a selected group of assets in the given fleet and thereby detect a set of anomalies that are each suggestive of a prior failure occurrence;

from the set of anomalies, identify a subset of anomalies that are each suggestive of a prior occurrence of the given failure type; and

create the supervised failure model using failure data for the identified subset of anomalies.

2. The computing system of claim 1, wherein the program instructions that are executable by the at least one processor to cause the computing system to obtain a respective set of cleaned historical operating data for the asset comprise program instructions stored on the non-transitory computer-readable medium that are executable by the at least one processor to cause the computing system to:

obtain historical operating data for the asset from a given range of time in the past; and

remove historical operating data from any period of time within the given range that is associated with a known prior failure occurrence.

3. The computing system of claim 1, wherein the program instructions that are executable by the at least one processor to cause the computing system to use the respective unsupervised failure model for each asset in the first subset to determine a set of deviation bounds for the first subset of assets comprise program instructions stored on the non-transitory computer-readable medium that are executable by the at least one processor to cause the computing system to:

for each asset in the first subset, (1) input the respective set of cleaned historical operating data for the asset into the unsupervised failure model for the asset and thereby produce a corresponding prediction of the respective set of cleaned historical operating data for the asset, (2) evaluate the difference between the respective set of cleaned historical operating data for the asset and the corresponding prediction, and (3) based on the difference, determine a respective set of asset-specific deviation bounds for the asset; and

aggregate the respective set of asset-specific deviation bounds for each asset in the first subset to produce the set of deviation bounds for the first subset of assets.

4. The computing system of claim 1, wherein the program instructions that are executable by the at least one processor to cause the computing system to classify each remaining asset in the given fleet into either the second subset of assets or the third subset of assets comprise program instructions stored on the non-transitory computer-readable medium that are executable by the at least one processor to cause the computing system to:

for each remaining asset, (a) use the respective unsupervised failure model for the asset to determine a set of asset-specific deviation bounds for the asset, (b) compare the set of asset-specific deviation bounds for the asset to the set of deviation bounds for the first subset of assets, and (c) based on the comparison, classify the asset into (1) the second subset of assets if the set of asset-specific deviation bounds for the asset meaningfully differs from the set of deviation bounds for the first subset of assets and (2) the third subset of assets if the set of asset-specific deviation bounds for the asset does not meaningfully differ from the set of deviation bounds for the first subset of assets.

5. The computing system of claim 1, wherein the set of anomaly thresholds defined based on the respective sets of deviation bounds for the first subset of assets and the third subset of assets comprises a final set of anomaly thresholds, the computing system further comprising program instructions stored on the non-transitory computer-readable medium that are executable by the at least one processor to cause the computing system to:

based on the set of deviation bounds for the first subset of assets, define an initial set of anomaly thresholds.

6. The computing system of claim 5, wherein the program instructions that are executable by the at least one processor to cause the computing system to classify each remaining asset in the given fleet into either the second subset of assets or the third subset of assets comprise program instructions stored on the non-transitory computer-readable medium that are executable by the at least one processor to cause the computing system to:

for each remaining asset, (a) input a second respective set of historical operating data for the asset into the respective unsupervised failure model for the asset and thereby produce a corresponding prediction of the second respective set of historical operating data for the asset, (b) evaluate the difference between the second respective set of historical operating data for the asset and the corresponding prediction as compared to the initial set of anomaly thresholds, and (c) based on the evaluation, classify the asset into (1) the second subset of assets if the difference between the second respective set of historical operating data for the asset and the corresponding prediction exceeds at least one anomaly threshold in the initial set of anomaly thresholds and (2) the third subset of assets if the difference between the second respective set of historical operating data for the asset and the corresponding prediction does not exceed any anomaly threshold in the initial set of anomaly thresholds.

7. The computing system of claim 1, wherein the program instructions that are executable by the at least one processor to cause the computing system to define the set of anomaly thresholds for the given fleet based on the respective sets of deviation bounds for the first subset of assets and the third subset of assets comprise program instructions stored on the non-transitory computer-readable medium that are executable by the at least one processor to cause the computing system to:

aggregate the respective sets of deviation bounds for the first subset of assets and the third subset of assets to produce a set of deviation bounds for the given fleet; and

multiplying each deviation bound in the set of deviation bounds for the given fleet by a respective multiplication factor to produce a corresponding anomaly threshold that is included in the set of anomaly thresholds.

8. The computing system of claim 1, wherein the respective unsupervised failure model for each asset in the first subset comprises a regression model, and wherein the respective unsupervised failure model for each remaining asset in the given fleet comprises a regression model.

9. The computing system of claim 1, wherein the selected group of assets in the given fleet comprise the first subset of assets.

10. The computing system of claim 1, further comprising program instructions stored on the non-transitory computer-readable medium that are executable by the at least one processor to cause the computing system to:

after applying the set of anomaly thresholds to the historical operating data for each of the selected group of assets in the given fleet, evaluate an extent of false positives and false negatives detected by the set of anomaly thresholds; and

tune the set of anomaly thresholds to reduce the extent of false positives and false negatives.

11. The computing system of claim 10, further comprising program instructions stored on the non-transitory computer-readable medium that are executable by the at least one processor to cause the computing system to:

while evaluating the extent of false positives and false negatives detected by the set of anomaly thresholds, identify at least one asset in the third subset appearing to have at least one suspected prior failure occurrence; and

re-classify the identified at least one asset from the third subset of assets to the second subset of assets.

12. The computing system of claim 10, further comprising program instructions stored on the non-transitory computer-readable medium that are executable by the at least one processor to cause the computing system to:

apply the supervised failure model to operating data received from a given asset of the given fleet and thereby output a prediction as to the occurrence of the given failure type at the given asset.

13. The computing system of claim 1, wherein the failure data comprises:

(1) indications of occurrences of the given failure type, and (2) a respective functional failure time for each of the identified prior failure occurrences.

14. A computer-implemented method comprising:

creating a supervised failure model for assets in a given fleet of assets that is configured to receive operating data as inputs and output a prediction as to an occurrence of a given failure type at an asset of the given fleet, wherein creating the supervised failure model comprises;

identifying a first subset of assets in the given fleet that have known prior failure occurrences;

for each asset in the first subset, obtaining a respective set of cleaned historical operating data for the asset and then use the respective set of cleaned historical operating data for the asset to create a respective unsupervised failure model for the asset;

using the respective unsupervised failure model for each asset in the first subset to determine a set of deviation bounds for the first subset of assets;

for each remaining asset in the given fleet, obtaining a respective set of historical operating data for the asset and then use the respective set of historical operating data for the asset to create a respective unsupervised failure model for the asset;

based on the respective unsupervised failure model for each remaining asset in the given fleet and the set of deviation bounds for the first subset of assets, classifying each remaining asset in the given fleet into either a second subset of assets that have suspected prior failure occurrences or a third subset of assets that do not have known or suspected prior failure occurrences;

using the respective unsupervised failure model for each asset in the third subset to determine a set of deviation bounds for the third subset of assets;

based on the respective sets of deviation bounds for the first subset of assets and the third subset of assets, defining a set of anomaly thresholds for the given fleet;

applying the set of anomaly thresholds to historical operating data for each of a selected group of assets in the given fleet and thereby detect a set of anomalies that are each suggestive of a prior failure occurrence;

from the set of anomalies, identifying a subset of anomalies that are each suggestive of a prior occurrence of the given failure type; and

creating the supervised failure model using failure data for the identified subset of anomalies.

15. The computer-implemented method of claim 14, wherein using the respective unsupervised failure model for each asset in the first subset to determine a set of deviation bounds for the first subset of assets comprises:

for each asset in the first subset, (1) inputting the respective set of cleaned historical operating data for the asset into the unsupervised failure model for the asset and thereby producing a corresponding prediction of the respective set of cleaned historical operating data for the asset, (2) evaluating the difference between the respective set of cleaned historical operating data for the asset and the corresponding prediction, and (3) based on the difference, determining a respective set of asset-specific deviation bounds for the asset; and

aggregating the respective set of asset-specific deviation bounds for each asset in the first subset to produce the set of deviation bounds for the first subset of assets.

16. The computer-implemented method of claim 14, wherein the set of anomaly thresholds defined based on the respective sets of deviation bounds for the first subset of assets and the third subset of assets comprises a final set of anomaly thresholds, the method further comprising:

based on the set of deviation bounds for the first subset of assets, defining an initial set of anomaly thresholds.

17. The computer-implemented method of claim 14, wherein defining the set of anomaly thresholds for the given fleet based on the respective sets of deviation bounds for the first subset of assets and the third subset of assets comprises:

aggregating the respective sets of deviation bounds for the first subset of assets and the third subset of assets to produce a set of deviation bounds for the given fleet; and

multiplying each deviation bound in the set of deviation bounds for the given fleet by a respective multiplication factor to produce a corresponding anomaly threshold that is included in the set of anomaly thresholds.

18. A non-transitory computer-readable medium comprising programs instructions stored thereon that are executable to cause a computing system to:

create a supervised failure model for assets in a given fleet that is configured to receive operating data as inputs and output a prediction as to an occurrence of a given failure type at an asset of the given fleet, wherein program instructions that are executable to cause the computing system to create the supervised failure model comprise program instructions stored on the non-transitory computer-readable medium that are executable by the at least one processor to cause the computing system to;

identify a first subset of assets in the given fleet that have known prior failure occurrences;

for each asset in the first subset, obtain a respective set of cleaned historical operating data for the asset and then use the respective set of cleaned historical operating data for the asset to create a respective unsupervised failure model for the asset;

use the respective unsupervised failure model for each asset in the first subset to determine a set of deviation bounds for the first subset of assets;

for each remaining asset in the given fleet, obtain a respective set of historical operating data for the asset and then use the respective set of historical operating data for the asset to create a respective unsupervised failure model for the asset;

based on the respective unsupervised failure model for each remaining asset in the given fleet and the set of deviation bounds for the first subset of assets, classify each remaining asset in the given fleet into either a second subset of assets that have suspected prior failure occurrences or a third subset of assets that do not have known or suspected prior failure occurrences;

use the respective unsupervised failure model for each asset in the third subset to determine a set of deviation bounds for the third subset of assets;

based on the respective sets of deviation bounds for the first subset of assets and the third subset of assets, define a set of anomaly thresholds for the given fleet;

apply the set of anomaly thresholds to historical operating data for each of a selected group of assets in the given fleet and thereby detect a set of anomalies that are each suggestive of a prior failure occurrence;

from the set of anomalies, identify a subset of anomalies that are each suggestive of a prior occurrence of the given failure type; and

create the supervised failure model using failure data for the identified subset of anomalies.

19. The non-transitory computer-readable medium of claim 18, wherein the set of anomaly thresholds defined based on the respective sets of deviation bounds for the first subset of assets and the third subset of assets comprises a final set of anomaly thresholds, the computing system further comprising program instructions stored on the non-transitory computer-readable medium that are executable to cause the computing system to:

based on the set of deviation bounds for the first subset of assets, define an initial set of anomaly thresholds.

20. The non-transitory computer-readable medium of claim 18, wherein the program instructions that are executable to cause the computing system to define the set of anomaly thresholds for the given fleet based on the respective sets of deviation bounds for the first subset of assets and the third subset of assets comprise program instructions stored on the non-transitory computer-readable medium that are executable to cause the computing system to:

aggregate the respective sets of deviation bounds for the first subset of assets and the third subset of assets to produce a set of deviation bounds for the given fleet; and

multiply each deviation bound in the set of deviation bounds for the given fleet by a respective multiplication factor to produce a corresponding anomaly threshold that is included in the set of anomaly thresholds.