Method and system to automate the maintenance of data-driven analytic models

• US 9,904,540 B2
• Filed: 06/05/2014
• Issued: 02/27/2018
• Est. Priority Date: 06/05/2014
• Status: Active Grant

• Current Assignee: GE Digital Holdings LLC (GE Vernova, Inc.)
• Original Assignee: General Electric Company
• Inventors: Bonissone, Piero Patrone, Iyer, Naresh Sundaram, Xue, Feng, Reimann, Johan Michael, Ardis, Paul
• Primary Examiner(s): JEON, JAE UK
• Application Number:
  US14/297,297
• Publication Number:
  US 20150355901A1
• Time in Patent Office: 1,363 Days
• Field of Search: 717101, 717104, 717120, 717121, 717122, 717123
• CPC Class Codes: G06F 8/355 Round-trip engineering
• CPC Class Codes: G06F 8/70 Software maintenance or management

Abstract

A method, system, and non-transitory computer-readable medium, the method including determining automatically, by a processor, whether behavior for a model representing a plurality of entities and relationships therebetween deviates from a reference behavior for the model; determining, in response to the determination that the model does deviate from the reference behavior, at least one basis for the deviation; automatically forecasting an estimate of a remaining useful life for the model; and modifying the model to compensate for the deviation by at least one of modifying the model to accommodate the deviation and updating the model based on at least one new requirement.

Claims

1. A computer-implemented method to control a maintenance of a model on a software platform executing the model, the method comprising:

automatically determining, by a processor, that a behavior of a model executing on a software platform and representing a plurality of entities and relationships between the plurality of entities deviates from a reference behavior for the model, the behavior of the model being represented as metadata that describes properties of the model and the determining including;

extracting features from the metadata; and

determining at least one statistical relationship between the extracted features of the metadata and a deviation of the model from a norm for the extracted features;

determining a pattern in the at least one statistical relationship between the extracted features of the metadata and the deviation of the model from a predetermined norm for the extracted features;

automatically determining, by the processor in response to the determined pattern in the at least one statistical relationship, at least one model failure mode for the model;

automatically forecasting, by the processor, a future deviation of the model based on the determined model deviation and in terms of an estimate of a remaining useful life for the model;

modifying, by the processor, the model to compensate for the determined model deviation by operating the model to accommodate the determined model deviation and to reduce a future estimated deviation for the model; and

updating, by the processor, the model based on at least one new requirement, the at least one new requirement being based at least on the modifications to accommodate the determined model deviation.

2. The method of claim 1, further comprising receiving the metadata from a deployed instance of the model.

3. The method of claim 1, further comprising capturing outcomes of the method and updating a model history to include the outcomes.

4. A non-transitory computer-readable medium having executable instructions stored thereon to control a maintenance of a model on a software platform executing the model, the medium comprising:

instructions to automatically determine whether behavior for a model representing a plurality of entities and relationships between the plurality of entities deviates from a reference behavior for the model, the behavior of the model being represented as metadata that describes properties of the model and the determining including;

extracting features from the metadata; and

determining at least one statistical relationship between the extracted features of the metadata and a deviation of the model from a norm for the extracted features;

instructions to determine a pattern in the at least one statistical relationship between the extracted features of the metadata and the deviation of the model from the norm for the extracted features;

instructions to automatically determine, in response to the determined pattern in the at least one statistical relationship, at least one model failure mode for the model;

instructions to automatically forecast a feature deviation of the model based on the determined model deviation and in terms of an estimate of a remaining useful life for the model;

instructions to modify the model to compensate for the determined deviation by operating the model to accommodate the determined deviation and to reduce a future estimated deviation for the model; and

instructions to update the model based on at least one new requirement, the at least one new requirement being based at least on the modifications to accommodate the deviation.

5. The medium of claim 4, further comprising instructions to receive metadata from the deployed model.

6. The medium of claim 4, further comprising:

instructions to capture outcomes of the method; and

instructions to update a model history to include the outcomes.

7. A system comprising:

a storage platform hosting a model;

a meta-model to control a maintenance of the meta-model deployed on the software platform;

a storage device; and

a processor in communication with the storage device and operable to;

automatically determine whether behavior for a model representing a plurality of entities and relationships between the plurality of entities deviates from a reference behavior for the model, the behavior of the model being represented as metadata that describes properties of the model and the determining including;

extracting features from the metadata; and

determine at least one statistical relationship between the extracted features of the metadata and a deviation of the model from a norm for the extracted features;

determine a pattern in the at least one statistical relationship between the extracted features of the metadata and the deviation of the model from the norm for the extracted features;

automatically determine, in response to the determined pattern in the at least one statistical relationship, at least one model failure mode for the model;

automatically determine, in response to the determined pattern in the at least one statistical relationship, at least one model failure mode for the model;

automatically forecast a future deviation of the model based on the determined model deviation and in terms of an estimate of a remaining useful life for the model;

modify the model to compensate for the determined deviation by operating the model to accommodate the determined deviation and to reduce a future estimated deviation for the model; and

update the model based on at least one new requirement, the at least one new requirement being based at least on the modifications to accommodate the deviation.

8. The system of claim 7, wherein the processor is further operable to receive metadata from the deployed model.

9. The system of claim 7, wherein the processor is further operable to:

capture outcomes of the method; and

update a model history to include the outcomes.