Equipment damage prediction system using neural networks

• US 10,268,913 B2
• Filed: 04/03/2017
• Issued: 04/23/2019
• Est. Priority Date: 04/03/2017
• Status: Active Grant

• Current Assignee: General Electric Company
• Original Assignee: General Electric Company
• Inventors: Lim, Ser Nam, Jain, Arpit, Diwinsky, David, Bondugula, Sravanthi, Lin, Yen-Liang, Bian, Xiao
• Primary Examiner(s): Misleh, Justin P.
• Application Number:
  US15/477,517
• Publication Number:
  US 20180286034A1
• Time in Patent Office: 750 Days
• Field of Search: None
• CPC Class Codes: G06F 18/2433 Single-class perspective, e.g. one-against-all classification; Novelty detection; Outlier detection
• CPC Class Codes: G06F 18/29 Graphical models, e.g. Bayesian networks
• CPC Class Codes: G06N 3/045 Combinations of networks
• CPC Class Codes: G06N 3/047 Probabilistic or stochastic networks
• CPC Class Codes: G06N 3/08 Learning methods
• CPC Class Codes: G06T 2207/20081 Training; Learning
• CPC Class Codes: G06T 2207/20084 Artificial neural networks [ANN]
• CPC Class Codes: G06T 2207/30164 Workpiece; Machine component
• CPC Class Codes: G06V 10/764 using classification, e.g. of video objects
• CPC Class Codes: G06V 10/84 using probabilistic graphical models from image or video features, e.g. Markov models or Bayesian networks
• CPC Class Codes: G06V 10/94 Hardware or software architectures specially adapted for image or video understanding
• CPC Class Codes: G06V 20/52 Surveillance or monitoring of activities, e.g. for recognising suspicious objects recognising microscopic objects G06V20/69

Abstract

A generative adversarial network (GAN) system includes a generator sub-network configured to examine one or more images of actual damage to equipment. The generator sub-network also is configured to create one or more images of potential damage based on the one or more images of actual damage that were examined. The GAN system also includes a discriminator sub-network configured to examine the one or more images of potential damage to determine whether the one or more images of potential damage represent progression of the actual damage to the equipment.

Claims

1. A repair system comprising:

a generative adversarial network (GAN) system comprising;

a generator sub-network configured to examine one or more images of actual damage to equipment, the generator sub-network also configured to create one or more images of potential damage based on the one or more images of actual damage that were examined; and

a discriminator sub-network configured to examine the one or more images of potential damage to determine whether the one or more images of potential damage represent progression of the actual damage to the equipment,

the repair system further comprising an automated system for automatically repairing at least one damaged portion of at least one component of the equipment based on the one or more images of potential damage, the automated system comprising a robotic system,

wherein the at least one component comprises at least one turbine blade, and

wherein the robotic system sprays an additive onto a thermal barrier coating of the at least one turbine blade.

2. The system of claim 1, wherein the discriminator sub-network is configured to determine one or more loss functions indicative of errors in the one or more images of potential damage,

wherein the at least one component comprises at least one turbine blade, and

wherein the robotic system sprays an additive onto a thermal barrier coating of the at least one turbine blade.

3. The system of claim 1, wherein the generator sub-network is configured to be trained using the one or more images of actual damage, and

wherein a maintenance schedule of the at least one component comprises a first time, the first time comprising an originally scheduled maintenance event, and

wherein the maintenance schedule is adjusted to provide maintenance at a different time from the first time.

4. The system of claim 3, wherein the generator sub-network is configured to be trained using the one or more images of actual damage by determining distributions of pixel characteristics of the one or more images of actual damage, and

wherein the maintenance schedule is adjusted to provide maintenance before the first time.

5. The system of claim 3, further comprising a controller configured to implement one or more actions responsive to determining that the one or more images of potential damage represent progression of the actual damage, and

wherein the maintenance schedule is adjusted to provide maintenance after the first time.

6. The system of claim 1, wherein the discriminator sub-network is configured determine whether the one or more images of potential damage represent the progression of the actual damage to the equipment by determining one or more loss functions of the one or more images of potential damage, and

wherein the at least one component comprises the equipment.

7. The system of claim 6, wherein the discriminator sub-network is configured determine that the one or more images of potential damage represent the progression of the actual damage to the equipment responsive to the one or more loss functions of the one or more images of potential damage not exceeding a designated threshold.

8. A method comprising:

examining one or more images of actual damage to equipment using a generator sub-network of a generative adversarial network (GAN);

creating one or more images of potential damage using the generator sub-network based on the one or more images of actual damage that were examined;

determining whether the one or more images of potential damage represent progression of the actual damage to the equipment by examining the one or more images of potential damage using a discriminator sub-network of the GAN, and

automatically repairing, using a robotic system, at least one damaged portion of the equipment based on the one or more images of potential damage,

wherein the equipment comprises at least one of a surface of a road and a surface of a sidewalk.

9. The method of claim 8, further comprising determining one or more loss functions indicative of errors in the one or more images of potential damage using the discriminator sub-network,

wherein different Gaussian distributions are determined for different portions of the one or more images.

10. The method of claim 9, further comprising training the generator subnetwork using the one or more images of actual damage.

11. The method of claim 10, wherein training the generator sub-network includes determining distributions of pixel characteristics of the one or more images of actual damage.

12. The method of claim 8, further comprising implementing one or more actions responsive to determining that the one or more images of potential damage represent progression of the actual damage,

wherein the determination of whether the one or more images of potential damage represent progression of the actual damage to the equipment is used to automatically repair the equipment from a damaged state to a repaired state.

13. The method of claim 8, wherein determining whether the one or more images of potential damage represent the progression of the actual damage to the equipment includes determining one or more loss functions of the one or more images of potential damage.

14. The method of claim 13, wherein determining that the one or more images of potential damage represent the progression of the actual damage to the equipment occurs responsive to the one or more loss functions of the one or more images of potential damage not exceeding a designated threshold, and

wherein the equipment comprises at least one of a surface of a road, a surface of a sidewalk, and a surface of a vehicle.

15. A repair system comprising:

a generative adversarial network (GAN) system comprising;

a generator sub-network configured to be trained using one or more images of actual damage to equipment, the one or more images comprising one or more pixels, the generator sub-network also configured to create one or more images of potential damage based on the one or more images of actual damage that were examined; and

a discriminator sub-network configured to examine the one or more images of potential damage to determine whether the one or more images of potential damage represent progression of the actual damage to the equipment,

the repair system further comprising an automated system for automatically repairing at least one damaged portion of the equipment based on the one or more images of potential damage, the automated system comprising a robotic system,

wherein the discriminator sub-network classifies the one or more pixels into different categories of objects, and

wherein the different categories of objects include at least one of a tree, a car, a person, a bird, spalling of a thermal barrier coating, a sign, and a crack in a surface.

16. The system of claim 15, wherein the discriminator sub-network is configured to determine one or more loss functions indicative of errors in the one or more images of potential damage.

17. The system of claim 15, wherein the generator sub-network is configured to be trained using the one or more images of actual damage by determining distributions of pixel characteristics of the one or more images of actual damage.

18. The system of claim 15, further comprising a controller configured to implement one or more actions responsive to determining that the one or more images of potential damage represent progression of the actual damage.

19. The system of claim 15, wherein the discriminator sub-network is configured determine whether the one or more images of potential damage represent the progression of the actual damage to the equipment by determining one or more loss functions of the one or more images of potential damage.

20. The system of claim 19, wherein the discriminator sub-network is configured determine that the one or more images of potential damage represent the progression of the actual damage to the equipment responsive to the one or more loss functions of the one or more images of potential damage not exceeding a designated threshold, and

wherein the different categories of objects include at least one of a tree, a car, a person, a bird, spalling of a thermal barrier coating, and a crack in a surface.