Automatic classification of aircraft component distress

• US 9,785,919 B2
• Filed: 12/10/2015
• Issued: 10/10/2017
• Est. Priority Date: 12/10/2015
• Status: Active Grant

• Current Assignee: General Electric Company
• Original Assignee: General Electric Company
• Inventors: Diwinsky, David Scott, Lim, Ser Nam, Bian, Xiao
• Primary Examiner(s): Sherali, Ishrat I
• Application Number:
  US14/964,922
• Publication Number:
  US 20170169400A1
• Time in Patent Office: 670 Days
• Field of Search: 382141, 382149, 382155-159
• CPC Class Codes: G06F 16/51 Indexing; Data structures therefor; Storage structures
• CPC Class Codes: G06F 18/24 Classification techniques
• CPC Class Codes: G06Q 10/20 Administration of product repair or maintenance
• CPC Class Codes: G06T 2207/20016 Hierarchical, coarse-to-fine, multiscale or multiresolution image processing; Pyramid transform
• CPC Class Codes: G06T 2207/20081 Training; Learning
• CPC Class Codes: G06T 2207/20084 Artificial neural networks [ANN]
• CPC Class Codes: G06T 2207/20112 Image segmentation details
• CPC Class Codes: G06T 2207/30164 Workpiece; Machine component
• CPC Class Codes: G06T 7/0004 Industrial image inspection
• CPC Class Codes: G06T 7/11 Region-based segmentation
• CPC Class Codes: G06V 10/764 using classification, e.g. of video objects
• CPC Class Codes: G06V 10/82 using neural networks

Abstract

Systems and methods for automatically identifying and classifying distress of an aircraft component are provided. In one embodiment, a method includes accessing one or more digital images captured of the aircraft component and providing the one or more digital images as an input to a multi-layer network image classification model. The method further includes generating a classification output for the one or more images from the multi-layer network image classification model and automatically classifying the distress of the aircraft component based at least in part on the classification output.

Claims

1. A method of automatically classifying distress of a component of a gas turbine engine, the method comprising:

accessing, by one or more computing devices, one or more digital images captured of the component of the gas turbine engine;

providing, by the one or more computing devices, the one or more digital images as an input to a multi-layer network image classification model;

generating, by the one or more computing devices, a classification output for the one or more images from the multi-layer network image classification model;

automatically classifying, by the one or more computing devices, the distress levels of the component of the gas turbine engine based at least in part on the classification output; and

providing the distress level of the component of the gas turbine for display on one or more display devices.

2. The method of claim 1, wherein the multi-layer image classification model comprises a multi-scale fully convolutional model.

3. The method of claim 2, wherein the multi-scale fully convolutional model comprises a plurality of multi-layer networks, each multi-layer network associated with a different size receptive field.

4. The method of claim 3, wherein generating, by the one or more computing devices, a classification output from the multi-layer network image classification model comprises:

concatenating outputs from each of the plurality of multi-layer networks to determine a concatenated output; and

generating the classification output based at least in part on the concatenated output.

5. The method of claim 1, wherein the classification output comprises a segmentation map, the segmentation map indicating one or more areas of distress on the component of the gas turbine engine.

6. The method of claim 5, wherein the segmentation map has a plurality of pixels, the pixel value of each of the plurality of pixels corresponding to whether the pixel is associated with an area of distress on the component of the gas turbine engine.

7. The method of claim 1, wherein classifying, by the one or more computing devices, the distress of the component of the gas turbine engine comprises classifying the distress into one of a plurality of distress levels based at least in part on the segmentation map.

8. The method of claim 7, wherein the method further comprises conducting a maintenance action based at least in part on the distress level in which the component of the gas turbine engine is classified.

9. The method of claim 1, wherein the component of the gas turbine engine comprises an engine blade.

10. A system for inspecting a component of a gas turbine, the system comprising:

one or more image capture devices configured to capture a digital images of a component of a gas turbine;

at least one display device;

one or more processors; and

one or more memory devices, the one or more memory devices storing a multi-layer network image classification model, the one or more memory devices further storing computer-readable instructions that when executed by the one or more processors cause the one or more processors to perform operations, the operations comprising;

accessing one or more digital images captured of the component of the gas turbine;

providing the one or more digital images as an input to the multi-layer network image classification model;

generating a classification output for the one or more images from the multi-layer network image classification model;

automatically classifying distress of the component of the gas turbine into one of a plurality of distress levels based at least in part on the classification output; and

providing the distress level of the component of the gas turbine for display on the one or more display devices.

11. The system of claim 10, wherein the multi-layer image classification model comprises a multi-scale fully convolutional model.

12. The system of claim 11, wherein the multi-scale fully convolutional model comprises a plurality of multi-layer networks, each multi-layer network associated with a different size receptive field.

13. The system of claim 12, wherein the operation of generating a classification output from the multi-layer network image classification model comprises:

concatenating outputs from each of the plurality of multi-layer networks to determine a concatenated output; and

generating the classification output based at least in part on the concatenated output.

14. The system of claim 11, wherein the classification output comprises a segmentation map, the segmentation map indicating one or more areas of distress on the component of the gas turbine.

15. The system of claim 14, wherein the classification level is determined based at least in part on a size of the area of distress in the segmentation map.

16. One or more tangible, non-transitory computer-readable media storing computer-readable instructions that when executed by one or more processors cause the one or more processors to perform operations, the operations comprising:

accessing one or more digital images captured of a component of a gas turbine engine;

providing the one or more digital images as an input to a multi-scale fully convolutional model;

generating a segmentation map for the one or more images from a multi-scale fully convolutional model, the segmentation map indicating one or more areas of distress on the component;

automatically classifying distress of the component into one of a plurality of distress levels based at least in part on the segmentation map; and

providing the distress level of the component of the gas turbine engine for display on one or more display devices.

17. The computer-readable media of claim 16, wherein the multi-scale fully convolutional model comprises a plurality of multi-layer networks, each multi-layer network associated with a different size receptive field.

18. The computer-readable media of claim 17, wherein the operation of generating a segmentation map from the multi-layer network image classification model comprises:

concatenating outputs from each of the plurality of multi-layer networks to determine a concatenated output; and

generating the segmentation map based at least in part on the concatenated output.

19. The computer-readable media of claim 16, wherein the segmentation map has a plurality of pixels, the pixel value of each of the plurality of pixels corresponding to whether the pixel is associated with an area of distress on the component of the gas turbine engine.

20. The computer-readable media of claim 19, wherein the gas turbine engine component is classified into one of the plurality of distress levels based at least in part on a number of pixels associated with an area of distress on the component of the gas turbine engine.