Cloud-based machine learning system and data fusion for the prediction and detection of corrosion under insulation

• US 10,533,937 B1
• Filed: 08/30/2018
• Issued: 01/14/2020
• Est. Priority Date: 08/30/2018
• Status: Active Grant

• Current Assignee: Avitas Systems, Inc. (General Electric Company), Saudi Arabian Oil Company (Government of Saudi Arabia)
• Original Assignee: Saudi Arabian Oil Company (Government of Saudi Arabia)
• Inventors: Shehri, Ali Al, Lim, Ser Nam, Amer, Ayman, Uzunbas, Mustafa, Aldabbagh, Ahmad, Ababtain, Muhammad, Cunningham, Vincent
• Primary Examiner(s): Maupin, Hugh
• Application Number:
  US16/117,937
• Time in Patent Office: 502 Days
• CPC Class Codes: G01N 17/00 Investigating resistance of materials to the weather, to corrosion, or to light
• CPC Class Codes: G01N 17/006 of metals
• CPC Class Codes: G01N 17/04 Corrosion probes
• CPC Class Codes: G01N 21/956 Inspecting patterns on the surface of objects contactless testing of electronic circuits G01R31/308; testing currency G07D; manufacturing processes per se of semiconductor devices implementing a measuring step H01L22/10
• CPC Class Codes: G01N 25/72 Investigating presence of flaws
• CPC Class Codes: G06F 18/214 Generating training patterns; Bootstrap methods, e.g. bagging or boosting
• CPC Class Codes: G06N 3/0418 using chaos or fractal principles
• CPC Class Codes: G06N 3/044 Recurrent networks, e.g. Hopfield networks
• CPC Class Codes: G06N 3/045 Combinations of networks
• CPC Class Codes: G06N 3/08 Learning methods
• CPC Class Codes: G06N 3/088 Non-supervised learning, e.g. competitive learning
• CPC Class Codes: G06V 20/20 in augmented reality scenes

Abstract

A system for predicting corrosion under insulation (CUI) in an infrastructure asset includes at least one infrared camera positioned to capture thermal images of the asset, at least one smart mount supporting and electrically coupled to the at least one infrared camera and including a wireless communication module, memory storage, a battery module operative to recharge the at least one infrared camera, an ambient sensor module adapted to obtain ambient condition data and a structural probe sensor to obtain CUI-related data from the asset. At least one computing device has a wireless communication module that communicates with the at least one smart mount and is configured with a machine learning algorithm that outputs a CUI prediction regarding the asset. A cloud computing platform receive and stores the received data and the prediction output and to receive verification data for updating the machine learning algorithm stored on the computing device.

Claims

1. A system for predicting and detecting of corrosion under insulation (CUI) in an infrastructure asset using machine learning and data fusion comprising:

at least one infrared camera positioned to capture thermal images of the asset;

at least one smart mount mechanically supporting and electrically coupled to the at least one infrared camera, the at least one smart mount including;

a wireless communication module;

memory storage adapted to store thermal image data received from the at least one camera;

a battery module operative to recharge the at least one infrared camera;

an ambient sensor module adapted to obtain ambient condition data; and

a structural probe sensor adapted to obtain CUI-related data from the asset;

at least one computing device having a wireless communication module that is communicatively coupled to the at least one smart mount, the computing device being configured with instructions for executing a machine learning algorithm taking as inputs thermal image data, ambient condition data and CUI-related data from the probe sensor providing data fusion, and adapted to output a CUI prediction regarding the asset; and

a cloud computing platform adapted to receive and store the thermal image data, ambient condition data and CUI-related data from the probe sensor, and the prediction output by the computing device, the cloud computing platform adapted to receive verification data for updating the machine learning algorithm stored on the computing device.

2. The system of claim 1, wherein the at least one smart mount includes a fixture for supporting the infrared camera, the fixture being rotatable and extendable to enable the infrared camera to be translated and tilted.

3. The system of claim 1, wherein the asset includes identification tags and at least one smart mount further includes a standard camera operative to scan the identification tags on the asset.

4. The system of claim 1, wherein the ambient sensor module is operative to detect temperature, humidity and air pressure.

5. The system of claim 1, wherein the structural probe sensor includes a magnetic sensor.

6. The system of claim 1, further comprising a control station communicatively coupled to the at least one smart mount and adapted to transmit configuration and control commands to the at least one smart mount.

7. The system of claim 1, wherein the machine learning algorithm employed by the at least one computing device includes a deep recurrent neural network.

8. The system of claim 7, wherein the deep recurrent neural network is a long short term memory (LSTM) network.

9. The system of claim 1, wherein the machine learning algorithm employed by the at least one computing device further includes a convolutional neural network.

10. The system of claim 1, wherein the at least one computing device is configured to perform noise reduction on the data received from the at least one smart mount.

11. The system of claim 1, wherein each of the at least one mounts can communicate with each other via their respective communication modules.

12. The system of claim 1, wherein the at least one smart mount is implemented as a tripod.

14. The method of claim 12, wherein the additional sensing mode is based on magnetometry.

15. The method of claim 12, wherein the ambient condition data includes temperature, humidity and air pressure measurements.

17. The method of claim 15, further comprising filtering the received data for noise.

18. The method of claim 15, further comprising generating synthetic thermal image data based on ambient conditions and parameters of the asset using a thermal dynamic model.

19. The method of claim 17, wherein, at the cloud computing platform, the synthetic thermal image data is combined with the stream of including thermal images of the asset, additional sensor probe data of the asset, and ambient conditions at the asset to create a data training set for training a machine learning model.

20. The method of claim 18, wherein the machine learning model including a deep recurrent neural network.

21. The method of claim 19, wherein the deep recurrent neural network includes a long short memory network (LSTM).

22. The method of claim 18, wherein the machine learning model further includes a convolutional neural network.

13. A method of obtaining data from an infrastructure asset for enabling prediction and detection of corrosion-under-insulation (CUI) comprising:

capturing thermal image data of the asset overtime;

probing the asset using an additional sensing mode to obtain additional probe over time;

measuring ambient conditions to obtain ambient condition data overtime;

combining the thermal image, additional probe and ambient condition data into a computer readable file; and

scanning the asset for identification tags to obtain tag photo data;

including the tag photo data in the computer readable file; and

transmitting the file to a computing device that uses an algorithm that uses the thermal image, additional probe and ambient condition data to predict whether the asset contains CUI.

16. A method of predicting corrosion-under-insulation (CUI) in an infrastructure asset using a cloud computing platform and data fusion, comprising:

receiving a stream of data including thermal images of the asset, additional sensor probe data of the asset, and ambient conditions at the asset;

executing, in real time, one or more machine learning algorithms using the received stream of data and weights received as updated from the cloud computing platform to generate a prediction as to whether the asset contains CUI; and

transmitting the received stream of data and prediction to the cloud computing platform.