Robot system path planning for asset health management

• US 10,618,168 B2
• Filed: 03/29/2017
• Issued: 04/14/2020
• Est. Priority Date: 05/13/2016
• Status: Active Grant

• Current Assignee: GE Infrastructure Technology, LLC (General Electric Company)
• Original Assignee: General Electric Company
• Inventors: Gros, Eric Michael, Tan, Huan, Castillo-Effen, Mauricio, Theurer, Charles Burton
• Primary Examiner(s): Nguyen, Robert T
• Application Number:
  US15/473,345
• Publication Number:
  US 20170326729A1
• Time in Patent Office: 1,112 Days
• CPC Class Codes: B25J 9/1602 characterised by the control system, structure, architecture
• CPC Class Codes: B25J 9/163 learning, adaptive, model based, rule based expert control
• CPC Class Codes: B25J 9/1661 characterised by task planning, object-oriented languages
• CPC Class Codes: B25J 9/1664 characterised by motion, path, trajectory planning
• CPC Class Codes: B25J 9/1671 characterised by simulation, either to verify existing program or to create and verify new program, CAD/CAM oriented, graphic oriented programming systems
• CPC Class Codes: B25J 9/1697 Vision controlled systems
• CPC Class Codes: F01D 5/005 Repairing methods or devices
• CPC Class Codes: F22B 37/00 Component parts or details of steam boilers venting devices F16K24/00; steam traps or like apparatus F16T
• CPC Class Codes: G05B 19/042 using digital processors G05B19/05 takes precedence
• CPC Class Codes: G05B 19/048 Monitoring; Safety
• CPC Class Codes: G05B 19/4065 Monitoring tool breakage, life or condition
• CPC Class Codes: G05B 19/4097 characterised by using design data to control NC machines, e.g. CAD/CAM G05B19/4093 takes precedence
• CPC Class Codes: G05B 2219/35134 3-D cad-cam
• CPC Class Codes: G05B 2219/40323 Modeling robot environment for sensor based robot system
• CPC Class Codes: G05B 2219/42329 Defective measurement, sensor failure
• CPC Class Codes: G05B 2219/49007 Making, forming 3-D object, model, surface
• CPC Class Codes: G05D 1/0094 involving pointing a payload, e.g. camera, weapon, sensor, towards a fixed or moving target
• CPC Class Codes: G06Q 10/08 Logistics, e.g. warehousing, loading or distribution; Inventory or stock management
• CPC Class Codes: Y02P 90/02 Total factory control, e.g. smart factories, flexible manufacturing systems [FMS] or integrated manufacturing systems [IMS]
• CPC Class Codes: Y10S 901/01 Mobile robot
• CPC Class Codes: Y10S 901/44 : inspection

Abstract

A robotic system includes a processing system comprising at least one processor. The processor generates a plan to monitor the asset. The plan comprises one or more tasks to be performed by the at least one robot. The processor receives sensor data from at least one sensor indicating one or more characteristics of the asset. The processor adjusts the plan to monitor the asset by adjusting or adding one or more tasks to the plan based on one or both of the quality of the acquired data or a potential defect of the asset. The adjusted plan causes the at least one robot to acquire additional data related to the asset when executed.

Claims

1. A method, comprising:

generating a flight plan to monitor an asset for defects, wherein the flight plan comprises one or more tasks to be performed by at least one robot via at least one effector of the at least one robot;

executing the flight plan to acquire sensor data via at least one sensor of the at least one robot indicating one or more characteristics of the asset at a location on the asset;

determining whether a quality of the sensor data acquired at the location on the asset is below a threshold quality level and, in response to a determination that the quality of the sensor data is below the threshold quality level at the location;

dynamically adjusting the flight plan to an updated flight plan by adjusting or adding one or more tasks to the flight plan to iteratively acquire additional sensor data indicative of the one or more characteristics of the asset at the location on the asset via the at least one sensor; and

determining, after each iterative acquisition of the additional sensor data, whether the quality of the sensor data combined with the additional sensor data acquired at the location on the asset is above the threshold quality level; and

generating a signal encoding or conveying instructions to control the at least one robot to address a defect of the asset identified from the sensor data and the additional sensor data upon a determination that the quality of the sensor data combined with the additional sensor data is above the threshold quality level.

2. The method of claim 1, comprising adjusting the flight plan to monitor the asset based on detection of the defect of the asset so as to cause the at least one robot to acquire additional data related to the defect.

3. The method of claim 1, comprising generating the flight plan based on one or more of coverage of the asset, excluded areas from visibility of the asset, or high risk areas of the asset more likely to have defects than other areas of the asset.

4. The method of claim 1, comprising determining whether the quality of the sensor data is below the threshold quality level based on a signal-to-noise ratio of the sensor data.

5. The method of claim 1, comprising determining a path for the at least one robot to travel that avoids interacting with an obstacle while completing the one or more tasks assigned to the at least one robot.

6. The method of claim 1, wherein addressing the defect comprises repairing or remediating the defect by applying a part via additive manufacturing, spraying a part of the asset, welding a part of the asset, replacing a part of the asset from an available inventory of parts, or any combination thereof.

7. A non-transitory, computer readable medium comprising instructions configured to be executed by a processor of a robotic system, wherein the instructions comprise instructions configured to cause the processor to:

generate a flight plan to monitor an asset for defects, wherein the flight plan comprises one or more tasks to be performed by at least one robot via at least one effector of the at least one robot;

execute the flight plan to acquire sensor data via at least one sensor of the at least one robot indicating one or more characteristics of the asset at a location on the asset;

determine whether a quality of the sensor data acquired at the location on the asset is below a threshold quality level and, in response to a determination that the quality of the sensor data is below the threshold quality level at the location;

dynamically adjust the flight plan to an updated flight plan to iteratively acquire additional sensor data indicative of the one or more characteristics of the asset at the location on the asset via the at least one sensor; and

determine, after each iterative acquisition of the additional sensor data, whether the quality of the sensor data combined with the additional sensor data acquired at the location on the asset is above the threshold quality level; and

generate a signal, encoding or conveying, instructions to control the at least one robot to acquire sensor data related to another location on the asset upon a determination that the quality of the sensor data combined with the additional sensor data is above the threshold quality level.

8. The non-transitory computer readable medium of claim 7, wherein the instructions are configured to cause the processor to dynamically adjust the flight plan to the updated flight plan by adjusting the one or more tasks or adding one or more additional tasks to the flight plan to cause the at least one robot to acquire the additional sensor data at the location on the asset.

9. The non-transitory computer readable medium of claim 7, wherein the instructions are configured to cause the processor to determine whether the quality of the sensor data is below the threshold quality level based on a signal-to-noise ratio of the sensor data.

10. The non-transitory computer readable medium of claim 7, comprising instructions configured to cause the processor to receive resource data indicative of available resources and to plan the one or more tasks based on the available resources.

11. A robotic system configured to monitor an asset, comprising:

at least one robot comprising at least one sensor capable of detecting one or more characteristics of the asset and at least one effector capable of performing a repair or maintenance operation on the asset; and

a processing system comprising at least one processor operatively coupled to at least one memory, wherein the at least one processor is configured to;

generate a flight plan to monitor the asset, wherein the flight plan comprises one or more tasks to be performed by the at least one robot;

execute the flight plan to acquire sensor data via the at least one sensor indicating the one or more characteristics of the asset at a location on the asset;

determine whether a quality of the sensor data acquired at the location on the asset is below a threshold quality level and, in response to a determination that the quality of the sensor data is below the threshold quality level at the location;

dynamically adjust the flight plan to an updated flight plan by adjusting or adding one or more tasks to the flight plan to iteratively acquire additional sensor data indicative of the one or more characteristics of the asset at the location on the asset via the at least one sensor; and

determine, after each iterative acquisition of the additional sensor data, whether the quality of the sensor data combined with the additional sensor data acquired at the location on the asset is above the threshold quality level; and

operate the at least one robot to acquire sensor data related to another location on the asset upon a determination that the quality of the sensor data combined with the additional sensor data is above the threshold quality level.

12. The robotic system of claim 11, wherein the processing system is configured to send a signal to the at least one robot to adjust a path traveled by the at least one robot during execution of the flight plan to acquire additional data related to a potential defect of the asset.

13. The robotic system of claim 11, wherein the processing system is configured to determine whether the quality of the sensor data is below the threshold quality level based on a signal-to-noise ratio of the sensor data.

14. The robotic system of claim 11, wherein the flight plan comprises an initial path along which the at least one robot is to travel when executed.

15. The robotic system of claim 11, wherein the one or more tasks comprise tasks to monitor locations on the asset associated with greater risk of failure relative to other locations on the asset at one or more of an increased frequency, an increased duration, or an increased resolution or sensing quality.

16. The robotic system of claim 11, wherein the processing system is configured to receive resource data indicative of available resources and to plan the one or more tasks to be preformed by the at least one robot based on the available resources.

17. The robotic system of claim 16, wherein the available resources comprise one or more of types of robots available, types of sensors available, or energy, fuel, or power available.

18. The robotic system of claim 11, wherein the processing system is configured to send one or more signals encoding or conveying instructions to control the at least one robot to acquire additional data related to a potential defect on the asset.