System and method for generating three-dimensional robotic inspection plan

• US 10,521,960 B2
• Filed: 05/03/2017
• Issued: 12/31/2019
• Est. Priority Date: 05/03/2017
• Status: Active Grant

• Current Assignee: General Electric Company
• Original Assignee: General Electric Company
• Inventors: Gray, Steven, Sen, Shiraj, Baloch, Ghulam Ali, Castillo-Effen, Mauricio, Theurer, Charles
• Primary Examiner(s): Tseng, Charles
• Application Number:
  US15/585,502
• Publication Number:
  US 20180322699A1
• Time in Patent Office: 972 Days
• CPC Class Codes: G05D 1/0094 involving pointing a payload, e.g. camera, weapon, sensor, towards a fixed or moving target
• CPC Class Codes: G06T 19/003 Navigation within 3D models or images
• CPC Class Codes: G08G 5/0013 with a ground station
• CPC Class Codes: G08G 5/0034 Assembly of a flight plan
• CPC Class Codes: G08G 5/006 in accordance with predefined flight zones, e.g. to avoid prohibited zones
• CPC Class Codes: G08G 5/0069 specially adapted for an unmanned aircraft
• CPC Class Codes: Y10S 901/01 Mobile robot

Abstract

Provided are systems and methods for generating an autonomous 3D inspection plan for an unmanned robot. In an example, the method may include receiving a selection of a plurality of regions of interest with respect to a virtual asset displayed in virtual space, detecting a 3D position of the regions of interest within a coordinate frame of the virtual space, auto-generating a travel path about a physical asset corresponding to the virtual asset by generating a virtual 3D travel path with respect to the virtual asset based on the detected 3D positions of the selected regions of interest within the coordinate frame, aligning the virtual 3D travel path in the virtual space with a physical travel path in a physical space, and outputting a robotic inspection plan comprising the auto-generated physical travel path for the unmanned robot.

Claims

1. A computing system, comprising:

a storage configured to store a plurality of regions of interest selected on a surface of a digital representation displayed in a virtual space, wherein the digital representation corresponds to a physical asset in a physical space, and to store a three-dimensional (3D) position of each of the plurality of regions of interest within a coordinate frame of the virtual space;

a processor configured to;

detect a virtual viewing angle from which a user interface is displaying a respective region of interest of the plurality of regions of interest;

determine a viewing vector that represents the virtual viewing angle from which the user interface is displaying the respective region of interest on the digital representation in the virtual space;

generate a virtual 3D travel path about the digital representation based on the 3D positions of the plurality of regions of interest within the coordinate frame of the virtual space;

align the virtual 3D travel path in the virtual space with the physical asset in the physical space to generate a physical travel path in the physical space;

convert respective viewing vectors associated with the plurality of regions of interest into corresponding physical viewing angles at which an unmanned robot is to be oriented relative to the physical asset along the physical travel path in the physical space; and

an output configured to output a robotic inspection plan for the unmanned robot comprising the physical travel path about the physical asset.

2. The computing system of claim 1, wherein the processor is configured to align the coordinate frame of the virtual space with a position associated with the physical asset in the physical space based on Global Positioning System (GPS) coordinates.

3. The computing system of claim 1, wherein the processor is further configured to:

receive, from the user interface, a selection of a view primitive for the respective region of interest on the digital representation; and

associate the view primitive with an amount of rotation that the unmanned robot is to travel about the physical asset in the physical space.

4. The computing system of claim 1, wherein the processor is further configured to:

receive, via the user interface, a selection of a desired resolution for the respective region of interest on the digital representation; and

determine a distance between the physical travel path and the physical asset in the physical space and store the distance in the robotic inspection plan to be performed by the unmanned robot.

5. The computing system of claim 1, wherein the virtual 3D travel path comprises a dynamically selected virtual start point comprising a first vertical location and a first lateral location in the virtual space, and a dynamically selected virtual end point comprising a second vertical location and a second lateral location in the virtual space which differs from the first vertical location and the first lateral location, respectively.

6. The computing system of claim 1, wherein the virtual 3D travel path comprises a virtual line between at least two of the plurality of regions of interest, wherein lateral and vertical positions of the virtual 3D travel path simultaneously and continuously change along the virtual line between the at least two of the plurality of regions of interest.

7. The computing system of claim 1, wherein the virtual 3D travel path is generated by the processor based on identification of a shortest path that extends through each 3D position of the plurality of regions of interest in the virtual space.

8. A method for generating a robotic inspection plan that is to be performed by an unmanned robot, the method comprising:

receiving, via a user interface, a selection of a plurality of regions of interest on a surface of a digital representation of a physical asset in a physical space and displayed in a virtual space;

detecting a three-dimensional (3D) position of each of the plurality of regions of interest within a coordinate frame of the virtual space;

detecting a virtual viewing angle from which the user interface is displaying a respective region of interest of the plurality of regions of interest;

determining a viewing vector that represents the virtual viewing angle from which the user interface is displaying the respective region of interest on the digital representation in the virtual space;

generating a virtual 3D travel path about the digital representation based on the 3D positions of the plurality of regions of interest within the coordinate frame of the virtual space;

aligning the virtual 3D travel path in the virtual space with the physical asset in the physical space to generate a physical travel path in the physical space;

converting respective viewing vectors associated with the plurality of regions of interest into corresponding physical viewing angles at which the unmanned robot is to be oriented relative to the physical asset along the physical travel path in the physical space; and

outputting the robotic inspection plan for the unmanned robot comprising the physical travel path about the physical asset.

9. The method of claim 8, wherein aligning the virtual 3D travel path comprises aligning the coordinate frame of the virtual space with a position associated with the physical asset in the physical space based on Global Positioning System (GPS) coordinates.

10. The method of claim 8, further comprising:

receiving, from the user interface a selection of a view primitive for the respective region of interest on the digital representation; and

associating the view primitive with an amount of rotation that the unmanned robot is to travel about the physical asset in the physical space.

11. The method of claim 8, further comprising:

receiving, via the user interface, a selection of a desired resolution for the respective region of interest; and

determining a distance between the physical travel path and the physical asset in physical space and storing the distance in the robotic inspection plan to be performed by the unmanned robot.

12. The method of claim 8, wherein the virtual 3D travel path comprises a dynamically selected virtual start point comprising a first vertical location and a first lateral location in the virtual space, and a dynamically selected virtual end point comprising a second vertical location and a second lateral location in the virtual space which differs from the first vertical location and the first lateral location, respectively.

13. The method of claim 8, wherein the virtual 3D travel path comprises a virtual line between at least two of the plurality of regions of interest, wherein lateral and vertical positions of the virtual 3D travel path simultaneously and continuously change along the virtual line between the at least two of the plurality of regions of interest.

14. The method of claim 8, wherein the virtual 3D travel path is generated based on identification of a shortest path that extends through each of the detected 3D positions of the plurality of selected regions of interest in the virtual space.

15. A non-transitory computer readable medium having stored therein instructions that, when executed by a processor, cause the processor to perform a method for generating a robotic inspection plan that is to be performed by an unmanned robot, the method comprising:

receiving, via a user interface, a selection of a plurality of regions of interest on a surface of a digital representation of a physical asset in a physical space and displayed in a virtual space;

detecting a three-dimensional (3D) position of each of the plurality of regions of interest within a coordinate frame of the virtual space;

detecting a virtual viewing angle from which the user interface is displaying a respective region of interest of the plurality of regions of interest;

determining a viewing vector that represents the virtual viewing angle from which the user interface is displaying the respective region of interest on the digital representation in the virtual space;

generating a virtual 3D travel path about the digital representation based on the detected 3D positions of the plurality of regions of interest within the coordinate frame of the virtual space;

aligning the virtual 3D travel path in the virtual space with the physical asset in the physical space to generate a physical travel path in the physical space;

converting respective viewing vectors associated with the plurality of regions of interest into corresponding physical viewing angles at which the unmanned robot is to be oriented relative to the physical asset along the physical travel path in the physical space; and

outputting the robotic inspection plan for the unmanned robot comprising the physical travel path about the physical asset.