Automatically scanning and representing an environment with collision avoidance

• US 10,341,639 B2
• Filed: 11/16/2015
• Issued: 07/02/2019
• Est. Priority Date: 11/16/2015
• Status: Active Grant

• Current Assignee: ABB Schweiz AG (ABB Limited)
• Original Assignee: ABB Schweiz AG (ABB Limited)
• Inventors: Wang, Jianjun, Zhang, Biao, Martinez, Carlos, Morato, Carlos W., Boca, Remus, Fuhlbrigge, Thomas
• Primary Examiner(s): Uhl, Lindsay J
• Application Number:
  US14/941,838
• Publication Number:
  US 20170138192A1
• Time in Patent Office: 1,324 Days
• Field of Search: None
• CPC Class Codes: E21B 7/022 Control of the drilling operation; Hydraulic or pneumatic means for activation or operation control circuits for drilling masts E21B15/045
• CPC Class Codes: E21D 9/003 Arrangement of measuring or indicating devices for use during driving of tunnels, e.g. for guiding machines
• CPC Class Codes: E21D 9/006 by making use of blasting methods explosive compositions C06B; blasting per se F42
• CPC Class Codes: G01C 7/06 of cavities, e.g. tunnels
• CPC Class Codes: G05D 1/0248 in combination with a laser lasers per se H01S
• CPC Class Codes: H04N 13/204 using stereoscopic image cameras stereoscopic photography G03B35/00

Abstract

Automatic scanning and representing an environment with collision avoidance includes, for example, obtaining a first representation of the environment using a first scanning path, determining a second scanning path based on the first representation of the environment operable to avoid contact with the environment when obtaining a second representation of the environment, obtaining the second representation of the environment based on the second scanning path, and wherein the second representation of the environment is different from the first representation of the environment. The method may be employed in imaging and/or representing a rock wall having a plurality of spaced-apart holes for receiving charges for mining.

Claims

1. A computer-implemented method comprising:

obtaining a first representation of an environment employing a 3D camera coupled to a robotic arm based on a first scanning path;

simulating a second scanning path of the 3D camera comprising movements of the robotic arm using the first representation of the environment;

determining the 3D camera cannot reach a portion of the second scanning path so as to complete the second scanning path in response to simulating the second scanning path;

adjusting the second scanning path in response to determining the robotic arm cannot reach the portion of the second scanning path;

obtaining a second representation of the environment employing the 3D camera based on the adjusted second scanning path; and

wherein the second representation of the environment is different from the first representation of the environment.

2. The computer-implemented method of claim 1 wherein the obtaining the first representation comprises obtaining the first representation of the environment using the first scanning path about a first region, and obtaining the second representation comprises obtaining the second representation of the environment using the second scanning path about a second region, and wherein the second region is greater than the first region.

3. The computer-implemented method of claim 1 wherein the obtaining the first representation comprises obtaining the first representation of the environment at a first distance from the environment, at a first resolution, or at a first scanning speed, and the obtaining the second representation comprises obtaining the second representation of the environment at a second distance from the environment different from the first distance, at a second resolution different from the first resolution, or at a second scanning speed different from the first scanning speed.

4. The computer-implemented method of claim 1 comprising determining the second scanning path based on the first representation of the environment operable to avoid contact with the environment when obtaining a second representation of the environment, wherein the determining the second scanning path comprises determining the second scanning path off site from the obtaining the first representation and the obtaining the second representation, and wherein the first representation includes a point cloud generated from an output of the 3D camera.

5. The computer-implemented method of claim 4 wherein the determining the second scanning path comprises simulating the second scanning path to confirm avoidance of contact with the environment.

6. The computer-implemented method of claim 1 wherein obtaining the first representation comprises obtaining a first plurality of frames of data of a plurality of portions of the environment, and merging together data from the first plurality of frames of data the plurality of portions of the environment.

7. The computer-implemented method of claim 4 wherein the determining the second scanning path comprises using a bounding box or a collision free path planner.

8. The computer-implemented method of claim 1 wherein the environment comprises a plurality of features, and further comprising determining an aspect of the plurality of features.

9. The computer-implemented method of claim 8 further comprising determining the second scanning path to avoid contact with the environment based on a predetermined map of the features.

10. The computer-implemented method of claim 1 wherein the 3D camera includes a robotically supported movable imager.

11. The computer-implemented method of claim 1 wherein the environment comprises a rock wall having a plurality of holes defining axes for receiving charges for mining, and the rock wall surrounded by sidewalls, a ceiling, and/or a ground.

12. The computer-implemented method of claim 11 wherein the determining the second scanning path is further based on a predetermined drill hole map of the plurality of holes.

13. A system comprising:

a memory; and

a physical processor in communications with the memory, wherein the system is configured to perform a method comprising;

obtaining a first representation of an environment employing a 3D camera coupled to a robotic arm based on a first scanning path by receiving a set of frames from the 3D camera and generating a point cloud using the set of frames;

determining a second scanning path of the 3D camera based on the first representation of the environment operable to avoid contact when obtaining a second representation of the environment;

simulating the second scanning path comprising robot arm movements;

determining the 3D camera cannot reach a portion of the second scanning path so as to complete the second scanning path in response to simulating the second scanning path;

generating a third scanning path in response to determining the robotic arm cannot reach the portion of the second scanning path;

obtaining a second representation of the environment employing the 3D camera based on the third scanning path; and

wherein the second representation of the environment is different from the first representation of the environment.

14. A non-transitory computer readable storage medium readable by a processor comprising:

instructions stored on the non-transitory computer readable storage medium for execution by the processor for performing a method comprising;

obtaining a first representation of an environment employing a 3D camera coupled to a robotic arm based on a first scanning path by merging a set of point clouds generated by the 3D camera;

determining a second scanning path based on the first representation of the environment operable to avoid contact with the environment when obtaining a second representation of the environment;

simulating the second scanning path including a plurality of robot arm movements;

determining the robot arm cannot reach a portion of the second scanning path so as to complete the second scanning path in response to simulating the second scanning path;

generating a third scanning path of the 3D camera in response to determining the robot arm cannot reach the portion of the second scanning path;

obtaining a second representation of the environment based on the third scanning path; and

wherein the second representation of the environment is different from the first representation of the environment.