Using Laser Sensors to Augment Stereo Sensor Readings for Robotic Devices

• US 20170308086A1
• Filed: 07/06/2017
• Published: 10/26/2017
• Est. Priority Date: 08/17/2015
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: X Development LLC (Alphabet Inc.)
• Inventors: Watts, Kevin William, Konolige, Kurt
• Granted Patent: US 10,488,523 B2
• Time in Patent Office: Days
• CPC Class Codes: G01S 17/86 Combinations of lidar systems with systems other than lidar, radar or sonar, e.g. with direction finders
• CPC Class Codes: G01S 17/87 Combinations of systems using electromagnetic waves other than radio waves
• CPC Class Codes: G01S 17/931 of land vehicles
• CPC Class Codes: G05D 1/0248 in combination with a laser lasers per se H01S
• CPC Class Codes: G05D 1/0251 extracting 3D information from a plurality of images taken from different locations, e.g. stereo vision stereoscopic image analysis H04N13/00; depth recovery from images G06T7/593

Abstract

An example system includes one or more laser sensors on a robotic device, where the one or more laser sensors are configured to produce laser sensor data indicative of a first area within a first distance in front of the robotic device. The system further includes one or more stereo sensors on the robotic device, where the stereo sensors on the robotic device are configured to produce stereo sensor data indicative of a second area past a second distance in front of the robotic device. The system also includes a controller configured to receive the laser sensor data, receive the stereo sensor data, detect one or more objects in front of the robotic device based on at least one of the laser sensor data and the stereo sensor data, and provide instructions for the robotic device to navigate based on the one or more detected objects.

Claims

1. A system comprising:

a first stereo sensor coupled to a robotic device, the first stereo sensor comprising a first primary optical element and a secondary optical element;

a second stereo sensor coupled to the robotic device, the second stereo sensor comprising a second primary optical element and the secondary optical element; and

a controller configured to;

receive, from the first and the second stereo sensors, stereo sensor data indicative of a scanned area;

based on the stereo sensor data, detect an object in the scanned area; and

based on the detected object, provide navigation instructions to the robotic device.

2. The system of claim 1, wherein the first stereo sensor is configured to scan a first area within the scanned area, and wherein the second stereo sensor is configured to scan a second area within the scanned area.

3. The system of claim 2, wherein the first area and the second area do not overlap.

4. The system of claim 2, wherein the first primary optical element and the secondary optical element are positioned closer together than the second primary optical element and the secondary optical element.

5. The system of claim 4, wherein the first area is closer to the robotic device than the second area.

6. The system of claim 1, wherein the optical elements of the first stereo sensor are spaced in a narrow baseline arrangement, and wherein the optical elements of the second stereo sensor are spaced in a wide baseline arrangement.

7. The system of claim 1, wherein the first stereo sensor and the second stereo sensor are oriented parallel to a direction of travel of the robotic device.

8. The system of claim 1, wherein the scanned area starts at a distance from the robotic device at which a field of view of the first stereo sensor is at least as wide as the robotic device.

9. The system of claim 1, wherein the scanned area is a first scanned area and wherein the system further comprises:

a laser sensor coupled to the robotic device, wherein the laser sensor is configured to produce laser sensor data indicative of a second scanned area, and wherein the second scanned area is closer to the robotic device than the first scanned area.

10. The system of claim 9, wherein the plurality of laser sensors are spaced across the robotic device so that contiguous laser sensors are closer together than a predetermined minimum obstacle size.

11. A method comprising:

receiving stereo sensor data from a first stereo sensor and a second stereo sensor coupled to a robotic device,

wherein the first stereo sensor comprises a first primary optical element and a secondary optical element,

wherein the second stereo sensor comprises a second primary optical element and the secondary optical element, and

wherein the stereo sensor data is indicative of a scanned area;

monitoring the scanned area;

detecting an object in the scanned area; and

providing navigation instructions to the robotic device such that the robotic device avoids colliding with the object.

12. The method of claim 11, wherein receiving stereo sensor data from a first stereo sensor and a second stereo sensor comprises:

causing the first stereo sensor to scan a first area within the scanned area;

causing the second stereo sensor to scan a second area within the scanned area; and

receiving first sensor data from the first stereo sensor and second sensor data from the second stereo sensor, wherein the first sensor data is indicative of the first area, and wherein the second sensor data is indicative of the second area.

13. The method of claim 11, wherein the first primary optical element and the secondary optical element are positioned closer together than the second primary optical element and the secondary optical element.

14. The method of claim 11, wherein monitoring the scanned area comprises:

monitoring the scanned area for objects located in a path of the robotic device.

15. The method of claim 11, providing navigation instructions to the robotic device comprises:

determining a distance of the object from the robotic device;

based on the distance, determining a margin of error in the stereo sensor data; and

based on the distance and the margin of error, determining the navigation instructions.

16. A robotic device comprising:

at least one motion component configured to cause motion of the robotic device;

a first stereo sensor comprising a first primary optical element and a secondary optical element;

a second stereo sensor comprising a second primary optical element and the secondary optical element; and

a controller configured to;

receive, from the first and the second stereo sensors, stereo sensor data indicative of a scanned area in front of the robotic device;

based on the stereo sensor data, detect an object in the scanned area; and

control the motion component such that the robotic device avoids colliding with the detected object.

17. The robotic device of claim 16, wherein the optical elements of the first stereo sensor are spaced in a narrow baseline arrangement, and wherein the optical elements of the second stereo sensor are spaced in a wide baseline arrangement.

18. The robotic device of claim 16, further comprising:

a laser sensor configured to project respective narrow laser beams in front of the robotic device.

19. The robotic device of claim 18, wherein the scanned area is a first scanned area, and wherein the laser sensor is configured to produce laser sensor data indicative of a second scanned area different from the first scanned area.

20. The robotic device of claim 19, wherein the second scanned area is closer to the robotic device than the first scanned area.