Solid object detection system using laser and radar sensor fusion

• US 9,097,800 B1
• Filed: 10/11/2012
• Issued: 08/04/2015
• Est. Priority Date: 10/11/2012
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Zhu, Jiajun
• Primary Examiner(s): To, Tuan C.
• Assistant Examiner(s): Hilgendorf, Dale W
• Application Number:
  US13/649,384
• Time in Patent Office: 1,027 Days
• Field of Search: 701/23, 701/24, 701/26, 701/27, 701/28
• US Class Current: 1/1
• CPC Class Codes: G01S 13/426 Scanning radar, e.g. 3D radar G01S13/66 takes precedence
• CPC Class Codes: G01S 13/865 Combination of radar systems with lidar systems
• CPC Class Codes: G01S 13/867 Combination of radar systems with cameras
• CPC Class Codes: G01S 13/931 of land vehicles
• CPC Class Codes: G01S 17/42 Simultaneous measurement of distance and other co-ordinates indirect measurement G01S17/46
• CPC Class Codes: G01S 17/89 for mapping or imaging
• CPC Class Codes: G01S 17/931 of land vehicles
• CPC Class Codes: G01S 2013/9318 Controlling the steering
• CPC Class Codes: G01S 2013/93185 Controlling the brakes
• CPC Class Codes: G01S 2013/9319 Controlling the accelerator
• CPC Class Codes: G01S 7/411 Identification of targets based on measurements of radar reflectivity G01S7/415 takes precedence
• CPC Class Codes: G01S 7/4802 using analysis of echo signal for target characterisation; Target signature; Target cross-section

Abstract

A light detection and ranging device associated with an autonomous vehicle scans through a scanning zone while emitting light pulses and receives reflected signals corresponding to the light pulses. The reflected signals indicate a three-dimensional point map of the distribution of reflective points in the scanning zone. A radio detection and ranging device scans a region of the scanning zone corresponding to a reflective feature indicated by the three-dimensional point map. Solid objects are distinguished from non-solid reflective features on the basis of a reflected radio signal that corresponds to the reflective feature. Positions of features indicated by the reflected radio signals are projected to estimated positions during the scan with the light detection and ranging device according to relative motion of the radio-reflective features indicated by a frequency shift in the reflected radio signals.

Claims

1. A method comprising:

scanning a light detection and ranging (LIDAR) device associated with an autonomous vehicle through a scanning zone while emitting light pulses;

determining a three-dimensional (3-D) point map of the scanning zone based on time delays between emitting the light pulses and receiving corresponding returning reflected signals and based on orientations of the LIDAR device while emitting the light pulses;

scanning, with a radio detection and ranging (RADAR) device associated with the autonomous vehicle, one or more regions of the scanning zone corresponding to one or more light-reflective features indicated by the 3-D point map, to identify one or more radio-reflective features;

determining a temporal offset between the scanning with the LIDAR device and the scanning with the RADAR device;

associating the one or more radio-reflective features with respective positions in the 3-D point map so as to account for a spatial displacement of the one or more radio-reflective features with respect to the 3-D point map during the temporal offset;

determining whether the one or more light-reflective features include a solid material by, for each of the one or more light-reflective features;

responsive to determining that the light-reflective feature is at a position in the 3-D point map that is associated with the radio-reflective feature, determining that the light-reflective feature includes a solid material; and

responsive to determining that the light-reflective feature is at a position in the 3-D point map that is not associated with the radio-reflective feature, determining that the light-reflective feature does not include a solid material; and

controlling the autonomous vehicle to avoid light-reflective features determined to include a solid material.

2. The method according to claim 1, further comprising:

determining a frequency shift of one or more reflected radio signals indicating a relative motion of the one or more radio-reflective features along a line of sight of the RADAR device; and

estimating a position of the one or more radio-reflective features during the scanning with the LIDAR device by projecting the one or more radio-reflective features along the line of sight of the RADAR device according to the indicated relative motion and the temporal offset between the scanning with the LIDAR device and the scanning with the RADAR device.

3. The method according to claim 2, wherein the scanning with the RADAR device precedes the scanning with the LIDAR device such that the estimated position of the one or more radio-reflective features is projected along the line of sight of the RADAR device in a direction of the indicated relative motion of the one or more radio-reflective features.

4. The method according to claim 2, wherein the scanning with the LIDAR device precedes the scanning with the RADAR device such that the estimated position of the one or more radio-reflective features is projected along the line of sight of the RADAR device in an opposite direction of the indicated relative motion of the one or more radio-reflective features.

5. The method according to claim 1, further comprising:

scanning the scanning zone with the RADAR device; and

associating one or more radio-reflective features indicated by reflected radio signals with estimated positions during the scanning with the LIDAR based on time delays and frequency shifts of the reflected radio signals, wherein the scanning with the LIDAR device is temporally offset from the scanning with the RADAR device.

6. The method according to claim 1, further comprising:

combining information from the LIDAR device and the RADAR device to generate a map of solid objects in the scanning zone, wherein light-reflective features indicated by the 3-D point map determined to not include a solid material are not included in the map of solid objects.

7. The method according to claim 6, further comprising:

analyzing the positions of the solid objects in the map of solid objects to identify obstacles relative to a current path of the autonomous vehicle;

determining a modified path of the autonomous vehicle that avoids the identified obstacles; and

controlling the autonomous vehicle to navigate along the modified path.

8. The method according to claim 1, further comprising:

identifying the region of the scanning zone for scanning with the RADAR device responsive to associating a group of points included in the light-reflective feature with a spatially discontinuous distribution indicative of reflection from a dispersed aerosol.

9. The method according to claim 1, further comprising:

identifying the region of the scanning zone for scanning with the RADAR device responsive to associating a group of points included in the light-reflective feature with splashing water or an exhaust plume.

10. An autonomous vehicle system comprising:

a light detection and ranging (LIDAR) device including;

a light source configured to be scanned through a scanning zone while emitting light pulses; and

a light detector configured to receive returning reflected signals from light-reflective features in the scanning zone, if any;

a radio detection and ranging (RADAR) device including;

an illuminating radio antenna configured to be scanned through the scanning zone; and

a radio receiver configured to detect reflected radio signals from radio-reflective features in the scanning zone, if any; and

a controller configured to;

instruct the LIDAR device to scan the scanning zone while emitting light pulses;

receive information from the LIDAR device indicative of the time delays between emission of the light pulses and reception of the corresponding returning reflected signals;

determine, based on the time delays and orientations of the LIDAR device while emitting the light pulses, a three dimensional (3-D) point map;

instruct the RADAR device to scan one or more regions of the scanning zone corresponding to one or more light-reflective features indicated by the 3-D point map, to identify one or more radio reflective features;

determining a temporal offset between the scanning with the LIDAR device and the scanning with the RADAR device;

associating the one or more radio-reflective features with respective positions in the 3-D point map so as to account for a spatial displacement of the one or more radio-reflective features with respect to the 3-D point map during the temporal offset;

determine whether the one or more light-reflective features include a solid material by, for each of the one or more light-reflective features;

responsive to determining that the light-reflective feature is at a position in the 3-D point map that is associated with the radio-reflective feature, determine that the light-reflective feature includes a solid material; and

responsive to determining that the light-reflective feature is at a position in the 3-D point map that is not associated with the radio-reflective feature, determine that the light-reflective feature does not include a solid material; and

control the autonomous vehicle to avoid light-reflective features determined to include a solid material.

11. The autonomous vehicle system according to claim 10, wherein the controller is further configured to:

determine a frequency shift of one or more reflected radio signals indicating a relative motion of the one or more radio-reflective features along a line of sight of the RADAR device; and

estimate a position of the one or more radio-reflective features during the scan with the LIDAR device by projecting the one or more radio-reflective features along the line of sight of the RADAR device according to the indicated relative motion and the temporal offset between the scan with the LIDAR device and the scan with the RADAR device.

12. The autonomous vehicle system according to claim 10, wherein the controller is further configured to:

instruct the RADAR device to scan the scanning zone; and

associate one or more radio-reflective features indicated by reflected radio signals with estimated positions during the scan with the LIDAR based on time delays and frequency shifts of the reflected radio signals, wherein the scan with the LIDAR device is temporally offset from the scan with the RADAR device.

13. The autonomous vehicle system according to claim 10, wherein the controller is further configured to:

combine information from the LIDAR device and the RADAR device to generate a map of solid objects in the scanning zone, wherein light-reflective features indicated by the 3-D point map determined to not include a solid material are not included in the map of solid objects.

14. The autonomous vehicle system according to claim 13, wherein the controller is further configured to:

analyze the positions of the solid objects in the map of solid objects to identify obstacles relative to a current path of the autonomous vehicle;

determine a modified path of the autonomous vehicle that avoids the identified obstacles; and

control the autonomous vehicle to navigate along the modified path.

15. The autonomous vehicle system according to claim 10, wherein the controller is further configured to:

identify the region of the scanning zone for scanning with the RADAR device responsive to associating a group of points included in the light-reflective feature with a spatially discontinuous distribution indicative of reflection from a dispersed aerosol.

16. A non-transitory computer readable medium storing instructions that, when executed by one or more processors in a computing device, cause the computing device to perform operations, the operations comprising:

scanning a light detection and ranging (LIDAR) device associated with an autonomous vehicle through a scanning zone while emitting light pulses;

determining a three-dimensional (3-D) point map of the scanning zone based on time delays between emitting the light pulses and receiving corresponding returning reflected signals and based on orientations of the LIDAR device while emitting the light pulses;

scanning, with a radio detection and ranging (RADAR) device associated with the autonomous vehicle, one or more regions of the scanning zone corresponding to one or more light-reflective features indicated by the 3-D point map, to identify one or more radio-reflective features;

determining a temporal offset between the scanning with the LIDAR device and the scanning with the RADAR device;

associating the one or more radio-reflective features with respective positions in the 3-D point map so as to account for a spatial displacement of the one or more radio-reflective features with respect to the 3-D point map during the temporal offset;

determining whether the one or more light-reflective features include a solid material by, for each of the one or more light-reflective features;

responsive to determining that the light-reflective feature is at a position in the 3-D point map that is associated with the radio-reflective feature, determining that the light-reflective feature includes a solid material; and

responsive to determining that the light-reflective feature is at a position in the 3-D point map that is not associated with the radio-reflective feature, determining that the light-reflective feature does not include a solid material; and

controlling the autonomous vehicle to avoid light-reflective features determined to include a solid material.

17. The non-transitory computer readable medium according to claim 16, wherein

associating the one or more radio-reflective features with respective positions in the 3-D point map is further based on a frequency shift associated with a reflected radio signal indicating the radio-reflective feature.

18. The non-transitory computer readable medium according to claim 16, wherein the operations further comprise:

combining information from the LIDAR device and the RADAR device to generate a map of solid objects in the scanning zone, wherein light-reflective features indicated by the 3-D point map determined to not include a solid material are not included in the map of solid objects;

analyzing the positions of the solid objects in the map of solid objects to identify obstacles relative to a current path of the autonomous vehicle;

determining a modified path of the autonomous vehicle that avoids the identified obstacles; and

controlling the autonomous vehicle to navigate along the modified path.