Characterizing optically reflective features via hyper-spectral sensor

• US 9,958,379 B1
• Filed: 12/01/2015
• Issued: 05/01/2018
• Est. Priority Date: 09/27/2012
• Status: Active Grant

• Current Assignee: Waymo LLC (Alphabet Inc.)
• Original Assignee: Waymo LLC (Alphabet Inc.)
• Inventors: Zhu, Jiajun, Ferguson, David I., Dolgov, Dmitri A., Dowdall, Jonathan Baldwin
• Primary Examiner(s): Woldemaryam, Assres H
• Application Number:
  US14/955,186
• Time in Patent Office: 882 Days
• Field of Search: 356 501- 515, 356 301- 316, 356 401- 41
• CPC Class Codes: F16L 55/00 Devices or appurtenances for use in, or in connection with, pipes or pipe systems the preceding groups and groups F16L57/00, F16L59/00 take precedence; repairing or joining pipes on or under water F16L1/26; nozzles B05B; cleaning of pipes B08B9/02; arrangements of draining devices for water main or service pipe systems E03B7/08; devices for preventing bursting of water pipes by freezing E03B7/10; draining devices for hydrants E03B9/14; for domestic plumbing installations E03C1/00; steam traps for draining of liquids from enclosures containing gases or vapours F16T; arrangements for sealing leaky tubes or conduits of heat-exchangers F28F11/00
• CPC Class Codes: G01C 3/08 Use of electric radiation detectors
• CPC Class Codes: G01N 21/27 using photo-electric detection G01N21/31 takes precedence; circuits for computing concentration logarithmic circuits G06G7/24; photometric circuits in general G01J
• CPC Class Codes: G01S 17/08 for measuring distance only indirect measurement G01S17/46; active triangulation systems G01S17/48
• CPC Class Codes: G01S 17/10 using transmission of interrupted, pulse-modulated waves determination of distance by phase measurements G01S17/32
• CPC Class Codes: G01S 17/42 Simultaneous measurement of distance and other co-ordinates indirect measurement G01S17/46
• CPC Class Codes: G01S 17/86 Combinations of lidar systems with systems other than lidar, radar or sonar, e.g. with direction finders
• CPC Class Codes: G01S 17/89 for mapping or imaging
• CPC Class Codes: G01S 17/931 of land vehicles
• CPC Class Codes: G01S 7/4808 Evaluating distance, position or velocity data
• CPC Class Codes: G01S 7/497 Means for monitoring or calibrating
• CPC Class Codes: G05D 1/0088 characterized by the autonomous decision making process, e.g. artificial intelligence, predefined behaviours using knowledge based models G06N5/00

Abstract

A light detection and ranging device associated with an autonomous vehicle scans through a scanning zone while emitting light pulses and receives reflected signals corresponding to the light pulses. The reflected signals indicate a three-dimensional point map of the distribution of reflective points in the scanning zone. A hyperspectral sensor images a region of the scanning zone corresponding to a reflective feature indicated by the three-dimensional point map. The output from the hyperspectral sensor includes spectral information characterizing a spectral distribution of radiation received from the reflective feature. The spectral characteristics of the reflective feature allow for distinguishing solid objects from non-solid reflective features, and a map of solid objects is provided to inform real time navigation decisions.

Claims

1. A method comprising:

causing a light detection and ranging (LIDAR) device associated with an autonomous vehicle to scan through a scanning zone while emitting light pulses;

determining a three-dimensional (3-D) point map of the scanning zone based on time delays between emitting the light pulses and receiving corresponding returning reflected signals and based on orientations of the LIDAR device while emitting the light pulses;

identifying a region of the scanning zone for spectral analysis based on the 3-D point map including a particular reflective feature that resembles a type of non-solid object;

imaging, with a hyperspectral sensor associated with the autonomous vehicle, the identified region of the scanning zone so as to generate spectral information characterizing a spectral distribution of radiation received from the particular reflective feature;

correlating the particular reflective feature with a non-solid material based on the spectral distribution of radiation received from the particular reflective feature including one or more spectral characteristics of the non-solid material;

in response to the correlating, determining that the particular reflective feature corresponds to a non-solid object;

combining information from the LIDAR device and the hyperspectral sensor, wherein the hyperspectral sensor is separate from the LIDAR device, and wherein the combined information indicates whether reflective features in the 3-D point map correspond to solid objects or non-solid objects;

analyzing the combined information to identify obstacles relative to a current path of the autonomous vehicle;

determining a modified path of the autonomous vehicle that avoids the identified obstacles; and

controlling the autonomous vehicle to navigate along the modified path.

2. The method according to claim 1, wherein the particular reflective feature resembles an exhaust plume, and wherein the non-solid material correlated with the particular reflective feature comprises carbon monoxide.

3. The method according to claim 1, wherein the particular reflective feature resembles a water spray pattern, and wherein the non-solid material correlated with the particular reflective feature comprises water or water vapor.

4. The method according to claim 1, wherein the hyperspectral sensor is configured to detect radiation with wavelengths between 400 nm and 2500 nm.

5. The method according to claim 1, wherein the LIDAR device is configured to complete a scan of the entire scanning zone during a scanning interval, and wherein the imaging the region is initiated during the scanning interval based on information indicative of returning reflected signals received during one or more previous scanning intervals.

12. The method according to claim 1, wherein the imaging with the hyperspectral sensor occurs in response to identifying the region of the scanning zone for spectral analysis.

13. The method according to claim 1, wherein the hyperspectral sensor outputs a pixelated image in which each pixel includes intensity values for multiple spectral bands.

14. The method according to claim 13, further comprising correlating a location of the particular reflective feature in the 3-D point map with a corresponding location of the particular reflective feature in the pixelated image.

6. An autonomous vehicle system comprising:

a light detection and ranging (LIDAR) device including;

a light source configured to be scanned through a scanning zone while emitting light pulses; and

a light detector configured to receive returning reflected signals from reflective features in the scanning zone, if any;

a hyperspectral sensor configured to image selected regions of the scanning zone and to characterize spectral distributions of radiation received from the imaged regions, wherein the hyperspectral sensor is separate from the LIDAR device; and

a controller configured to;

cause the LIDAR device to scan the scanning zone while emitting light pulses;

receive information from the LIDAR device indicative of the time delays between the emission of the light pulses and the reception of corresponding returning reflected signals;

determine, based on the time delays and orientations of the LIDAR device while emitting the light pulses, a three dimensional (3-D) point map;

identify a region of the scanning zone for spectral analysis based on the 3-D point map including a particular reflective feature that resembles a type of non-solid object;

cause the hyperspectral sensor to image the identified region of the scanning zone;

receive, from the hyperspectral sensor, spectral information characterizing a spectral distribution of radiation received from the particular reflective feature;

correlate the particular reflective feature with a non-solid material based on the spectral distribution of radiation received from the particular reflective feature including one or more spectral characteristics of the non-solid material;

in response to the correlating, determining that the particular reflective feature corresponds to a non-solid object;

combining information from the LIDAR device and the hyperspectral sensor, wherein the combined information indicates whether reflective features in the 3-D point map correspond to solid objects or non-solid objects;

analyze the combined information to identify obstacles relative to a current path of the autonomous vehicle;

determine a modified path of the autonomous vehicle that avoids the identified obstacles; and

control the autonomous vehicle to navigate along the modified path.

7. The autonomous vehicle system according to claim 6, wherein the particular reflective feature resembles an exhaust plume, and wherein the non-solid material correlated with the particular reflective feature comprises carbon monoxide.

8. The autonomous vehicle system according to claim 6, wherein the particular reflective feature resembles a water spray pattern, and wherein the non-solid material correlated with the particular reflective feature comprises water or water vapor.

15. The autonomous vehicle system according to claim 6, wherein the controller is configured to cause the hyperspectral sensor to image the identified region of the scanning zone in response to identifying the region of the scanning zone for spectral analysis.

16. The autonomous vehicle system according to claim 6, wherein the hyperspectral sensor outputs a pixelated image in which each pixel includes intensity values for multiple spectral bands.

17. The autonomous vehicle system according to claim 16, wherein the controller is further configured to correlate a location of the particular reflective feature in the 3-D point map with a corresponding location of the particular reflective feature in the pixelated image.

9. A computer readable medium storing instructions that, when executed by one or more processors in a computing device, cause the computing device to perform operations, the operations comprising:

scanning a light detection and ranging (LIDAR) device associated with an autonomous vehicle through a scanning zone while emitting light pulses;

determining a three-dimensional (3-D) point map of the scanning zone based on time delays between emitting the light pulses and receiving corresponding returning reflected signals and based on orientations of the LIDAR device while emitting the light pulses;

identifying a region of the scanning zone for spectral analysis based on the 3-D point map including a particular reflective feature that resembles a type of non-solid object;

imaging, with a hyperspectral sensor associated with the autonomous vehicle, the identified region of the scanning zone so as to generate spectral information characterizing a spectral distribution of radiation received from the particular reflective feature;

correlating the particular reflective feature with a non-solid material based on the spectral distribution of radiation received from particular reflective feature including one or more spectral characteristics of the non-solid material;

in response to the correlating, determining that the particular reflective feature corresponds to a non-solid object;

combining information from the LIDAR device and the hyperspectral sensor, wherein the hyperspectral sensor is separate from the LIDAR device, and wherein the combined information indicates whether reflective features in the 3-D point map correspond to solid objects or non-solid objects;

analyzing the combined information to identify obstacles relative to a current path of the autonomous vehicle;

determining a modified path of the autonomous vehicle that avoids the identified obstacles; and

controlling the autonomous vehicle to navigate along the modified path.

10. The computer readable medium according to claim 9, wherein the particular reflective feature resembles an exhaust plume, and wherein the non-solid material correlated with the particular reflective feature comprises carbon monoxide.

11. The computer readable medium according to claim 9, wherein the particular reflective feature resembles a water spray pattern, and wherein the non-solid material correlated with the particular reflective feature comprises water or water vapor.

18. The computer readable medium according to claim 9, wherein the imaging with the hyperspectral sensor occurs in response to identifying the region of the scanning zone for spectral analysis.

19. The computer readable medium according to claim 9, wherein the hyperspectral sensor outputs a pixelated image in which each pixel includes intensity values for multiple spectral bands.

20. The computer readable medium according to claim 19, wherein the operations further comprise correlating a location of the particular reflective feature in the 3-D point map with a corresponding location of the particular reflective feature in the pixelated image.