Characterizing optically reflective features via hyper-spectral sensor

• US 9,234,618 B1
• Filed: 09/27/2012
• Issued: 01/12/2016
• Est. Priority Date: 09/27/2012
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Zhu, Jiajun, Ferguson, David I., Dolgov, Dmitri A., Dowdall, Jonathan Baldwin
• Primary Examiner(s): Ratcliffe, Luke
• Assistant Examiner(s): Woldemaryam, Assres H
• Application Number:
  US13/629,378
• Time in Patent Office: 1,202 Days
• Field of Search: 356 501- 515, 356 301- 316, 356 401- 41
• US Class Current: 1/1
• CPC Class Codes: F16L 55/00 Devices or appurtenances for use in, or in connection with, pipes or pipe systems the preceding groups and groups F16L57/00, F16L59/00 take precedence; repairing or joining pipes on or under water F16L1/26; nozzles B05B; cleaning of pipes B08B9/02; arrangements of draining devices for water main or service pipe systems E03B7/08; devices for preventing bursting of water pipes by freezing E03B7/10; draining devices for hydrants E03B9/14; for domestic plumbing installations E03C1/00; steam traps for draining of liquids from enclosures containing gases or vapours F16T; arrangements for sealing leaky tubes or conduits of heat-exchangers F28F11/00
• CPC Class Codes: G01C 3/08 Use of electric radiation detectors
• CPC Class Codes: G01N 21/27 using photo-electric detection G01N21/31 takes precedence; circuits for computing concentration logarithmic circuits G06G7/24; photometric circuits in general G01J
• CPC Class Codes: G01S 17/08 for measuring distance only indirect measurement G01S17/46; active triangulation systems G01S17/48
• CPC Class Codes: G01S 17/10 using transmission of interrupted, pulse-modulated waves determination of distance by phase measurements G01S17/32
• CPC Class Codes: G01S 17/42 Simultaneous measurement of distance and other co-ordinates indirect measurement G01S17/46
• CPC Class Codes: G01S 17/86 Combinations of lidar systems with systems other than lidar, radar or sonar, e.g. with direction finders
• CPC Class Codes: G01S 17/89 for mapping or imaging
• CPC Class Codes: G01S 17/931 of land vehicles
• CPC Class Codes: G01S 7/4808 Evaluating distance, position or velocity data
• CPC Class Codes: G01S 7/497 Means for monitoring or calibrating
• CPC Class Codes: G05D 1/0088 characterized by the autonomous decision making process, e.g. artificial intelligence, predefined behaviours using knowledge based models G06N5/00

Abstract

A light detection and ranging device associated with an autonomous vehicle scans through a scanning zone while emitting light pulses and receives reflected signals corresponding to the light pulses. The reflected signals indicate a three-dimensional point map of the distribution of reflective points in the scanning zone. A hyperspectral sensor images a region of the scanning zone corresponding to a reflective feature indicated by the three-dimensional point map. The output from the hyperspectral sensor includes spectral information characterizing a spectral distribution of radiation received from the reflective feature. The spectral characteristics of the reflective feature allow for distinguishing solid objects from non-solid reflective features, and a map of solid objects is provided to inform real time navigation decisions.

Claims

1. A method comprising:

causing a light detection and ranging (LIDAR) device associated with an autonomous vehicle to scan through a scanning zone while emitting light pulses based on a structured light pattern;

determining a three-dimensional (3-D) point map of the scanning zone based on time delays between emitting the light pulses and receiving corresponding returning reflected signals, based on orientations of the LIDAR device while emitting the light pulses, and further based on distortions in the returning reflected signals compared to the structured light pattern;

imaging, with a hyperspectral sensor associated with the autonomous vehicle, a region of the scanning zone corresponding to a plurality of reflective features indicated by the 3-D point map so as to generate spectral information characterizing a spectral distribution of radiation received from the reflective features;

correlating the spectral distribution of radiation received from a first reflective feature with one or more spectral characteristics of carbon monoxide;

in response to the correlating, determining the first reflective feature does not include a solid material;

analyzing the spectral information to determine whether a remainder of the reflective

features includes a solid material;

combining information from the LIDAR device and the hyperspectral sensor to generate a map of solid objects in the scanning zone, wherein reflective features determined to not include a solid material are not included in the map;

analyzing the positions of the solid objects in the map to identify obstacles relative to a current path of the autonomous vehicle;

determining a modified path of the autonomous vehicle that avoids the identified obstacles; and

controlling the autonomous vehicle to navigate along the modified path.

2. The method according to claim 1, wherein analyzing the spectral information comprises:

correlating the spectral distribution of radiation received from a second reflective feature with one or more spectral characteristic of water or water vapor; and

in response to the correlating determining the second reflective feature does not include a solid material.

3. The method according to claim 1, further comprising:

identifying the region of the scanning zone for spectral analysis responsive to associating a group of points included in the reflective feature with splashing water within the scanning zone.

4. The method according to claim 1, further comprising:

identifying the region of the scanning zone for spectral analysis responsive to associating a group of points included in the reflective feature with water vapor within the scanning zone.

5. The method according to claim 1, further comprising:

identifying the region of the scanning zone for spectral analysis responsive to associating a group of points included in the reflective feature with an exhaust plume within the scanning zone.

6. The method according to claim 1, further comprising:

identifying the region of the scanning zone for spectral analysis responsive to failing to associate a group of points included in the reflective feature with a physical object within the scanning zone.

7. The method according to claim 1, wherein the hyperspectral sensor is configured to detect radiation with wavelengths between 400 nm and 2500 nm.

8. The method according to claim 1, wherein the LIDAR device is configured to complete a scan of the entire scanning zone during a scanning interval, and wherein the imaging the region is initiated during the scanning interval based on information indicative of returning reflected signals received during one or more previous scanning intervals.

9. An autonomous vehicle system comprising:

a light detection and ranging (LIDAR) device including;

a light source configured to be scanned through a scanning zone while emitting light pulses based on a structured light pattern; and

a light detector configured to receive returning reflected signals from reflective features in the scanning zone, if any;

a hyperspectral sensor configured to image selected regions of the scanning zone and to characterize spectral distributions of radiation received from the imaged regions; and

a controller configured to;

instruct the LIDAR device to scan the scanning zone while emitting light pulses;

receive information from the LIDAR device indicative of the time delays between the emission of the light pulses and the reception of corresponding returning reflected signals;

determine, based on the time delays and orientations of the LIDAR device while emitting the light pulses, and further based on distortions in the returning reflected signals compared to the structured light pattern;

a three dimensional (3-D) point map;

instruct the hyperspectral sensor to image a region of the scanning zone corresponding to a plurality of reflective features indicated by the 3-D point map;

receive from the hyperspectral sensor spectral information characterizing a spectral distribution of radiation received from the plurality of reflective features;

correlate the spectral distribution of radiation received from a first reflective feature with one or more spectral characteristics of carbon monoxide; and

in response to the correlating, determining the first reflective feature does not include a solid material;

determine whether a remainder of the reflective features includes a solid material based on the spectral information;

combining information from the LIDAR device and the hyperspectral sensor to generate a map of solid objects in the scanning zone, wherein reflective features determined to not include a solid material are not included in the map;

analyze the positions of solid objects in the map to identify obstacles relative to a current path of the autonomous vehicle;

determine a modified path of the autonomous vehicle that avoids the identified obstacles; and

control the autonomous vehicle to navigate along the modified path.

10. The autonomous vehicle system according to claim 9, wherein the controller is further configured to determine whether the remainder of the reflective features includes a solid material by:

correlating the spectral distribution of radiation received from a second reflective feature with one or more spectral characteristic of water or water vapor; and

in response to the correlating determining the second reflective feature does not include a solid material.

11. The autonomous vehicle system according to claim 9, wherein the controller is further configured to:

identify the region of the scanning zone for spectral analysis responsive to associating a group of points included in the reflective feature with splashing water within the scanning zone.

12. The autonomous vehicle system according to claim 9, wherein the controller is further configured to:

identify the region of the scanning zone for spectral analysis responsive to associating a group of points included in the reflective feature with water vapor within the scanning zone.

13. The autonomous vehicle system according to claim 9, wherein the controller is further configured to:

identify the region of the scanning zone for spectral analysis responsive to associating a group of points included in the reflective feature with an exhaust plume within the scanning zone.

14. A computer readable medium storing instructions that, when executed by one or more processors in a computing device, cause the computing device to perform operations, the operations comprising:

scanning a light detection and ranging (LIDAR) device associated with an autonomous vehicle through a scanning zone while emitting light pulses based on a structured light pattern;

determining a three-dimensional (3-D) point map of the scanning zone based on time delays between emitting the light pulses and receiving corresponding returning reflected signals, based on orientations of the LIDAR device while emitting the light pulses, and further based on distortions in the returning reflected signals compared to the structured light pattern;

imaging, with a hyperspectral sensor associated with the autonomous vehicle, a region of the scanning zone corresponding to a plurality of reflective features indicated by the 3-D point map so as to generate spectral information characterizing a spectral distribution of radiation received from the reflective features;

correlating the spectral distribution of radiation received from a first reflective feature with one or more spectral characteristics of carbon monoxide;

in response to the correlating, determining the first reflective feature does not include a solid material;

analyzing the spectral information to determine whether a remainder of the reflective feature include a solid material

combining information from the LIDAR device and the hyperspectral sensor to generate a map of solid objects in the scanning zone, wherein reflective features determined to not include a solid material are not included in the map;

analyzing the positions of solid objects in the map to identify obstacles relative to a current path of the autonomous vehicle;

determining a modified path of the autonomous vehicle that avoids the identified obstacles; and

controlling the autonomous vehicle to navigate along the modified path.

15. The computer readable medium according to claim 14, wherein the operations further comprise:

correlating the spectral distribution of radiation received from a second reflective feature with one or more spectral characteristics of water or water vapor; and

in response to the correlating, determining the second reflective feature does not include a solid material.