Methods and systems for detecting weather conditions including fog using vehicle onboard sensors

• US 8,983,705 B2
• Filed: 04/30/2013
• Issued: 03/17/2015
• Est. Priority Date: 04/30/2013
• Status: Active Grant

• Current Assignee: Waymo LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Zhu, Jiajun, Dolgov, Dmitri, Ferguson, Dave
• Primary Examiner(s): Cheung, Mary
• Assistant Examiner(s): BUTLER, RODNEY ALLEN
• Application Number:
  US13/873,442
• Publication Number:
  US 20140324266A1
• Time in Patent Office: 686 Days
• Field of Search: 701/23, 701/27, 701/28, 701/301, 382/153, 356/5.01, 356/28.5, 342/89, 342/90, 342/118
• US Class Current: 701/23
• CPC Class Codes: B60W 2555/20 Ambient conditions, e.g. wind or rain
• CPC Class Codes: B60W 30/00 Purposes of road vehicle drive control systems not related to the control of a particular sub-unit, e.g. of systems using conjoint control of vehicle sub-units
• CPC Class Codes: B60W 50/14 Means for informing the driver, warning the driver or prompting a driver intervention
• CPC Class Codes: G01S 13/865 Combination of radar systems with lidar systems
• CPC Class Codes: G01S 13/931 of land vehicles
• CPC Class Codes: G01S 17/06 Systems determining position data of a target
• CPC Class Codes: G01S 17/86 Combinations of lidar systems with systems other than lidar, radar or sonar, e.g. with direction finders
• CPC Class Codes: G01S 17/931 of land vehicles
• CPC Class Codes: G01S 17/95 for meteorological use
• CPC Class Codes: G01S 7/4802 using analysis of echo signal for target characterisation; Target signature; Target cross-section
• CPC Class Codes: G01W 1/00 Meteorology
• CPC Class Codes: G01W 1/02 Instruments for indicating weather conditions by measuring two or more variables, e.g. humidity, pressure, temperature, cloud cover or wind speed G01W1/10 takes precedence
• CPC Class Codes: Y02A 90/10 Information and communication technologies [ICT] supporting adaptation to climate change, e.g. for weather forecasting or climate simulation

Abstract

Methods and systems for detecting weather conditions including fog using vehicle onboard sensors are provided. An example method includes receiving laser data collected from scans of an environment of a vehicle, and associating, by a computing device, laser data points of with one or more objects in the environment. The method also includes comparing laser data points that are unassociated with the one or more objects in the environment with stored laser data points representative of a pattern due to fog, and based on the comparison, identifying by the computing device an indication that a weather condition of the environment of the vehicle includes fog.

Claims

1. A method comprising:

receiving laser data collected from scans of an environment of a vehicle, wherein the laser data includes a plurality of laser data points;

associating, by a computing device, laser data points of the plurality of laser data points with one or more objects in the environment by;

for a first scan of the environment, associating, by the computing device, the laser data points of the plurality of laser data points with the one or more objects in the environment;

for a second scan of the environment, determining laser data points that match to the one or more objects based on a location of an object represented by the laser data points;

determining laser data points that are unassociated with the one or more objects in the environment based on a lack of a match to the one or more objects in the first scan;

comparing the laser data points that are unassociated with the one or more objects in the environment with stored laser data points representative of a pattern due to fog; and

based on the comparison, identifying by the computing device an indication that a weather condition of the environment of the vehicle includes fog.

2. The method of claim 1, wherein identifying by the computing device the indication that the weather condition of the environment of the vehicle includes fog comprises:

determining that a number of the laser data points unassociated with the one or more objects in the environment is above a threshold number.

3. The method of claim 1, further comprising:

receiving additional data collected for the environment of the vehicle from one or more additional sensors, wherein the additional data is indicative of a presence of one or more objects in the environment of the vehicle;

determining laser data points of the plurality of laser data points that are associated with the one or more objects in the environment indicated by the additional data; and

based on laser data points being unassociated with the one or more objects in the environment indicated by the additional data, identifying the indication that the weather condition of the environment of the vehicle includes fog.

4. The method of claim 1, further comprising:

receiving radar data collected for the environment of the vehicle, wherein the radar data is indicative of a presence of one or more objects in the environment of the vehicle;

determining laser data points of the plurality of laser data points that are associated with the one or more objects in the environment indicated by the radar data; and

based on laser data points being unassociated with the one or more objects in the environment indicated by the radar data, identifying the indication that the weather condition of the environment of the vehicle includes fog.

5. The method of claim 1, further comprising:

receiving image data captured from a camera coupled to the vehicle; and

identifying a second indication that the weather condition of the environment of the vehicle includes fog based on the image data.

6. The method of claim 1, further comprising:

receiving image data collected for the environment of the vehicle;

determining, by the computing device, one or more objects in the image data that are associated with the one or more objects in the environment; and

based on an object of the one or more objects in the image data being unassociated with the one or more objects in the environment, identifying by the computing device a second indication that the weather condition of the environment of the vehicle includes fog.

7. The method of claim 1, wherein the vehicle is configured to operate in an autonomous mode, and the method further comprises based on the indication that the weather condition of the environment of the vehicle includes fog, determining a driving decision for the vehicle.

8. The method of claim 1, wherein the vehicle is configured to operate in an autonomous mode, and the method further comprises based the indication that the weather condition of the environment of the vehicle includes fog, providing instructions to indicate a request to transition to a manual mode.

9. The method of claim 1, further comprising:

receiving weather information for a location of the vehicle from a server over a network; and

identifying a second indication that the weather condition of the environment of the vehicle includes the fog based on the weather information.

10. The method of claim 1, further comprising:

receiving a current temperature for a location of the vehicle from a server over a network; and

identifying a second indication that the weather condition of the environment of the vehicle includes fog based on the current temperature.

11. The method of claim 4, further comprising:

determining an object represented by laser data points that are unassociated with the one or more objects in the environment;

determining a distance between the object and a given object of the one or more objects in the environment indicated by the radar data and associated with laser data points; and

determining an estimation of a density of the fog based on the distance.

12. The method of claim 4, wherein identifying by the computing device the indication that the weather condition of the environment of the vehicle includes fog comprises:

determining that the laser data points unassociated with the one or more objects in the environment indicated by the radar data are representative of data points approximately around one of the one or more objects indicated by the radar data.

13. The method of claim 5, wherein identifying the second indication that the weather condition of the environment of the vehicle includes fog based on the images comprises:

determining that a number of pixels in the image data are substantially grey in color.

14. The method of claim 12, further comprising determining that the data points approximately around one of the one or more objects indicated by the radar data are representative of a substantially circular shape having a radius of a fog.

15. A non-transitory computer readable storage medium having stored therein instructions, that when executed by a computing device, cause the computing device to perform functions comprising:

receiving laser data collected from scans of an environment of a vehicle, wherein the laser data includes a plurality of laser data points;

associating laser data points of the plurality of laser data points with one or more objects in the environment;

receiving additional data collected for the environment of the vehicle from one or more additional sensors, wherein the additional data is indicative of a presence of one or more objects in the environment of the vehicle;

determining laser data points of the plurality of laser data points that are associated with the one or more objects in the environment indicated by the additional data; and

comparing laser data points that are unassociated with the one or more objects in the environment with stored laser data points representative of a pattern due to fog; and

based on the comparison and on laser data points being unassociated with the one or more objects in the environment indicated by the additional data, identifying by the computing device an indication that a weather condition of the environment of the vehicle includes fog.

16. A system, comprising:

at least one processor; and

data storage comprising instructions executable by the at least one processor to cause the system to perform functions comprising;

receiving laser data collected from scans of an environment of a vehicle, wherein the laser data includes a plurality of laser data points;

associating laser data points of the plurality of laser data points with one or more objects in the environment by;

for a first scan of the environment, associating the laser data points of the plurality of laser data points with the one or more objects in the environment;

for a second scan of the environment, determining laser data points that match to the one or more objects based on a location of an object represented by the laser data points;

determining laser data points that are unassociated with the one or more objects in the environment based on a lack of a match to the one or more objects in the first scan;

comparing the laser data points that are unassociated with the one or more objects in the environment with stored laser data points representative of a pattern due to fog; and

based on the comparison, identifying by the computing device an indication that a weather condition of the environment of the vehicle includes fog.

17. The system of claim 16, wherein the functions further comprise:

receiving radar data collected for the environment of the vehicle, wherein the radar data is indicative of a presence of one or more objects in the environment of the vehicle;

determining an object in the environment indicated by the radar data that is unassociated with laser data points;

determining a distance between the object in the environment indicated by the radar data unassociated with laser data points and a given object of the one or more objects in the environment indicated by the radar data and associated with laser data points; and

determining an estimation of a density of the fog based on the distance.

18. The system of claim 16, wherein the vehicle is configured to operate in an autonomous mode, and the functions further comprise:

based on the indication that the weather condition of the environment of the vehicle includes fog, determining a driving decision for the vehicle; and

providing instructions to indicate a request to transition to a manual mode.