Methods and systems for detecting weather conditions using vehicle onboard sensors

• US 10,247,854 B2
• Filed: 03/03/2017
• Issued: 04/02/2019
• Est. Priority Date: 05/07/2013
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Waymo LLC (Alphabet Inc.)
• Inventors: Zhu, Jiajun, Dolgov, Dmitri, Ferguson, Dave
• Primary Examiner(s): Raymond, Edward
• Application Number:
  US15/449,097
• Publication Number:
  US 20170176641A1
• Time in Patent Office: 760 Days
• Field of Search: 702 3, 702 4, 702 6, 702182-185
• CPC Class Codes: G01N 21/4738 Diffuse reflection precedence is given to G01N21/55 - G01N21/57 if specular component is taken into consideration, e.g. also for testing fluids, fibrous materials
• CPC Class Codes: G01S 17/86 Combinations of lidar systems with systems other than lidar, radar or sonar, e.g. with direction finders
• CPC Class Codes: G01S 17/87 Combinations of systems using electromagnetic waves other than radio waves
• CPC Class Codes: G01W 1/00 Meteorology
• CPC Class Codes: G01W 1/14 Rainfall or precipitation gauges measuring volume in general G01F
• CPC Class Codes: G05D 1/0088 characterized by the autonomous decision making process, e.g. artificial intelligence, predefined behaviours using knowledge based models G06N5/00
• CPC Class Codes: G05D 1/024 in combination with a laser lasers per se H01S
• CPC Class Codes: G05D 1/0246 using a video camera in combination with image processing means
• CPC Class Codes: G05D 1/0257 using a radar radar systems designed for anti-collision purposes between land vehicles or between land vehicle and fixed obstacles G01S13/931
• CPC Class Codes: G05D 1/0276 using signals provided by a source external to the vehicle involving a plurality of vehicles G05D1/0287; automatically controlling vehicle speed responsive to externally generated signals B60K31/0058
• CPC Class Codes: H04N 7/183 for receiving images from a single remote source
• CPC Class Codes: Y02A 90/10 Information and communication technologies [ICT] supporting adaptation to climate change, e.g. for weather forecasting or climate simulation

Abstract

Example methods and systems for detecting weather conditions using vehicle onboard sensors are provided. An example method includes receiving laser data collected for an environment of a vehicle, and the laser data includes a plurality of laser data points. The method also includes associating, by a computing device, laser data points of the plurality of laser data points with one or more objects in the environment, and determining given laser data points of the plurality of laser data points that are unassociated with the one or more objects in the environment as being representative of an untracked object. The method also includes based on one or more untracked objects being determined, identifying by the computing device an indication of a weather condition of the environment.

Claims

1. A method comprising:

receiving laser data collected for an environment of a vehicle, wherein the laser data includes a plurality of laser data points;

associating, by a computing device, laser data points of the plurality of laser data points with one or more objects in the environment;

determining given laser data points of the plurality of laser data points that are unassociated with the one or more objects in the environment as being representative of an untracked object;

receiving additional data collected for the environment of the vehicle from one or more additional sensors, wherein the additional data is indicative of a presence of one or more objects in the environment of the vehicle;

determining laser data points of the plurality of laser data points that are associated with the one or more objects in the environment indicated by the additional data;

based on laser data points being unassociated with the one or more objects in the environment indicated by the additional data, identifying by the computing device an indication of a weather condition in the environment; and

controlling the vehicle in an autonomous mode based on the indication of the weather condition.

2. The method of claim 1, wherein receiving the laser data collected for the environment of the vehicle comprises receiving the laser data collected by performing one or more scans of the environment, and the method further comprises:

tracking the one or more objects in the environment as the vehicle moves through the environment based on laser data received in the one or more scans of the environment.

3. The method of claim 1, further comprising:

for a first scan of the environment, associating, by the computing device, the laser data points of the plurality of laser data points with one or more objects in the environment;

for a second scan of the environment, determining laser data points that match to the one or more objects based on a location of an object represented by the laser data points; and

determining the given laser data points that are unassociated with the one or more objects in the environment based on a lack of a match to the one or more objects in the first scan.

4. The method of claim 1, wherein identifying by the computing device the indication of the weather condition of the environment comprises:

determining that a number of the given laser data points unassociated with the one or more objects in the environment is above a threshold number.

5. The method of claim 1, further comprising:

receiving radar data collected for the environment of the vehicle, wherein the radar data is indicative of a presence of one or more objects in the environment of the vehicle;

determining laser data points of the plurality of laser data points that are associated with the one or more objects in the environment indicated by the radar data; and

based on laser data points being unassociated with the one or more objects in the environment indicated by the radar data, identifying the indication of the weather condition of the environment.

6. The method of claim 1, further comprising:

receiving image data captured from a camera coupled to the vehicle; and

identifying a second indication of the weather condition of the environment based on the image data.

7. The method of claim 1, further comprising:

receiving weather information for a location of the vehicle from a server over a network; and

identifying a second indication of the weather condition of the environment based on the weather information.

8. The method of claim 1, further comprising:

receiving a current temperature for a location of the vehicle from a server over a network; and

identifying a second indication of the weather condition of the environment based on the current temperature.

9. The method of claim 1, further comprising:

receiving precipitation data from a precipitation sensor coupled to the vehicle; and

identifying a second indication of the weather condition of the environment based on the precipitation data.

10. The method of claim 1, wherein the vehicle is configured to operate in an autonomous mode, and the method further comprises based on the indication of the weather condition of the environment, determining a driving decision for the vehicle.

11. The method of claim 1, further comprising:

determining a speed of the vehicle, and

wherein identifying by the computing device the indication of the weather condition of the environment includes identifying that a surface on which the vehicle travels is wet based on the speed of the vehicle being above a threshold.

12. The method of claim 1, further comprising:

comparing the given laser data points that are unassociated with the one or more objects in the environment with stored laser data points representative of a pattern due to fog; and

based on the comparison, identifying by the computing device the weather condition of the environment to include fog.

13. The method of claim 1, further comprising:

determining that the given laser data points of the plurality of laser data points that are unassociated with the one or more objects in the environment are representative of an untracked object at a given position with respect to the vehicle;

determining that the untracked object remains at a substantially same relative position with respect to the vehicle as the vehicle moves; and

identifying by the computing device the weather condition of the environment to be sunny.

14. The method of claim 1, further comprising:

determining a geographic location of the vehicle and a time of day;

based on the geographic location of the vehicle and the time of day, determining an approximate position of the sun relative to the vehicle; and

identifying by the computing device a second indication of the weather condition of the environment based on the approximate position of the sun relative to the vehicle.

15. The method of claim 1, wherein the weather condition includes one or more of wetting of a roadway, a fog or sunlight.

16. The method of claim 1, wherein controlling the vehicle in an autonomous mode based on the indication of the weather condition comprises transitioning vehicle control to a manual mode.

17. A non-transitory computer readable storage medium having stored therein instructions, that when executed by a computing device, cause the computing device to perform functions comprising:

receiving laser data collected for an environment of a vehicle, wherein the laser data includes a plurality of laser data points;

associating laser data points of the plurality of laser data points with one or more objects in the environment;

determining given laser data points of the plurality of laser data points that are unassociated with the one or more objects in the environment as being representative of an untracked object;

receiving additional data collected for the environment of the vehicle from one or more additional sensors, wherein the additional data is indicative of a presence of one or more objects in the environment of the vehicle;

determining laser data points of the plurality of laser data points that are associated with the one or more objects in the environment indicated by the additional data;

based on laser data points being unassociated with the one or more objects in the environment indicated by the additional data, identifying by the computing device an indication of a weather condition in the environment; and

controlling the vehicle in an autonomous mode based on the indication of the weather condition.

18. The non-transitory computer readable storage medium of claim 17, wherein receiving the laser data collected for the environment of the vehicle comprises receiving the laser data collected by performing one or more scans of the environment, and the functions further comprise:

tracking the one or more objects in the environment as the vehicle moves through the environment based on laser data received in the one or more scans of the environment.

19. The non-transitory computer readable storage medium of claim 17, wherein controlling the vehicle in an autonomous mode based on the indication of the weather condition comprises transitioning vehicle control to a manual mode.

20. A system, comprising:

at least one processor; and

data storage comprising instructions executable by the at least one processor to cause the system to perform functions comprising;

receiving laser data collected for an environment of a vehicle, wherein the laser data includes a plurality of laser data points;

associating laser data points of the plurality of laser data points with one or more objects in the environment;

determining given laser data points of the plurality of laser data points that are unassociated with the one or more objects in the environment as being representative of an untracked object;

receiving additional data collected for the environment of the vehicle from one or more additional sensors, wherein the additional data is indicative of a presence of one or more objects in the environment of the vehicle;

determining laser data points of the plurality of laser data points that are associated with the one or more objects in the environment indicated by the additional data; and

based on laser data points being unassociated with the one or more objects in the surrounding environment indicated by the additional data, identifying an indication of a weather condition in the environment; and

controlling the vehicle in an autonomous mode based on the indication of the weather condition.

21. The system of claim 20, wherein the vehicle is traveling along a roadway, and the functions further comprise:

receiving radar data collected for the environment of the vehicle, wherein the radar data is indicative of a presence of one or more objects in the environment of the vehicle;

determining laser data points of the plurality of laser data points that are associated with the one or more objects in the environment indicated by the radar data; and

based on laser data points being unassociated with the one or more objects in the environment indicated by the radar data, identifying the indication of the weather condition of the environment.

22. The system of claim 20, wherein the weather condition includes one or more of wetting of a roadway, a fog or sunlight.

23. The system of claim 20, wherein controlling the vehicle in an autonomous mode based on the indication of the weather condition comprises transitioning vehicle control to a manual mode.