Method and system for weather condition detection with image-based road characterization

• US 8,436,902 B2
• Filed: 08/29/2008
• Issued: 05/07/2013
• Est. Priority Date: 08/30/2007
• Status: Active Grant

• Current Assignee: Valeo Schalter und Sensoren GmbH (Valeo SE)
• Original Assignee: Valeo Schalter und Sensoren GmbH (Valeo SE)
• Inventors: Kuehnle, Andreas
• Primary Examiner(s): Avellino, Joseph
• Assistant Examiner(s): KHAN, AFTAB N
• Application Number:
  US12/674,925
• Publication Number:
  US 20110074955A1
• Time in Patent Office: 1,712 Days
• Field of Search: 348/143, 348/148, 348/161, 348/149, 348/142, 348/158, 382/104, 382/E07.085, 382/100, 382/199, 382/274, 382/194, 382/221, 382/195
• US Class Current: 348/148
• CPC Class Codes: G01W 1/14 Rainfall or precipitation gauges measuring volume in general G01F
• CPC Class Codes: G06T 2207/30252 Vehicle exterior; Vicinity of vehicle
• CPC Class Codes: G06T 7/41 based on statistical description of texture
• CPC Class Codes: G06V 20/588 Recognition of the road, e.g. of lane markings; Recognition of the vehicle driving pattern in relation to the road

Abstract

A method for image-based weather condition detection, using image processing methods in moving vehicles, is described. The method includes detecting road weather conditions by identifying a group of bright pixels in video images produced by an image-based road characterization, and determining whether the bright pixels are caused by reflections due to one of rainfall, snowfall, a wet surface, or a snowy surface, where a driving assistant system includes a lane-keeping system.

Claims

1. A method for image-based weather condition detection, using image processing methods in moving vehicles comprising a driving assistant system, the method comprising:

detecting road weather conditions by;

identifying a group of bright pixels in video images produced by an image-based road characterization, and

determining whether said bright pixels are caused by reflections due to one selected from a group consisting of rainfall, snowfall, a wet surface, or a snowy surface,

wherein falling snow is detected by a proximity of the group of bright pixels to each other, and wherein the proximity is evaluated using a histogram of nearest distances between bright pixels,

wherein the proximity of the group of bright pixels is evaluated by calculating which fraction of bright pixels of a total number of bright pixels is near enough to another bright pixel to qualify as road markings, wherein;

a ratio of

2. The method according to claim 1, wherein identifying the group of bright pixels in video images comprises one of comparing a relative brightness of pixels to neighboring pixels, or comparing the relative brightness of pixels to an average brightness value.

3. The method according to claim 1, wherein identifying the bright pixels in video images comprises considering day and night conditions.

4. The method according to claim 1 wherein determining whether snow is falling, the ratio is compared with a threshold, wherein:

when the ratio is less than the threshold, snow is falling, and

when the ratio is greater than or equal to the threshold, snow is not falling.

5. The method according to claim 1, wherein an averaging process is used to remove periodic rises and falls in the ratio, wherein the rises and the falls are a result of dashed road markings coming into and passing out of a field of view.

6. The method according to claim 1, wherein falling snow is detected by using an expected motion of the group of bright pixels initially defined as candidate marking objects, wherein the candidate marking objects are used to determine whether the group of bright pixels are a part of markings.

7. The method according to claim 1, wherein the snowy surface of a road is detected by identifying bright pixels forming wide strips on the road that are lighter than the surrounding roadway.

8. The method according to claim 1, wherein the snowy surface of a road is detected by identifying bright pixels forming light strips comprising varying width while traveling on the road.

9. The method according to claim 1, wherein a the wet surface on a road is detected by identifying bright pixels belonging to reflections and originating from one of overhead roadside lights or vehicles ahead, wherein the persistence of said reflections is used to determine whether one is currently in a wet area.

10. The method according to claim 1, wherein searching for the group of bright pixels in video images belonging to reflections comprises choosing an area of reflections located on a hood of the vehicle to analyze light sources vertically above the road reflections.

11. The method according to claim 1, wherein an intensity of falling rain is detected by an edge width of reflections given by the group of bright pixels identified in video images, said reflections having a plateau-like appearance on a graph of intensity, with a plateau on a top and a falloff at the sides of the graph, wherein increased extended sides correlate with increased scattering and heavier rainfall.

12. The method according to claim 11, wherein determining extensions of the sides of the graph of intensity comprises defining a half-intensity point as a location on each side of the plateau where the intensity decreases to half of a maximum value in a center of the reflections, wherein the distance of the half-intensity point from an edge of the plateau is indicative of the intensity of the falling rain.

13. A computer program product stored on a non-transitory computer usable medium comprising computer readable program instructions for causing a computer to perform the method of claim 1, wherein said computer program product is executed on the computer.

14. A system for road weather condition detection with an image processing apparatus in a moving vehicle comprising a driving assistant system with a lane-keeping system, comprising:

an electronic camera contained in the image processing apparatus for a video-based detection of the road, wherein the electronic camera is configured to capture scenes in front of the moving vehicle; and

a computer-based electronic circuit configured to evaluate a group of pixels in the video images to determine whether bright pixels are caused by reflections of one selected from a group consisting of rainfall, snowfall, a wet surface, or a snowy surface,

wherein falling snow is detected by a proximity of the group of bright pixels to each other, and wherein the proximity is evaluated using a histogram of nearest distances between bright pixels

wherein the proximity of the group of bright pixels is evaluated by calculating which fraction of bright pixels of a total number of bright pixels is near enough to another bright pixel to qualify as road markings, wherein;

a ratio of

Number of bright pixels near enough to another bright pixel

Number of bright pixels total

close to a value of 1.0 indicates that the group of bright pixels have nearby neighbors and are a part of road markings, and wherein the ratio close to a value of 0.0 indicates none of the group of bright pixels have nearby neighbors and are a result of snow.