Method and system for reflection detection in road video images

• US 8,472,673 B2
• Filed: 01/23/2008
• Issued: 06/25/2013
• Est. Priority Date: 01/23/2007
• Status: Active Grant

• Current Assignee: Valeo Schalter und Sensoren GmbH (Valeo SE)
• Original Assignee: Valeo Schalter und Sensoren GmbH (Valeo SE)
• Inventors: Kuehnle, Andreas, Boon, Cathy
• Primary Examiner(s): Wang, Claire X
• Application Number:
  US12/523,847
• Publication Number:
  US 20100086211A1
• Time in Patent Office: 1,980 Days
• Field of Search: 382/103, 382/104, 348/342
• US Class Current: 382/104
• CPC Class Codes: G06T 7/246 using feature-based methods, e.g. the tracking of corners or segments
• CPC Class Codes: G06T 7/60 Analysis of geometric attributes
• CPC Class Codes: G06V 20/588 Recognition of the road, e.g. of lane markings; Recognition of the vehicle driving pattern in relation to the road

Abstract

A method and system for reflection detection in road video images, is provided. One implementation involves detecting road surface reflections, by receiving an image of a road in front of a vehicle from an image capturing device, determining a region of interest in an identified road in the image, and detecting road surface reflections in the region of interest.

Claims

1. A method of detecting road surface reflections, comprising the steps of

receiving an image of a road in front of a vehicle from an image capturing device;

determining a region of interest in an identified road in the image using a region of interest processing module;

identifying road surface reflections in the region of interest using a reflection detection device, wherein the identified road surface reflections are not lane markings;

determining luminance gradients in the region of interest, wherein determining luminance gradients includes determining edge strength in the region of interest to identify potential reflection edges;

wherein identifying road surface reflections includes

comparing the luminance gradients to a strength threshold and comparing edge strength to said strength threshold;

identifying a pair of edges as potential reflection edges by comparing each edge strength to said strength threshold;

determining spacing between the pair of edges;

comparing said spacing to a spacing threshold, wherein if the edge spacing is greater than the spacing threshold, then the edge pair is eliminated from further consideration, otherwise the edge pair may represent a reflection;

determining lengths of possible reflection edges; and

comparing each edge length against a length threshold, wherein if each edge length is less than the length threshold, then the edge pair is eliminated from further consideration;

wherein remaining edges represent reflection.

2. The method of claim 1 wherein:

the image comprises a two-dimensional array of pixels arranged in rows and columns; and

detecting road surface reflections further includes;

determining peaks and valleys in luminance gradients of image pixels, wherein a gradient peak represents an edge in the image, and a gradient valley represents an edge in the image;

determining column location of each pixel luminance gradient peak, and storing the column location in a one dimensional (1-D) reflection edge start histogram; and

determining column location of each pixel luminance gradient valley, and storing the column location in a one dimensional (1-D) reflection edge end histogram.

3. The method of claim 2 further including determining edge entries in the histograms exceeding a length threshold to eliminate from consideration those edges that are too short to represent reflections.

4. The method of claim 3 further including:

forming reflection edge start and end edge pairs based on the entries in the start and end histograms, respectively; and

determining spacing between edges in each pair, comparing the spacing to a spacing threshold and eliminating too widely spaced edges as not being reflections;

wherein the remaining edges represent reflection pixels.

5. The method of claim 1 wherein:

the image comprises a two-dimensional array of pixels arranged in rows and columns; and

detecting road surface reflections includes;

determining peaks and valleys in luminance gradients of image pixels, wherein a gradient peak represents an edge in the image, and a gradient valley represents an edge in the image;

comparing spacing between a peak and valley pair in each pixel row against a spacing threshold, wherein if the spacing is greater than the spacing threshold, then eliminating that peak and valley pair from further consideration;

comparing peak and valley pair lengths against a length threshold, wherein if peak and valley pair lengths are greater than the length threshold, then eliminating that peak and valley pair from further consideration;

for a group of remaining peak and valley pairs, determining center pixel of each peak-valley pair as gradient center locations;

performing least-squares fitting of a line to the gradient center locations; and

identifying all peak-valley pairs that contributed to a line fit slope of magnitude above a slope threshold, as reflection pixels.

6. The method of claim 1 wherein detecting road surface reflections in the region of interest further includes using spectral characteristics of the reflections in a color image of the road to identify road surface reflections.

7. The method of claim 6 wherein the color image comprises an RGB image wherein a red channel of the color RGB image is analyzed for features of interest.

8. An apparatus for detecting road surface reflections, comprising:

an image processing module configured to receive an image of a road in front of a vehicle from an image capturing device, and determining a region of interest in an identified road in the image;

a reflection detection module configured to identify road surface reflections in the region of interest, wherein the identified road surface reflections are not lane markings;

a gradient determination module configured to determine luminance gradients in the region of interest;

wherein the reflection detection module is further configured to identify road surface reflections by comparing the luminance gradients to a strength threshold;

wherein the gradient determination module is further configured to determine edge strength in the region of interest to identify potential reflection edges,

wherein the reflection detection module is further configured to compare edge strength to said strength threshold;

wherein the reflection detection module is further configured to identify a pair of edges as potential reflection edges by comparing each edge strength to said strength threshold;

a spacing check module configured to determine spacing between the pair of edges and comparing said spacing to a spacing threshold, wherein if the edge spacing is greater than the spacing threshold, then the edge pair is eliminated from further consideration, otherwise the edge pair may represent a reflection; and

a reflection length check module configured to determine lengths of possible reflection edges, and comparing each edge length against a length threshold, wherein if each edge length is less than the length threshold, then the edge pair is eliminated from further consideration, wherein remaining edges represent reflection edges.

9. The apparatus of claim 8 wherein:

the image comprises a two-dimensional array of pixels arranged in rows and columns; and

the apparatus further comprising a gradient determination module configured to determine peaks and valleys in luminance gradients of image pixels, wherein a gradient peak represents an edge in the image, and a gradient valley represents an edge in the image;

the reflection detection module is further configured to determine column location of each pixel luminance gradient peak, and store the column location in a one dimensional (1-D) reflection edge start histogram; and

determine column location of each pixel luminance gradient valley, and store the column location in a one dimensional (1-D) reflection edge end histogram.

10. The apparatus of claim 9 further comprising an edge length check module configured to determine edge entries in the histograms exceeding a length threshold to eliminate from consideration those edges that are too short to represent reflections.

11. The apparatus of claim 10 further comprising an edge spacing check module configured to:

form reflection edge start and end edge pairs based on the entries in the start and end histograms, respectively; and

determine spacing between edges in each pair, compare the spacing to a spacing threshold and eliminate too widely spaced edges as not being reflections;

wherein the remaining edges represent reflection pixels.

12. The apparatus of claim 8 wherein:

the image comprises a two-dimensional array of pixels arranged in rows and columns; and

the apparatus further includes;

a gradient determination module configured to determine pixel gradients in the image, and determine peaks and valleys in luminance gradients of image pixels, wherein a gradient peak represents an edge in the image, and a gradient valley represents an edge in the image;

a spacing check module configured to compare spacing between a peak and valley pair in each pixel row against a spacing threshold, wherein if the spacing is greater than the spacing threshold, then eliminating that peak and valley pair from further consideration;

a length check module configured to compare peak and valley pair lengths against a length threshold, wherein if peak and valley pair lengths are greater than the length threshold, then eliminating that peak and valley pair from further consideration;

wherein the reflection detection module is further configured such that for a group of remaining peak and valley pairs, the reflection detection module determines center pixel of each peak-valley pair as gradient center locations, performs least-squares fitting of a line to the gradient center locations, and selects all peak-valley pairs that contributed to a line fit slope of magnitude above a slope threshold, as reflection pixels.

13. The apparatus of claim 8 wherein the reflection detection module is further configured to use spectral characteristics of the reflections in a color image of the road to identify road surface reflections.

14. A system for detecting road surface reflections, comprising:

an image processing module configured to receive an image of a road in front of a vehicle from an image capturing device, and determine a region of interest in an identified road in the image;

a reflection detection module configured to identify, and suppress, road surface reflections in the region of interest, wherein the identified road surface reflections are not lane markings;

a road boundary detector to detect road boundaries from the region of interest;

a lane departure detector to detect vehicle proximity to the detected road boundaries;

a gradient determination module configured to determine luminance gradients in the region of interest;

wherein the reflection detection module is further configured to identify road surface reflections by comparing the luminance gradients to a strength threshold;

wherein the gradient determination module is further configured to determine edge strength in the region of interest to identify potential reflection edges;

wherein the reflection detection module is further configured to compare edge strength to said strength threshold;

wherein the reflection detection module is further configured to identify a pair of edges as potential reflection edges by comparing each edge strength to said strength threshold;

a spacing check module configured to determine spacing between the pair of edges and comparing said spacing to a spacing threshold, wherein if the edge spacing is greater than the spacing threshold, then the edge pair is eliminated from further consideration, otherwise the edge pair may represent a reflection; and

a reflection length check module configured to determine lengths of possible reflection edges, and compare each edge length against a length threshold, wherein if each edge length is less than the length threshold, then the edge pair is eliminated from further consideration, wherein remaining edges represent reflection edges.