Method and apparatus for detecting and assessing road reflections

• US 10,442,438 B2
• Filed: 05/04/2016
• Issued: 10/15/2019
• Est. Priority Date: 05/06/2015
• Status: Active Grant

• Current Assignee: Continental Teves AG & Co. OHG (Continental AG)
• Original Assignee: Continental Teves AG & Co. OHG (Continental AG)
• Inventors: Fritz, Stefan, Hartmann, Bernd, Amthor, Manuel, Denzler, Joachim
• Primary Examiner(s): Saini, Amandeep
• Application Number:
  US15/572,010
• Publication Number:
  US 20180141561A1
• Time in Patent Office: 1,259 Days
• Field of Search: 382108
• CPC Class Codes: B60W 2552/00 Input parameters relating to infrastructure
• CPC Class Codes: B60W 40/06 Road conditions
• CPC Class Codes: G06T 2207/30252 Vehicle exterior; Vicinity of vehicle
• CPC Class Codes: G06T 7/55 from multiple images
• CPC Class Codes: G06V 10/247 by affine transforms, e.g. correction due to perspective effects; Quadrilaterals, e.g. trapezoids
• CPC Class Codes: G06V 10/464 using a plurality of salient features, e.g. bag-of-words [BoW] representations
• CPC Class Codes: G06V 10/60 relating to illumination properties, e.g. using a reflectance or lighting model
• CPC Class Codes: G06V 20/56 exterior to a vehicle by using sensors mounted on the vehicle
• CPC Class Codes: G06V 20/588 Recognition of the road, e.g. of lane markings; Recognition of the vehicle driving pattern in relation to the road

Abstract

In a method for detecting and assessing reflections on a road (7), a camera (2) produces at least two digital images of at least one point (3) of the road, respectively from different recording perspectives (A, B) of the camera (2). Diffuse reflection and specular reflection of the road (7) are then detected by assessing differences in the appearances of the point (3) of the road in the at least two digital images using digital image processing algorithms. Road reflections are preferably assessed using an approximative approach. An item of road condition information is determined based on the detected reflection, preferably an item of road condition information which states whether the road is dry, wet, snow-covered or icy. Also provided are an apparatus (1) for carrying out the above-mentioned method, and a vehicle having such an apparatus.

Claims

1. A method for detecting and assessing reflections of at least one point on a road, comprising the method steps:

providing a camera;

producing at least two digital images of the at least one point of the road by the camera, wherein the images are produced from different recording perspectives of the camera;

distinguishing diffuse reflection and specular reflection of the road by assessing differences in appearances of the at least one point of the road in the at least two digital images using digital image processing algorithms; and

determining an item of road condition information based on the distinguished reflection;

wherein the at least two images are averaged to produce an average image, and wherein an absolute difference or a quadratic difference is calculated between each pixel in the average image and an associated image column mean value.

2. The method according to claim 1, characterized by producing the at least two digital images of the at least one point of the road by the camera which is a stereo camera, wherein the images are produced from the different recording perspectives of imaging elements of the stereo camera.

3. The method according to claim 1, characterized by

distinguishing between the diffuse reflection and the specular reflection based on digital image processing by differentiating between appearances fixed by the road and independent of the road caused by the relative movement of the camera, therefore

reliably separating shadows and reflected infrastructure on the road.

4. The method according to claim 1, characterized by

communicating the item of road condition information to a driver assistance system of a vehicle, and

adjusting times for issuing an alert or for intervention by the driver assistance system based on the item of road condition information.

5. The method according to claim 1, characterized by

incorporating the item of road condition information into a function of an automated vehicle, and

adjusting a driving strategy and establishing handover points between an automated system and a driver based on the item of road condition information.

6. The method according to claim 1, characterized by

producing the at least two digital images of a plurality of the points of the road, which form a trapezoidal region, from the different recording perspectives by the camera, and

transforming the trapezoidal region by an estimated homography into a rectangular top view.

7. The method according to claim 1, characterized by

providing the camera in a vehicle;

producing a first one of the digital images in a first position of the vehicle from a first one of the recording perspectives;

moving the vehicle into a second position which differs from the first position;

producing a second one of the digital images in the second position of the vehicle from a second one of the recording perspectives;

transforming the at least two digital images from the different recording perspectives respectively into at least two top views;

registering the at least two top views produced with digital image processing, incorporating driving dynamics parameters of the vehicle; and

comparing the appearances of the at least one point of the road in the at least two registered top views.

8. The method according to claim 7, characterized by extracting features of the at least one point of the road, which features capture a change in the appearance in the at least two registered top views.

9. The method according to claim 8, characterized by

forming a feature vector from the extracted features, and

assigning the feature vector to a class by a classifier.

10. The method according to claim 7, characterized by

producing the average image by averaging the at least two top views.

11. The method according to claim 1, characterized by extracting features of the average image, taking account of the image column mean values, during which an occurrence of particular prototype values or value tuples is captured based on a histogram.

12. The method according to claim 1, characterized by incorporating camera parameters of the camera into the assessing of the differences in the appearances.

13. An apparatus for detecting and assessing reflections of at least one point on a road, comprising a camera which is configured to produce at least two digital images of the at least one point of the road from different recording perspectives,

wherein the apparatus is configured to:

assess differences in appearances of the at least one point of the road using digital image processing algorithms and, as a result, detect diffuse reflections and specular reflections of the road,

determine an item of road condition information based on the detected reflection,

average the at least two images to produce an average image, and

calculate an absolute difference or a quadratic difference between each pixel in the average image and an associated image column mean value.

14. A combination comprising the apparatus according to claim 13 and a vehicle.