Method and apparatus for determining a road condition

• US 10,147,002 B2
• Filed: 02/14/2014
• Issued: 12/04/2018
• Est. Priority Date: 02/19/2013
• Status: Active Grant

• Current Assignee: Continental Teves AG & Co. OHG (Continental AG), Conti Temic Microelectronic GmbH (Continental AG)
• Original Assignee: Continental Teves AG & Co. OHG (Continental AG), Conti Temic Microelectronic GmbH (Continental AG)
• Inventors: Hartmann, Bernd, Kroekel, Dieter, Fritz, Stefan, Denzler, Joachim, Froehlich, Bjoern, Kemmler, Michael, Schubert, Stefan, Bach, Eric
• Primary Examiner(s): Lee, Y
• Assistant Examiner(s): CARTER, RICHARD BRUCE
• Application Number:
  US14/764,782
• Publication Number:
  US 20150371095A1
• Time in Patent Office: 1,754 Days
• Field of Search: 348148, 381 56, 382108, 701 1, 701 37
• CPC Class Codes: G06F 18/22 Matching criteria, e.g. proximity measures
• CPC Class Codes: G06F 18/24 Classification techniques
• CPC Class Codes: G06T 3/40 Scaling of whole images or parts thereof, e.g. expanding or contracting
• CPC Class Codes: G06T 7/11 Region-based segmentation
• CPC Class Codes: G06V 20/56 exterior to a vehicle by using sensors mounted on the vehicle
• CPC Class Codes: G06V 20/588 Recognition of the road, e.g. of lane markings; Recognition of the vehicle driving pattern in relation to the road
• CPC Class Codes: G06V 30/422 Technical drawings; Geographical maps
• CPC Class Codes: H04N 13/20 Image signal generators
• CPC Class Codes: H04N 23/635 Region indicators; Field of view indicators

Abstract

A method and an apparatus determine a road condition using a vehicle camera. At least one image is acquired using the camera. A first image area that includes an image of the road surface is determined in the at least one image. A classifier assigns the first image area to at least one class (among pre-established classes) that represents a specific road condition of the road surface. Information regarding this specific road condition is output.

Claims

1. A method of determining a road condition of a road surface of a road on which a subject vehicle is traveling, comprising:

with a camera of the subject vehicle, acquiring successive camera images including the road surface, wherein the successive camera images include an earlier image acquired at an earlier time and a later image acquired at a later time after the earlier time;

in the later image, determining a first image area including a first surface area on the road surface;

in the earlier image, determining a second image area including a second surface area on the road surface;

performing image processing including classification of the first image area in the later image to classify the first surface area on the road surface at the later time into a first class among plural pre-determined classes respectively representing different road conditions;

performing image processing including classification of the second image area in the earlier image to classify the second surface area on the road surface at the earlier time into a second class among the plural pre-determined classes respectively representing different road conditions;

performing merging or plausibility-checking, wherein the merging comprises merging the classification of the first surface area on the road surface at the later time in the first class with the classification of the second surface area on the road surface at the earlier time in the second class to produce merged information regarding the road condition, and wherein the plausibility-checking comprises checking a plausibility of the classification of the first surface area on the road surface at the later time in the first class relative to the classification of the second surface area on the road surface at the earlier time in the second class to produce plausibility-checked information regarding the road condition; and

outputting the merged information or the plausibility-checked information as output information.

2. The method according to claim 1, wherein the first image area and the second image area do not overlap with one another.

3. The method according to claim 1, wherein the second image area covers a section of the road surface that lies farther ahead of the subject vehicle than a section of the road surface covered by the first image area.

4. The method according to claim 1, wherein the second surface area on the road surface at the earlier time corresponds at least partially with the first surface area on the road surface at the later time as the subject vehicle travels along the road, so that the classification of the second surface area at the earlier time provides an expected preview of the classification of the first surface area at the later time.

5. The method according to claim 4, wherein a correspondence of the second surface area on the road surface at the earlier time with the first surface area on the road surface at the later time is achieved using odometry data of the subject vehicle traveling along the road.

6. The method according to claim 1, comprising the performing of the plausibility-checking, and the outputting of the plausibility-checked information.

7. The method according to claim 6, wherein in the plausibility-checking,

when the first class corresponds with the second class then producing the plausibility-checked information to indicate that the road condition is classified in the second class, and

when the first class differs from the second class then determining a probability value for the first class from the classification of the first surface area on the road surface at the later time, comparing the probability value to a threshold value, and producing the plausibility-checked information to indicate that the road condition is classified in the first class when the probability value exceeds the threshold value or to indicate that the road condition is classified in the second class when the probability value does not exceed the threshold value.

8. The method according to claim 6, wherein the plausibility-checked information indicates that the road condition is classified in the first class unless the plausibility-checking determines that the first class is an implausible outlier.

9. The method according to claim 1, comprising the performing of the merging, and the outputting of the merged information.

10. The method according to claim 1, further comprising determining a coefficient of friction of the first surface area on the road surface based on at least the output information.

11. The method according to claim 1, wherein the plural pre-determined classes include four classes representing the road surface condition respectively as wet, dry, snow-covered and icy.

12. The method according to claim 1, wherein the successive camera images further include plural successive images acquired at successive times before the later time, and wherein the method further comprises respectively classifying the first image area in the plural successive images to classify the first surface area on the road surface at the successive times respectively into one or more of the plural pre-determined classes, and performing temporal filtering on the classifications of the first surface area on the road surface at the successive times and at the later time before performing the merging or the plausibility-checking.

13. The method according to claim 12,

wherein the temporal filtering comprises comparing with one another the respective classifications of the first surface area at at least one of the successive times and at the later time,

when the classification of the first surface area at the later time corresponds to the classification of the first surface area at the at least one successive time then performing the merging or the plausibility-checking on the classification of the first surface area at the later time, and

when the classification of the first surface area at the later time differs from the classification of the first surface area at the at least one successive time then determining a probability value for the first class from the classification of the first surface area at the later time, and performing the merging or the plausibility-checking on the classification of the first surface area at the later time when the probability value exceeds a threshold value or performing the merging or the plausibility-checking on the classification of the first surface area at the at least one successive time when the probability value does not exceed the threshold value.

14. The method according to claim 1, wherein the camera is a mono camera.

15. The method according to claim 1, wherein the camera is a stereo camera.

16. An apparatus for performing the method according to claim 1 for determining the road condition of the road surface, wherein said apparatus comprises:

the camera configured to perform the acquiring of the successive camera images;

an image processor configured to perform the determining of the first and second image areas, the classification of the first and second image areas, and the merging or the plausibility-checking; and

an output unit configured to perform the outputting of the merged information or the plausibility-checked information.

17. A method of determining a road condition using a vehicle camera of a subject vehicle, comprising:

acquiring plural images with the vehicle camera;

determining, in the images, a first image area that includes an image of a road surface;

providing the first image area to a classifier, and with the classifier assigning the first image area to at least a first road condition class among plural road condition classes that respectively represent different road conditions;

determining, in the images, a second image area that includes an image of a section of the road surface that lies farther ahead than the first image area;

providing the second image area to the classifier, and with the classifier assigning the second image area to at least a second road condition class among the plural road condition classes;

checking a plausibility of the assignment of the first image area of a currently acquired one of the images to at least the first road condition class, relative to the assignment of the second image area of a previously acquired one of the images to at least the second road condition class; and

outputting information regarding at least one plausibility-checked road condition based on a result of the checking of the plausibility.

18. The method according to claim 17, wherein the first and second image areas do not overlap one another in any one of the plural images.

19. A method of determining a road condition using a vehicle camera of a subject vehicle, comprising:

acquiring plural images with the vehicle camera;

determining, in the images, a first image area that includes an image of a road surface;

providing the first image area to a classifier, and with the classifier assigning the first image area to at least a first road condition class among plural road condition classes that respectively represent different road conditions;

determining, in the images, a second image area that includes an image of a section of the road surface that lies farther ahead than the first image area;

providing the second image area to the classifier, and with the classifier assigning the second image area to at least a second road condition class among the plural road condition classes;

merging the assignment of the first image area of a currently acquired one of the images to at least the first road condition class, with the assignment of the second image area of a previously acquired one of the images to at least the second road condition class; and

outputting information regarding at least one merged road condition based on a result of the merging.

20. The method according to claim 19, wherein the first and second image areas do not overlap one another in any one of the plural images.