Method for Determining a State of a Pavement from Surroundings Sensor Data

• US 20150224925A1
• Filed: 12/09/2013
• Published: 08/13/2015
• Est. Priority Date: 12/20/2012
• Status: Active Grant

• Current Assignee: Continental Teves AG & Co. OHG (Continental AG)
• Original Assignee: Continental Teves AG & Co. OHG (Continental AG)
• Inventors: Hartmann, Bernd
• Granted Patent: US 9,676,331 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: B60W 2420/403 Image sensing, e.g. optical camera
• CPC Class Codes: B60W 2520/28 Wheel speed
• CPC Class Codes: B60W 40/068 Road friction coefficient
• CPC Class Codes: G06V 20/56 exterior to a vehicle by using sensors mounted on the vehicle
• CPC Class Codes: H04N 7/183 for receiving images from a single remote source

Abstract

The invention relates to a method/a device for determining a state of a pavement from surroundings sensor data.

For determining a state of a pavement from surroundings sensor data, a merger of data received from at least one device that measures a local pavement state or coefficient of friction with data received from a camera (4) for covering a pavement (1) extending in front of the vehicle is provided. To this end, when analyzing the camera image data, the locally measured pavement state or coefficient of friction may be assigned to individual image sectors of a camera image whilst taking odometric and time information into account and taken into account for the support and/or plausibilization of an anticipatory and locally resolved coefficient-of-friction estimation or state-of-pavement determination on the basis of camera data.

Claims

1. A method of determining a state of a pavement on which a vehicle is driving, comprising

merging locally measured data received from at least one locally measuring device that measures a local pavement state or coefficient of friction of the pavement, with camera data received from a camera (4), and

performing an image analysis comprising analyzing the camera data,

wherein the locally measured data represents the local pavement state or coefficient of friction which is respectively assigned to individual image sectors of a camera image in the camera data whilst taking odometric information of the vehicle and time information into account, and is taken into account for support and/or plausibilization of an anticipatory and locally resolved coefficient-of-friction estimation or state-of-pavement determination based on the camera data.

2. The method according to claim 1, wherein the image analysis includes assigning the local pavement state or coefficient of friction to at least one pavement segment in the camera image if consideration of the odometric information and the time information reveals that a predicted pavement state or coefficient of friction of this pavement segment has been locally measured in the locally measured data thereafter.

3. The method according to claim 1, wherein the image analysis provides a classification (K1, K2, . . . , Kn) of individual image sectors or pavement segments in the camera image based on particular features.

4. The method according to claim 1, wherein the camera image is subdivided into a two-dimensional grid (G) in a plane of the pavement and the local pavement state or coefficient of friction is assigned to a cell of the grid (G).

5. The method according to claim 4, wherein a total number of the cells of the grid (G) is determined by a homogeneity of the pavement (1).

6. The method according to claim 4, wherein a total number of the cells of the grid (G) is determined by a current driving situation and/or a criticality thereof.

7. The method according to claim 4, wherein a total number of the cells of the grid (G) is determined by an available computing power for performing the method.

8. The method according to claim 1, wherein a result of the image analysis of the camera data is predictively applied afterwards, whilst taking the local pavement state or coefficient of friction assigned to the camera image into account, to a subsequently acquired camera image.

9. The method according to claim 1, further comprising calculating a vehicle corridor from a predicted movement trajectory (T) of the vehicle (5), by means of which vehicle corridor positions of individual wheels (R1-R4) of the vehicle (5) and/or of a locally measuring sensor of the at least one locally measuring device are predictively assigned to pavement segments in the camera image, said pavement segments extending in front of the vehicle.

10. The method according to claim 1, further comprising assigning probability values to an image sector or a pavement segment, said probability values indicating with what probability the image sector or the pavement segment is to be assigned to a first class and to at least a second class.

11. The method according to claim 1, wherein a mono camera is used as the camera (4).

12. The method according to claim 1, wherein a stereo camera is used as the camera (4).

13. The method according to claim 1, wherein a sensor or an optical sensor (6) is used as the locally measuring device, said sensor or said optical sensor (6) locally determining a three-dimensional shape of a pavement surface of the pavement.

14. The method according to claim 1, wherein at least one measuring device that measures and/or derives a local coefficient of friction from speed signals of a vehicle wheel (R1-R4) is used as the locally measuring device.

15. A device for determining a state of a pavement on which a vehicle is driving, comprising a camera (4) that provides camera data, at least one locally measuring device configured to measure a local pavement state or coefficient of friction of the pavement, and a camera data analysis device configured to take the local pavement state or coefficient of friction into account during camera data analysis of the camera data, wherein, the camera data analysis device is further configured to assign the local pavement state or coefficient of friction to individual image sectors of a camera image of the camera data whilst taking odometric information of the vehicle and time information into account and to take the local pavement state or coefficient of friction into account for support and/or plausibilization of an anticipatory and locally resolved coefficient-of-friction estimation or state-of-pavement determination based on the camera data.

16. A method of determining a surface condition of a driving surface on which a vehicle is driving, comprising steps:

a) with a camera on said vehicle, producing camera data including a camera image of a selected surface area of said driving surface ahead in front of said vehicle;

b) performing an image analysis of said camera image using an image analysis algorithm to determine an estimated surface condition comprising an estimated pavement state or an estimated coefficient of friction of said selected surface area;

c) driving said vehicle forward whereby said selected surface area comes into a sensing range of a locally measuring sensor on said vehicle, and using time information and odometric information of said vehicle to achieve sensing registration of said locally measuring sensor with said selected surface area;

d) with said locally measuring sensor, sensing locally measured data representing an actual surface condition comprising an actual pavement state or an actual coefficient of friction of said selected surface area;

e) comparing said actual surface condition with said estimated surface condition, and dependent on any discrepancy therebetween updating said image analysis algorithm in a manner that would reduce said discrepancy; and

f) repeating said steps a) to e), using said updated image analysis algorithm in said step b), with regard to a subsequent selected surface area of said driving surface ahead in front of said vehicle.