ROAD VERTICAL CONTOUR DETECTION USING A STABILIZED COORDINATE FRAME

• US 20190034739A1
• Filed: 10/01/2018
• Published: 01/31/2019
• Est. Priority Date: 02/26/2015
• Status: Active Grant

• Current Assignee: MobilEye Vision Technologies Ltd. (Intel Corporation)
• Original Assignee: MobilEye Vision Technologies Ltd. (Intel Corporation)
• Inventors: Stein, Gideon, Blumenthal, Itay, Shaag, Nadav, Kingsley, Yaakov
• Granted Patent: US 10,452,928 B2
• Time in Patent Office: Days
• CPC Class Codes: G06T 2207/30252 Vehicle exterior; Vicinity of vehicle
• CPC Class Codes: G06T 5/80 Geometric correction
• CPC Class Codes: G06T 7/13 Edge detection
• CPC Class Codes: G06T 7/20 Analysis of motion motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51
• CPC Class Codes: G06T 7/40 Analysis of texture depth or shape recovery from texture G06T7/529
• CPC Class Codes: G06T 7/60 Analysis of geometric attributes
• CPC Class Codes: G06T 7/74 involving reference images or patches
• CPC Class Codes: G06V 20/56 exterior to a vehicle by using sensors mounted on the vehicle
• CPC Class Codes: G06V 20/588 Recognition of the road, e.g. of lane markings; Recognition of the vehicle driving pattern in relation to the road
• CPC Class Codes: H04N 7/18 Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast

Abstract

In some embodiments, a first homography, created from two images of a roadway, is decomposed to determine an ego-motion, and the ego-motion is used to adjust a previous estimate of a road plane. The adjusted previous estimate of the road plane is combined with the current estimate of the plane to create a second homography, and the second homography is used to determine residual motion and vertical deviation in the surface of the roadway. In some embodiments, multiple road profiles each corresponding to a common portion of a roadway are adjusted in slope and offset by optimizing a function having a data term, a smoothness term and a regularization term; and the adjusted road profiles are combined into a multi-frame road profile. In some embodiments, road profile information for a predetermined number of data points is transmitted in periodic data bursts, with more than one data point per data burst.

Claims

1. A method of transmitting road profile information in a driver-assistance system mounted in a vehicle, wherein the driver assistance system comprises a camera operatively connectible to a processor and a data interface, the method comprising:

selecting a first distance in front of a wheel of the vehicle,

sampling a predetermined number of data points from a road profile along a projected path of the vehicle, wherein the road profile is based at least in part on images captured by the camera, and wherein the data points are sampled along a segment of the path with an endpoint corresponding to the selected first distance, and wherein one or more of the data points is generated in accordance with a stabilized reference plane, and

transmitting road profile information corresponding to the predetermined number of data points in a predetermined number of data transmissions, via the data interface, wherein the number of data transmissions is fewer than the number of data points.

2. The method of claim 1, comprising:

selecting a frequency at which data is transmitted via the data interface, wherein the selected frequency defines the frequency with which the predetermined number of data transmissions are transmitted.

3. The method of claim 1, comprising:

estimating a second distance traveled by the vehicle in a time period equal to an inverse of the selected frequency, wherein the data points are sampled along a segment of the path corresponding to the estimated second distance beyond the selected first distance.

4. The method of claim 3, wherein estimating the second distance comprises determining that the second distance will be traversed in the period at a known current speed of the vehicle.

5. The method of claim 1, wherein the road profile information transmitted over the data interface is configured such that the road profile information corresponding to a respective data point may be individually read by a receiving system and correlated with a respective portion of a road represented by the road profile.

6. The method of claim 1, wherein selecting the first distance in front of the wheel of the vehicle comprises accounting for system latency.

7. The method of claim 1, wherein one or more of the data points are associated with a respective confidence value.

8. The method of claim 1, wherein a spacing of the data points is determined such that it estimated that the vehicle will travel from one data point to a next data point in a predetermined time period.

9. The method of claim 1, wherein one or more of the data points is generated at least in part by combining profile information from multiple frames.

10. The method of claim 1, further comprising:

associating a confidence value with a point in the road profile, and

transmitting, to a vehicle control system, data corresponding to the point in the road profile and to the confidence value associated with the point in the road profile.

11. The method of claim 10, wherein the data corresponding to the point in the road profile comprises a height of the roadway.

12. The method of claim 10, wherein the data corresponding to the point in the road profile comprises a distance in front of the vehicle.

13. The method of claim 10, wherein associating a confidence value with a point in the road profile comprises determining the confidence value in accordance with a forward/backward verification performed between two of the multiple images.

14. The method of claim 10, wherein associating a confidence value with a point in the road profile comprises determining the confidence value in accordance with a curvature of a score matrix associated with the comparison of two images spaced in time.

15. The method of claim 10, comprising:

transmitting, with the data corresponding to the point in the road profile and to the confidence value associated with the point in the road profile, a multi-frame confidence value, wherein the multi-frame confidence value is determined in accordance with a number of height measurements from different images for the point in the road profiles, and the variance of the heights measurements.

16. A driver assistance system mounted in a vehicle, the system comprising:

a processor,

a camera operatively connectible to the processor and a data interface, and

a memory storing instructions that, when executed by the processor, cause the system to;

select a first distance in front of a wheel of the vehicle,

sample a predetermined number of data points from a road profile along a projected path of the vehicle, wherein the road profile is based at least in part on images captured by the camera, and wherein the data points are sampled along a segment of the path with an endpoint corresponding to the selected first distance, and wherein one or more of the data points is generated in accordance with a stabilized reference plane, and

transmit road profile information corresponding to the predetermined number of data points in a predetermined number of data transmissions, via the data interface, wherein the number of data transmissions is fewer than the number of data points.

17. The system of claim 16, further comprising instructions that cause the system to:

associate a confidence value with a point in the road profile, and

transmit, to a vehicle control system, data corresponding to the point in the road profile and to the confidence value associated with the point in the road profile.

20. The system of claim 16, further comprising instructions that cause the system to:

select a frequency at which data is transmitted via the data interface, wherein the selected frequency defines the frequency with which the predetermined number of data transmissions are transmitted.

21. The system of claim 16, wherein one or more of the data points are associated with a respective confidence value.

22. The system of claim 16, wherein one or more of the data points is generated at least in part by combining profile information from multiple frames.

18. A non-transitory computer readable storage medium storing instructions that, when executed by a processor operatively coupled to a driver assistance system mounted in a host vehicle, cause the system to:

select a first distance in front of a wheel of the vehicle,

sample a predetermined number of data points from a road profile along a projected path of the vehicle, wherein the road profile is based at least in part on images captured by a camera operatively connectible to the processor, and wherein the data points are sampled along a segment of the path with an endpoint corresponding to the selected first distance, and wherein one or more of the data points is generated in accordance with a stabilized reference plane, and

transmit road profile information corresponding to the predetermined number of data points in a predetermined number of data transmissions, via a data interface, wherein the number of data transmissions is fewer than the number of data points.

19. The non-transitory computer readable storage medium of claim 18, further comprising instructions that cause the system to:

associate a confidence value with a point in the road profile, and

transmit, to a vehicle control system, data corresponding to the point in the road profile and to the confidence value associated with the point in the road profile.