ROAD PLANE OUTPUT WITH LATERAL SLOPE

• US 20170270372A1
• Filed: 03/15/2017
• Published: 09/21/2017
• Est. Priority Date: 03/15/2016
• Status: Active Grant

• Current Assignee: MobilEye Vision Technologies Ltd. (Intel Corporation)
• Original Assignee: MobilEye Vision Technologies Ltd. (Intel Corporation)
• Inventors: STEIN, Gideon
• Granted Patent: US 10,210,402 B2
• Time in Patent Office: Days
• CPC Class Codes: B60W 10/20 including control of steering systems
• CPC Class Codes: B60W 10/22 including control of suspension systems
• CPC Class Codes: B60W 2420/403 Image sensing, e.g. optical camera
• CPC Class Codes: B60W 2552/15 Road slope, i.e. the inclination of a road segment in the longitudinal direction
• CPC Class Codes: B60W 2552/35 Road bumpiness, e.g. potholes
• CPC Class Codes: B60W 2900/00 Indexing codes relating to the purpose of, or problem solved of road vehicle drive control systems not otherwise provided for in groups B60W30/00
• CPC Class Codes: B60W 30/02 Control of vehicle driving stability
• CPC Class Codes: G06T 2207/10004 Still image; Photographic image
• CPC Class Codes: G06T 2207/20112 Image segmentation details
• CPC Class Codes: G06T 2207/30256 Lane; Road marking
• CPC Class Codes: G06T 7/11 Region-based segmentation
• CPC Class Codes: G06T 7/248 involving reference images or patches
• CPC Class Codes: G06T 7/60 Analysis of geometric attributes
• CPC Class Codes: G06V 10/751 Comparing pixel values or logical combinations thereof, or feature values having positional relevance, e.g. template matching
• CPC Class Codes: G06V 20/56 exterior to a vehicle by using sensors mounted on the vehicle
• CPC Class Codes: G06V 20/588 Recognition of the road, e.g. of lane markings; Recognition of the vehicle driving pattern in relation to the road
• CPC Class Codes: H04N 7/18 Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast

Abstract

The present disclosure generally relates to processing visual data of a road surface that includes a vertical deviation with a lateral slope. In some embodiments, a system determines a path expected to be traversed by at least one wheel of the vehicle on a road surface. In some embodiments, a system determines, using at least two images captured by one or more cameras, a height of the road surface for at least one point along the path to be traversed by the wheel. In some embodiments, a system computes an indication of a lateral slope of the road surface at the at least one point along the path. In some embodiments, a system outputs, on a vehicle interface bus, an indication of the height of the point and an indication of the lateral slope at the at least one point along the path.

Claims

1. A method for processing visual data of a road surface, the method comprising:

determining a path expected to be traversed by at least one wheel of the vehicle on a road surface;

determining, using at least two images captured by one or more cameras on the vehicle, a height of the road surface for at least one point along the path to be traversed by the wheel;

computing an indication of a lateral slope of the road surface at the at least one point along the path; and

outputting, on a vehicle interface bus on the vehicle, an indication of the height of the point and an indication of the lateral slope at the at least one point along the path.

2. The method according to claim 1, further comprising:

accessing visual data representing the road surface;

determining an initial road profile of the road surface, wherein the road profile is derived from residual motion along a vehicle path associated with the road surface;

segmenting a portion of the visual data representing the road surface into a first segment and a second segment, wherein the portion of the visual data that is segmented includes visual data representing a vertical deviation on the road surface;

determining a first segment road profile for the first segment of the portion of the visual data;

determining a second segment road profile for the second segment of the portion of the visual data;

comparing one or more of the first segment road profile, the second segment road profile, and the initial road profile; and

based at least in part on results of the comparison, outputting the indication of the height of the point and the indication of the lateral slope at the at least one point along the path.

3. The method according to claim 2, wherein outputting an indication of the height of the point and an indication of the lateral slope at the at least one point along the path comprises outputting one or more of:

a slope measurement associated with the vertical deviation on the road surface;

a direction of a slope associated with the vertical deviation on the road surface; and

a confidence value for a residual motion or height value associated with a road profile associated with the vertical deviation on the road surface.

4. The method according to claim 2, wherein the first segment road profile, the second segment road profile, and the initial road profile each include data derived from residual motion along a path associated with the road surface, and wherein comparing the road profiles comprises:

determining whether residual motion at a point along the path of each of the road profiles is within a threshold value of each other.

5. The method according to claim 2, further comprising:

determining a lateral slope of the vertical deviation based at least in part on residual motion data of the road profile for the first segment and residual motion data of the road profile for the second segment.

6. The method according to claim 5, wherein determining a lateral slope comprises:

determining an estimated height for the vertical deviation based on one or more of the first segment road profile, the second segment road profile, and the initial road profile; and

calculating the lateral slope based at least in part on the estimated height.

7. The method according to claim 2, further comprising:

determining a confidence score for one or more of the road profiles based in part on a result of the comparison of the first segment road profile, the second segment road profile, and the initial road profile.

8. The method according to claim 2, wherein determining a road profile comprises:

determining a residual motion for a sequence of pixels in visual data representing the road surface.

9. The method according to claim 8, wherein the sequence of pixels is a line of pixels along the center of the visual data representing the road profile.

10. The method according to claim 2, wherein the first segment of the portion of the visual data is different than the second segment of the portion of the visual data.

11. The method according to claim 2, further comprising:

adjusting a vehicle component setting based in part on a result of the comparison of the first segment road profile, the second segment road profile, and the initial road profile.

12. The method according to claim 11, wherein adjusting a vehicle component setting comprises:

adjusting a vehicle steering setting to modify a vehicle path, wherein the modified vehicle path avoids passing over the vertical deviation on the road surface.

13. The method according to claim 11, wherein adjusting a vehicle component setting comprises adjusting a vehicle suspension setting.

14. The method according to claim 2, further comprising:

determining that a vehicle path passes over the vertical deviation on the road surface, wherein the vertical deviation is represented in the initial road profile by an elevated residual motion; and

in accordance with a determination that the vehicle path passes over the vertical deviation on the road surface;

segmenting the portion of the visual data representing the road surface into the first segment and the second segment.

15. A system mounted on a vehicle, the system comprising:

one or more cameras; and

one or more processors, wherein the one or more processors are configured to execute instructions for;

determining a path expected to be traversed by at least one wheel of the vehicle on a road surface;

determining, using at least two images captured by the one or more cameras, a height of the road surface for at least one point along the path to be traversed by the wheel;

computing an indication of a lateral slope of the road surface at the at least one point along the path; and

outputting, on a vehicle interface bus on the vehicle, an indication of the height of the point and an indication of the lateral slope at the at least one point along the path.

16. The system according to claim 15, the processor further configured to execute instructions for:

accessing visual data representing the road surface;

determining an initial road profile of the road surface, wherein the road profile is derived from residual motion along a vehicle path associated with the road surface;

segmenting a portion of the visual data representing the road surface into a first segment and a second segment, wherein the portion of the visual data that is segmented includes visual data representing a vertical deviation on the road surface;

determining a first segment road profile for the first segment of the portion of the visual data;

determining a second segment road profile for the second segment of the portion of the visual data;

comparing one or more of the first segment road profile, the second segment road profile, and the initial road profile; and

based at least in part on results of the comparison, outputting the indication of the height of the point and the indication of the lateral slope at the at least one point along the path.

17. The system according to claim 16, wherein outputting an indication of the height of the point and an indication of the lateral slope at the at least one point along the path comprises outputting one or more of:

a slope measurement associated with the vertical deviation on the road surface;

a direction of a slope associated with the vertical deviation on the road surface; and

a confidence value for a residual motion or height value associated with a road profile associated with the vertical deviation on the road surface.

18. The system according to claim 16, wherein the second segment road profile, and the initial road profile each include data derived from residual motion along a path associated with the road surface, and wherein comparing the road profiles comprises:

determining whether residual motion at a point along the path of each of the road profiles is within a threshold value of each other.

19. The system according to claim 16, the one or more programs further including instructions for:

determining a lateral slope of the vertical deviation based at least in part on residual motion data of the road profile for the first segment and residual motion data of the road profile for the second segment.

20. The system according to claim 19, wherein determining a lateral slope comprises:

determining an estimated height for the vertical deviation based on one or more of the first segment road profile, the second segment road profile, and the initial road profile; and

calculating the lateral slope based at least in part on the estimated height.

21. The system according to claim 16, the one or more programs further including instructions for:

determining a confidence score for one or more of the road profiles based in part on a result of the comparison of the first segment road profile, the second segment road profile, and the initial road profile.

22. The system according to claim 16, wherein determining a road profile comprises:

determining a residual motion for a sequence of pixels in visual data representing the road surface.

23. The system according to claim 22, wherein the sequence of pixels is a line of pixels along the center of the visual data representing the road profile.

24. The system according to claim 16, wherein the first segment of the portion of the visual data is different than the second segment of the portion of the visual data.

25. The system according to claim 16, the one or more programs further including instructions for:

adjusting a vehicle component setting based in part on a result of the comparison of the first segment road profile, the second segment road profile, and the initial road profile.

26. The system according to claim 25, wherein adjusting a vehicle component setting comprises:

adjusting a vehicle steering setting to modify a vehicle path, wherein the modified vehicle path avoids passing over the vertical deviation on the road surface.

27. The system according to claim 25, wherein adjusting a vehicle component setting comprises adjusting a vehicle suspension setting.

28. The system according to claim 16, the one or more programs further including instructions for:

determining that a vehicle path passes over the vertical deviation on the road surface, wherein the vertical deviation is represented in the initial road profile by an elevated residual motion; and

in accordance with a determination that the vehicle path passes over the vertical deviation on the road surface;

segmenting the portion of the visual data representing the road surface into the first segment and the second segment.

29. A non-transitory computer-readable storage medium storing one or more programs, the one or more programs comprising instructions, which, when executed by one or more processors of a vehicle, cause the processors to:

determine a path expected to be traversed by at least one wheel of the vehicle on a road surface;

determine, using at least two images captured by the one or more cameras on the vehicle, a height of the road surface for at least one point along the path to be traversed by the wheel;

compute an indication of a lateral slope of the road surface for the at least one point along the path; and

output, on a vehicle interface bus on the vehicle, an indication of the height of the point and an indication of the lateral slope at the at least one point along the path.

30. The storage medium according to claim 29, the one or more programs further including instructions for:

accessing visual data representing the road surface;

determining an initial road profile of the road surface, wherein the road profile is derived from residual motion along a vehicle path associated with the road surface;

segmenting a portion of the visual data representing the road surface into a first segment and a second segment, wherein the portion of the visual data that is segmented includes visual data representing a vertical deviation on the road surface;

determining a first segment road profile for the first segment of the portion of the visual data;

determining a second segment road profile for the second segment of the portion of the visual data;

comparing one or more of the first segment road profile, the second segment road profile, and the initial road profile; and

based at least in part on results of the comparison, outputting the indication of the height of the point and the indication of the lateral slope at the at least one point along the path.

31. The storage medium according to claim 30, wherein outputting an indication of the height of the point and an indication of the lateral slope at the at least one point along the path comprises outputting one or more of:

a slope measurement associated with the vertical deviation on the road surface;

a direction of a slope associated with the vertical deviation on the road surface; and

a confidence value for a residual motion or height value associated with a road profile associated with the vertical deviation on the road surface.

32. The storage medium according to claim 30, wherein the second segment road profile, and the initial road profile each include data derived from residual motion along a path associated with the road surface, and wherein comparing the road profiles comprises:

determining whether residual motion at a point along the path of each of the road profiles is within a threshold value of each other.

33. The storage medium according to claim 30, the one or more programs further including instructions for:

determining a lateral slope of the vertical deviation based at least in part on residual motion data of the road profile for the first segment and residual motion data of the road profile for the second segment.

34. The storage medium according to claim 33, wherein determining a lateral slope comprises:

determining an estimated height for the vertical deviation based on one or more of the first segment road profile, the second segment road profile, and the initial road profile; and

calculating the lateral slope based at least in part on the estimated height.

35. The storage medium according to claim 30, the one or more programs further including instructions for:

determining a confidence score for one or more of the road profiles based in part on a result of the comparison of the first segment road profile, the second segment road profile, and the initial road profile.

36. The storage medium according to claim 30, wherein determining a road profile comprises:

determining a residual motion for a sequence of pixels in visual data representing the road surface.

37. The storage medium according to claim 36, wherein the sequence of pixels is a line of pixels along the center of the visual data representing the road profile.

38. The storage medium according to claim 30, wherein the first segment of the portion of the visual data is different than the second segment of the portion of the visual data.

39. The storage medium according to claim 30, the one or more programs further including instructions for:

adjusting a vehicle component setting based in part on a result of the comparison of the first segment road profile, the second segment road profile, and the initial road profile.

40. The storage medium according to claim 39, wherein adjusting a vehicle component setting comprises:

adjusting a vehicle steering setting to modify a vehicle path, wherein the modified vehicle path avoids passing over the vertical deviation on the road surface.

41. The storage medium according to claim 39, wherein adjusting a vehicle component setting comprises adjusting a vehicle suspension setting.

42. The storage medium according to claim 30, the one or more programs further including instructions for:

determining that a vehicle path passes over the vertical deviation on the road surface, wherein the vertical deviation is represented in the initial road profile by an elevated residual motion; and

in accordance with a determination that the vehicle path passes over the vertical deviation on the road surface;

segmenting the portion of the visual data representing the road surface into the first segment and the second segment.