ROAD VERTICAL CONTOUR DETECTION USING A STABILIZED COORDINATE FRAME
First Claim
1. A computerized road surface deviation detection method, the method performed by a driver assistance system mounted in a vehicle, wherein the driver assistance system comprises a camera operatively connectible to a processor, the method comprising:
- capturing multiple images via the camera, the multiple images comprising a first image of the road surface captured at a first time and a second image of the road surface captured at a second time, the second time being subsequent to the first time,determining, based on at least the first image and the second image, a first estimate of a plane normal of the road and a second estimate of the plane normal of the road,creating a model of the road surface based on at least the first estimate of the plane normal and the second estimate of the plane normal,using the model of the road surface to determine a residual motion along a projected path of the vehicle,computing a vertical deviation of the road surface based on the residual motion, andtransmitting vertical deviation data to a vehicle control system.
1 Assignment
0 Petitions
Accused Products
Abstract
In some embodiments, a first homography, created from two images of a roadway, is decomposed to determine an ego-motion, and the ego-motion is used to adjust a previous estimate of a road plane. The adjusted previous estimate of the road plane is combined with the current estimate of the plane to create a second homography, and the second homography is used to determine residual motion and vertical deviation in the surface of the roadway. In some embodiments, multiple road profiles each corresponding to a common portion of a roadway are adjusted in slope and offset by optimizing a function having a data term, a smoothness term and a regularization term; and the adjusted road profiles are combined into a multi-frame road profile. In some embodiments, road profile information for a predetermined number of data points is transmitted in periodic data bursts, with more than one data point per data burst.
40 Citations
52 Claims
-
1. A computerized road surface deviation detection method, the method performed by a driver assistance system mounted in a vehicle, wherein the driver assistance system comprises a camera operatively connectible to a processor, the method comprising:
-
capturing multiple images via the camera, the multiple images comprising a first image of the road surface captured at a first time and a second image of the road surface captured at a second time, the second time being subsequent to the first time, determining, based on at least the first image and the second image, a first estimate of a plane normal of the road and a second estimate of the plane normal of the road, creating a model of the road surface based on at least the first estimate of the plane normal and the second estimate of the plane normal, using the model of the road surface to determine a residual motion along a projected path of the vehicle, computing a vertical deviation of the road surface based on the residual motion, and transmitting vertical deviation data to a vehicle control system. - View Dependent Claims (2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12)
-
-
13. A driver assistance system mounted in a vehicle, the system comprising:
-
a processor, a camera operatively connectible to the processor, and a memory storing instructions that, when executed by the processor, cause the system to; capture multiple images via the camera, the multiple images comprising a first image of the road surface captured at a first time and a second image of the road surface captured at a second time, the second time being subsequent to the first time, determine, based on at least the first image and the second image, a first estimate of a plane normal of the road and a second estimate of the plane normal of the road, create a model of the road surface based on at least the first estimate of the plane normal and the second estimate of the plane normal, use the model of the road surface to determine a residual motion along a projected path of the vehicle, compute a vertical deviation of the road surface based on the residual motion, and transmit vertical deviation data to a vehicle control system.
-
-
14. A non-transitory computer readable storage medium storing instructions that, when executed by a processor operatively coupled to a driver assistance system mounted in a host vehicle, cause the system to:
-
capture multiple images via a camera operatively connectible to the processor, the multiple images comprising a first image of the road surface captured at a first time and a second image of the road surface captured at a second time, the second time being subsequent to the first time, determine, based on at least the first image and the second image, a first estimate of a plane normal of the road and a second estimate of the plane normal of the road, create a model of the road surface based on at least the first estimate of the plane normal and the second estimate of the plane normal, use the model of the road surface to determine a residual motion along a projected path of the vehicle, compute a vertical deviation of the road surface based on the residual motion, and transmit vertical deviation data to a vehicle control system.
-
-
15. A computerized road surface deviation detection method, the method performed by a driver assistance system mounted in a vehicle, wherein the driver assistance system comprises a camera operatively connectible to a processor, the method comprising:
-
selecting at least two road profiles, wherein each of the at least two road profiles comprises a common portion of a road, optimizing a function comprising a data term, a smoothness term, and a regularization term to align a first profile from the at least two profiles, and combining the at least two road profiles into a multi-frame profile. - View Dependent Claims (16, 17, 18)
-
-
19. A driver assistance system mounted in a vehicle, the system comprising:
-
a processor, a camera operatively connectible to the processor, and a memory storing instructions that, when executed by the processor, cause the system to; select at least two road profiles, wherein each of the at least two road profiles comprises a common portion of a road, optimize a function comprising a data term, a smoothness term, and a regularization term to align a first profile from the at least two profiles, and combine the at least two road profiles into a multi-frame profile.
-
-
20. A non-transitory computer readable storage medium storing instructions that, when executed by a processor operatively coupled to a driver assistance system mounted in a host vehicle, cause the system to:
-
select at least two road profiles, wherein each of the at least two road profiles comprises a common portion of a road, optimize a function comprising a data term, a smoothness term, and a regularization term to align a first profile from the at least two profiles, and combine the at least two road profiles into a multi-frame profile.
-
-
21. A method of transmitting road profile information in a driver-assistance system mounted in a vehicle, wherein the driver assistance system comprises a camera operatively connectible to a processor and a data interface, the method comprising:
-
selecting a first distance in front of a wheel of the vehicle, sampling a predetermined number of data points from a road profile along a projected path of the vehicle, wherein the road profile is based at least in part on images captured by the camera, and wherein the data points are sampled along a segment of the path with an endpoint corresponding to the selected first distance, and wherein one or more of the data points is generated in accordance with a stabilized reference plane, and transmitting road profile information corresponding to the predetermined number of data points in a predetermined number of data transmissions, via the data interface, wherein the number of data transmissions is fewer than the number of data points. - View Dependent Claims (22, 23, 24, 25, 26, 27, 28, 29)
-
-
30. A driver assistance system mounted in a vehicle, the system comprising:
-
a processor, a camera operatively connectible to the processor, and a memory storing instructions that, when executed by the processor, cause the system to; select a first distance in front of a wheel of the vehicle, sample a predetermined number of data points from a road profile along a projected path of the vehicle, wherein the road profile is based at least in part on images captured by the camera, and wherein the data points are sampled along a segment of the path with an endpoint corresponding to the selected first distance, and wherein one or more of the data points is generated in accordance with a stabilized reference plane, and transmit road profile information corresponding to the predetermined number of data points in a predetermined number of data transmissions, via the data interface, wherein the number of data transmissions is fewer than the number of data points.
-
-
31. A non-transitory computer readable storage medium storing instructions that, when executed by a processor operatively coupled to a driver assistance system mounted in a host vehicle, cause the system to:
-
select a first distance in front of a wheel of the vehicle, sample a predetermined number of data points from a road profile along a projected path of the vehicle, wherein the road profile is based at least in part on images captured by a camera operatively connectible to the processor, and wherein the data points are sampled along a segment of the path with an endpoint corresponding to the selected first distance, and wherein one or more of the data points is generated in accordance with a stabilized reference plane, and transmit road profile information corresponding to the predetermined number of data points in a predetermined number of data transmissions, via the data interface, wherein the number of data transmissions is fewer than the number of data points.
-
-
32. A computerized road surface deviation detection method, the method performed by a driver assistance system mounted in a vehicle, wherein the driver assistance system comprises a camera operatively connectible to a processor, the method comprising:
-
capturing, via the camera, multiple images depicting a portion of a roadway, determining whether a characteristic of the images is attributable to a shadow cast by a tree branch, in accordance with the determination that a characteristic of the image is not attributable to a shadow cast by a tree branch, processing the characteristic to determine vertical deviations in the portion of the roadway, and in accordance with the determination that a characteristic of the image is attributable to a shadow cast by a tree branch, decreasing consideration of the characteristic in determining vertical deviations in the portion of the roadway. - View Dependent Claims (33, 34, 35, 36, 37, 38)
-
-
39. A driver assistance system mounted in a vehicle, the system comprising:
-
a processor, a camera operatively connectible to the processor, and a memory storing instructions that, when executed by the processor, cause the system to; capture, via the camera, multiple images depicting a portion of a roadway, determine whether a characteristic of the images is attributable to a shadow cast by a tree branch, in accordance with the determination that a characteristic of the image is not attributable to a shadow cast by a tree branch, process the characteristic to determine vertical deviations in the portion of the roadway, and in accordance with the determination that a characteristic of the image is attributable to a shadow cast by a tree branch, decrease consideration of the characteristic in determining vertical deviations in the portion of the roadway.
-
-
40. A non-transitory computer readable storage medium storing instructions that, when executed by a processor operatively coupled to a driver assistance system mounted in a host vehicle, cause the system to:
-
capture, via a camera operatively connectible to the processor, multiple images depicting a portion of a roadway, determine whether a characteristic of the images is attributable to a shadow cast by a tree branch, in accordance with the determination that a characteristic of the image is not attributable to a shadow cast by a tree branch, process the characteristic to determine vertical deviations in the portion of the roadway, and in accordance with the determination that a characteristic of the image is attributable to a shadow cast by a tree branch, decrease consideration of the characteristic in determining vertical deviations in the portion of the roadway.
-
-
41. A computerized road surface deviation detection method, the method performed by a driver assistance system mounted in a vehicle, wherein the driver assistance system comprises a single camera operatively connectible to a processor, the method comprising:
-
capturing, via the camera, multiple images depicting a portion of a roadway, generating a road profile based at least in part on one or more of the multiple images of the roadway, associating a confidence value with a point in the road profile, and transmitting, to a vehicle control system, data corresponding to the point in the road profile and to the confidence value associated with the point in the road profile. - View Dependent Claims (42, 43, 44, 45, 46)
-
-
47. A driver assistance system mounted in a vehicle, the system comprising:
-
a processor, a single camera operatively connectible to the processor, and a memory storing instructions that, when executed by the processor, cause the system to; capture, via the camera, multiple images depicting a portion of a roadway, generate a road profile based at least in part on one or more of the multiple images of the roadway, associate a confidence value with a point in the road profile, and transmit, to a vehicle control system, data corresponding to the point in the road profile and to the confidence value associated with the point in the road profile.
-
-
48. A non-transitory computer readable storage medium storing instructions that, when executed by a processor operatively coupled to a driver assistance system mounted in a host vehicle, cause the system to:
-
capture, via a single camera operatively connectible to the processor, multiple images depicting a portion of a roadway, generate a road profile based at least in part on one or more of the multiple images of the roadway, associate a confidence value with a point in the road profile, and transmit, to a vehicle control system, data corresponding to the point in the road profile and to the confidence value associated with the point in the road profile.
-
-
49. A computerized road surface deviation detection method, the method performed by a driver assistance system mounted in a vehicle, wherein the driver assistance system comprises a camera, having rolling-shutter and radial distortion, operatively connectible to a processor, the method comprising:
-
capturing, via the camera, multiple images depicting a portion of a roadway, compensating for the effects of rolling shutter and radial distortion, generating, based on at least the multiple images depicting the portion of the roadway, a profile of the portion of the roadway, transmitting information regarding the profile of the portion of the roadway to a vehicle control system. - View Dependent Claims (50)
-
-
51. A driver assistance system mounted in a vehicle, the system comprising:
-
a processor, a camera, having rolling-shutter and radial distortion, operatively connectible to the processor, and a memory storing instructions that, when executed by the processor, cause the system to; capture, via the camera, multiple images depicting a portion of a roadway, compensate for the effects of rolling shutter and radial distortion, generate, based on at least the multiple images depicting the portion of the roadway, a profile of the portion of the roadway, transmit information regarding the profile of the portion of the roadway to a vehicle control system.
-
-
52. A non-transitory computer readable storage medium storing instructions that, when executed by a processor operatively coupled to a driver assistance system mounted in a host vehicle, cause the system to:
-
capture, via a camera operatively connectible to the processor, multiple images depicting a portion of a roadway, the camera having rolling-shutter and radial distortion, compensate for the effects of rolling shutter and radial distortion, generate, based on at least the multiple images depicting the portion of the roadway, a profile of the portion of the roadway, transmit information regarding the profile of the portion of the roadway to a vehicle control system.
-
Specification