Fused raised pavement marker detection for autonomous driving using lidar and camera
First Claim
1. An in-vehicle system for estimating and tracking a lane boundary formed of pavement markers disposed on a roadway, the system comprising:
- a camera located on a vehicle for obtaining image data along the roadway regarding stationary reflective raised pavement markers and non-reflective raised pavement markers as the vehicle advances along the roadway;
an image processor for processing frames of image data captured along the roadway over time by the camera and generating a first probabilistic model for the lane boundary;
a lidar detector located on the vehicle for obtaining lidar data along the roadway regarding stationary reflective raised pavement markers as the vehicle advances along the roadway;
a lidar processor for processing frames of lidar data captured along the roadway over time by the lidar detector and generating a second probabilistic model for the lane boundary; and
means for fusing the first probabilistic model and the second probabilistic model to generate a fused probabilistic model and for estimating and tracking the lane boundary based on the fused probabilistic model;
wherein the first probabilistic model includes a first mixture of Gaussian distributions;
the second probabilistic model includes a second mixture of Gaussian distributions; and
the means for fusing the first probabilistic model and the second probabilistic model includes a processor and a non-transitory data storage on which is stored computer code which, when executed on the processor, causes the in-vehicle system to compute a product of the first probabilistic model and the second probabilistic model.
1 Assignment
0 Petitions
Accused Products
Abstract
An in-vehicle system for estimating a lane boundary based on raised pavement markers that mark the boundary. The in-vehicle system includes a camera for obtaining image data regarding reflective raised pavement markers and non-reflective raised pavement markers, an image processor for processing frames of image data captured by the camera, a lidar detector for obtaining lidar data regarding reflective raised pavement markers, and a lidar processor for processing frames of lidar data captured by the lidar detector. The image processor generates a first probabilistic model for the lane boundary and the lidar processor generates a second probabilistic model for the lane boundary. The in-vehicle system fuses the first probabilistic model and the second probabilistic model to generate a fused probabilistic model and estimates the lane boundary based on the fused probabilistic model.
-
Citations
16 Claims
-
1. An in-vehicle system for estimating and tracking a lane boundary formed of pavement markers disposed on a roadway, the system comprising:
-
a camera located on a vehicle for obtaining image data along the roadway regarding stationary reflective raised pavement markers and non-reflective raised pavement markers as the vehicle advances along the roadway; an image processor for processing frames of image data captured along the roadway over time by the camera and generating a first probabilistic model for the lane boundary; a lidar detector located on the vehicle for obtaining lidar data along the roadway regarding stationary reflective raised pavement markers as the vehicle advances along the roadway; a lidar processor for processing frames of lidar data captured along the roadway over time by the lidar detector and generating a second probabilistic model for the lane boundary; and means for fusing the first probabilistic model and the second probabilistic model to generate a fused probabilistic model and for estimating and tracking the lane boundary based on the fused probabilistic model; wherein the first probabilistic model includes a first mixture of Gaussian distributions; the second probabilistic model includes a second mixture of Gaussian distributions; and the means for fusing the first probabilistic model and the second probabilistic model includes a processor and a non-transitory data storage on which is stored computer code which, when executed on the processor, causes the in-vehicle system to compute a product of the first probabilistic model and the second probabilistic model. - View Dependent Claims (2, 3, 4)
-
-
5. An in-vehicle system for estimating a lane boundary on a roadway, the system comprising:
-
one or more detectors located on a vehicle that capture image data and lidar data of raised pavement markers along the roadway over time as the vehicle advances along the roadway; a processor and a non-transitory data storage on which is stored computer code which, when executed on the processor, causes the in-vehicle system to; generate a first probabilistic model for the lane boundary based on the image data; generate a second probabilistic model for the lane boundary based on the lidar data; fuse the first probabilistic model and the second probabilistic model to generate a fused probabilistic model; and estimate and track the lane boundary based on the fused probabilistic model; wherein the first probabilistic model includes a first mixture of Gaussian distributions; the second probabilistic model includes a second mixture of Gaussian distributions; and the means for fusing the first probabilistic model and the second probabilistic model includes a processor and a non-transitory data storage on which is stored computer code which, when executed on the processor, causes the in-vehicle system to compute a product of the first probabilistic model and the second probabilistic model; and wherein the computer code, when executed on the processor, causes the in-vehicle system to generate the first probabilistic model by; detecting candidate raised pavement markers; tracking the candidate raised pavement markers over time; eliminating false positives among the candidate raised pavement markers; fitting one or more splines to the candidate raised pavement markers; segmenting the one or more splines; and generating a mixture of probability distributions corresponding to respective segments of the one or more splines. - View Dependent Claims (6, 7, 8, 9, 10)
-
-
11. A computer-implemented method for estimating and tracking a lane boundary on a roadway, the method comprising:
-
capturing image data along the roadway by an image capture device located on a vehicle; receiving the image data by at least one processor; obtaining lidar data along the roadway by a lidar device located on the vehicle; receiving lidar data by at least one processor; generating a first probabilistic model for the lane boundary based on the image data; generating a second probabilistic model for the lane boundary based on the lidar data; fusing the first probabilistic model and the second probabilistic model to generate a fused probabilistic model; estimating and tracking the lane boundary based on the fused probabilistic model; detecting candidate reflective and non-reflective raised pavement markers in the image data; tracking the candidate reflective and non-reflective raised pavement markers over time; eliminating false positives among the candidate reflective and non-reflective raised pavement markers; fitting one or more splines to the candidate reflective and non-reflective raised pavement markers; segmenting the one or more splines; and generating the first probabilistic model based on the segmented one or more splines; wherein the first probabilistic model includes a first mixture of Gaussian distributions; the second probabilistic model includes a second mixture of Gaussian distributions; and the means for fusing the first probabilistic model and the second probabilistic model includes a processor and a non-transitory data storage on which is stored computer code which, when executed on the processor, causes the in-vehicle system to compute a product of the first probabilistic model and the second probabilistic model. - View Dependent Claims (12, 13, 14, 15, 16)
-
Specification