Rolling shutter correction for images captured by a camera mounted on a moving vehicle
First Claim
1. A computer implemented method for performing rolling shutter correction of data captured by sensors of a vehicle, comprising:
- receiving an image captured by a rolling shutter camera mounted on a vehicle, the rolling shutter camera capturing the image via an image scan, the image comprising a plurality of scan-lines, the image associated with a distortion axis, wherein each scan-line of the image maps to a distinct point along the distortion axis;
identifying a plurality of three-dimensional (3D) data points corresponding to the scene captured by the image, each 3D data point having a location in a 3D space;
for each of the plurality of 3D points;
projecting the 3D point to the image coordinates to obtain a projected point;
determining based on the position of the projected point, an estimate of a distance travelled by the vehicle between the time of capture of the 3D point by the rolling shutter camera and the time of completion of the image scan;
translating the 3D point to location along a direction of movement of the vehicle by the estimate of the distance;
storing the translated 3D points for displaying in conjunction with the image captured by the rolling shutter camera.
2 Assignments
0 Petitions
Accused Products
Abstract
A system performs rolling shutter correction to transform data captured by sensors of a vehicle, for example, cameras or lidar mounted on a vehicle, for example, an autonomous vehicle. The images captured by a rolling shutter camera mounted on a moving vehicle show rolling shutter distortion. The rolling shutter correction transforms the data representing points of scenes to perform rolling shutter compensation, i.e., compensation for the rolling shutter distortion. The rolling shutter compensation ensures that data representing as three dimensional points, for example, data captured by lidar is consistent with data represented in images captured by a rolling shutter camera. The system performs rolling shutter compensation by estimating a distance travelled by the vehicle between the time that the camera captured a point and the time that the image scan was completed and translating the 3D points by the estimated distance.
14 Citations
20 Claims
-
1. A computer implemented method for performing rolling shutter correction of data captured by sensors of a vehicle, comprising:
-
receiving an image captured by a rolling shutter camera mounted on a vehicle, the rolling shutter camera capturing the image via an image scan, the image comprising a plurality of scan-lines, the image associated with a distortion axis, wherein each scan-line of the image maps to a distinct point along the distortion axis; identifying a plurality of three-dimensional (3D) data points corresponding to the scene captured by the image, each 3D data point having a location in a 3D space; for each of the plurality of 3D points; projecting the 3D point to the image coordinates to obtain a projected point; determining based on the position of the projected point, an estimate of a distance travelled by the vehicle between the time of capture of the 3D point by the rolling shutter camera and the time of completion of the image scan; translating the 3D point to location along a direction of movement of the vehicle by the estimate of the distance; storing the translated 3D points for displaying in conjunction with the image captured by the rolling shutter camera. - View Dependent Claims (2, 3, 4, 5, 6, 7, 8)
-
-
9. A non-transitory computer readable storage medium storing instructions for performing rolling shutter correction of data captured by sensors of a vehicle, wherein the instructions when executed by a processor, cause the processor to perform the steps comprising:
-
receiving an image captured by a rolling shutter camera mounted on a vehicle, the rolling shutter camera capturing the image via an image scan, the image comprising a plurality of scan-lines, the image associated with a distortion axis, wherein each scan-line of the image maps to a distinct point along the distortion axis; identifying a plurality of three-dimensional (3D) data points corresponding to the scene captured by the image, each 3D data point having a location in a 3D space; for each of the plurality of 3D points; projecting the 3D point to the image coordinates to obtain a projected point; determining based on the position of the projected point, an estimate of a distance travelled by the vehicle between the time of capture of the 3D point by the rolling shutter camera and the time of completion of the image scan; translating the 3D point to location along a direction of movement of the vehicle by the estimate of the distance; storing the translated 3D points for displaying in conjunction with the image captured by the rolling shutter camera. - View Dependent Claims (10, 11, 12, 13, 14, 15, 16)
-
-
17. A computer system comprising:
-
a processor; and a non-transitory computer readable storage medium storing instructions for performing rolling shutter correction of data captured by sensors of a vehicle, wherein the instructions when executed by a processor, cause the processor to perform the steps comprising; receiving an image captured by a rolling shutter camera mounted on a vehicle, the rolling shutter camera capturing the image via an image scan, the image comprising a plurality of scan-lines, the image associated with a distortion axis, wherein each scan-line of the image maps to a distinct point along the distortion axis; identifying a plurality of three-dimensional (3D) data points corresponding to the scene captured by the image, each 3D data point having a location in a 3D space; for each of the plurality of 3D points; projecting the 3D point to the image coordinates to obtain a projected point; determining based on the position of the projected point, an estimate of a distance travelled by the vehicle between the time of capture of the 3D point by the rolling shutter camera and the time of completion of the image scan; translating the 3D point to location along a direction of movement of the vehicle by the estimate of the distance; storing the translated 3D points for displaying in conjunction with the image captured by the rolling shutter camera. - View Dependent Claims (18, 19, 20)
-
Specification