Apparatus and methods for rolling shutter compensation for multi-camera systems
First Claim
1. A computerized system configured to obtain composite images, the system comprising:
- a processor adapted to execute a plurality of computer instructions; and
non-transitory storage medium including the plurality of the computer instructions which, when executed by the processor, cause the processor to;
obtain component images, the component images including a first component image comprised of a first plurality of pixels captured by a first imaging sensor and a second component image comprised of a second plurality of pixels captured by a second imaging sensor, the component images captured by the first imaging sensor and the second imaging sensor on a row-by-row basis, the row-by-row capture of the component images resulting in capture of individual rows of the component images at different acquisition times;
generate a first composite image by performing a first transformation operation on the component images;
for pixels in the first composite image, determine corresponding rows in the component images such that for a first set of pixels in the first composite image a first row of the first component image is determined to be corresponding, and for a second set of pixels in the first composite image a second row of the second component image is determined to be corresponding;
determine acquisition times of the component images associated with row locations corresponding to the pixels in the first composite image such that a first acquisition time is determined for the first row of the first component image and a second acquisition time is determined for the second row of the second component image;
determine orientations of the first imaging sensor and the second imaging sensor based on the acquisition times and orientation information of the first imaging sensor and the second imaging sensor such that a first orientation of the first imaging sensor is determined for capture of the first row of the first component image by the first imaging sensor based on the first acquisition time and a second orientation of the second imaging sensor is determined for capture of the second row of the second component image by the second sensor based on the second acquisition time; and
perform a second transformation operation on the component images based on the first orientation of the first imaging sensor and the second orientation of the second imaging sensor to generate a second composite image such that the second transformation operation compensates for a difference between the first acquisition time of the first row of the first component image and the second acquisition time of the second row of the second component image and orientations of the first imaging sensor and the second image sensor during capture of the corresponding rows in the component images.
5 Assignments
0 Petitions
Accused Products
Abstract
Images may be obtained using a moving camera comprised of two or more rigidly mounted image sensors. Camera motion may change camera orientation when different portions of an image are captured. Pixel acquisition time may be determined based on image exposure duration and position of the pixel in the image array (pixel row index). Orientation of the sensor may at the pixel acquisition time instance may be determined. Image transformation may be performed wherein a given portion of the image may be associated with a respective transformation characterized by the corrected sensor orientation. In some implementations of panoramic image acquisition, multiple source images may be transformed to, e.g., equirectangular plane, using sensor orientation that is corrected for the time of pixel acquisition. Use of orientation correction may improve quality of stitching by, e.g., reducing contrast of border areas between portions of the transformed image obtained by different image sensors.
132 Citations
20 Claims
-
1. A computerized system configured to obtain composite images, the system comprising:
-
a processor adapted to execute a plurality of computer instructions; and non-transitory storage medium including the plurality of the computer instructions which, when executed by the processor, cause the processor to; obtain component images, the component images including a first component image comprised of a first plurality of pixels captured by a first imaging sensor and a second component image comprised of a second plurality of pixels captured by a second imaging sensor, the component images captured by the first imaging sensor and the second imaging sensor on a row-by-row basis, the row-by-row capture of the component images resulting in capture of individual rows of the component images at different acquisition times; generate a first composite image by performing a first transformation operation on the component images; for pixels in the first composite image, determine corresponding rows in the component images such that for a first set of pixels in the first composite image a first row of the first component image is determined to be corresponding, and for a second set of pixels in the first composite image a second row of the second component image is determined to be corresponding; determine acquisition times of the component images associated with row locations corresponding to the pixels in the first composite image such that a first acquisition time is determined for the first row of the first component image and a second acquisition time is determined for the second row of the second component image; determine orientations of the first imaging sensor and the second imaging sensor based on the acquisition times and orientation information of the first imaging sensor and the second imaging sensor such that a first orientation of the first imaging sensor is determined for capture of the first row of the first component image by the first imaging sensor based on the first acquisition time and a second orientation of the second imaging sensor is determined for capture of the second row of the second component image by the second sensor based on the second acquisition time; and perform a second transformation operation on the component images based on the first orientation of the first imaging sensor and the second orientation of the second imaging sensor to generate a second composite image such that the second transformation operation compensates for a difference between the first acquisition time of the first row of the first component image and the second acquisition time of the second row of the second component image and orientations of the first imaging sensor and the second image sensor during capture of the corresponding rows in the component images. - View Dependent Claims (2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15)
-
-
16. A computer implemented method of improving quality of a digital panorama image obtained based on stitching of source digital images, the method comprising:
-
for a portion of the digital panorama image, determining a corresponding location in at least one of the source digital images, the at least one of the source digital images acquired by an image sensor on a location-by-location basis such that individual locations of the at least one of the source digital images are acquired at different acquisition times; determining an acquisition time at which the image sensor acquired the corresponding location in the at least one of the source digital images; determining an orientation of the image sensor during the acquisition time; performing a stitching operation configured to transform a portion of the at least one source digital image from a coordinate representation of the image sensor into a coordinate representation of the digital panorama image, the transform configured based on the orientation of the image sensor during the acquisition time to compensate for acquisition of the individual locations of the at least one of the source digital images are different acquisition times and the orientation of the image sensor; wherein the quality of the digital panorama image obtained based on the orientation of the image sensor is improved when compared to another quality of another digital panorama image obtained at a nominal sensor orientation. - View Dependent Claims (17, 18, 19)
-
-
20. A non-transitory computer readable medium comprising a plurality of computer instructions configured to, when executed by a processor, produce a composite panoramic image in a panoramic image plane from first and second source images comprised of rows of pixels, the first and second source images captured on a row-by-row basis such that individual rows of the first and second source images are captured at different acquisition times, by at least:
-
for a location in the panoramic image plane, determine corresponding rows in the first and second source images such that a corresponding first row of pixels in the first source image and a corresponding second row of pixels of the second source image are determined; determine acquisition times of the first and second source images associated with the corresponding rows such that a first acquisition time is determined for the first row of pixels in the first source image and a second acquisition time is determined for the second row pixels in the second source image; determine sensor orientations based on the acquisition times and orientation information such that a first sensor orientation is determined for a first sensor at the first acquisition time and a second sensor orientation is determined for a second sensor at on the second acquisition time; and perform a transformation operation on the first source image and the second source image based on the first sensor orientation and the second sensor orientation to obtain the composite panoramic image such that the transformation operation compensates for a difference between the first acquisition time of the first row of pixels in the first source image and the second acquisition time of the second row of pixels in the second source image and orientations of the first sensor and the second sensor during capture of the corresponding rows in the first and second source images.
-
Specification