Image stitching
First Claim
1. A method comprising:
- receiving a first set of image frames that are captured by a first set of camera modules at a particular time, wherein (1) the first set of camera modules includes a first camera module and a second camera module that have an overlapping field of view, (2) the first set of image frames includes a first image captured by the first camera module and a second image captured by the second camera module, and (3) the first image includes a first sub-image that overlaps with a second sub-image of the second image on the overlapping field of view;
interpolating a first virtual camera between the first camera module and the second camera module;
determining a first set of disparity maps between the first set of camera modules by;
determining a first disparity map that maps disparity of pixels from the first sub-image to the second sub-image;
determining a second disparity map that maps disparity of pixels from the second sub-image to the first sub-image; and
for each corresponding pixel location in the overlapping field of view;
determining similarity scores between a first pixel of the first sub-image at a first pixel location and second pixels of the second sub-image at second pixel locations, wherein a corresponding distance between the first pixel location and each of the second pixel locations is equal to a different disparity value;
determining a highest similarity score from the similarity scores;
determining a disparity value associated with the highest similarity score; and
assigning the disparity value associated with the highest similarity score to the first pixel location in the first disparity map;
generating, by a processor-based computing device programmed to perform the generating, a first virtual camera image associated with the particular time for the first virtual camera from the first set of image frames, the first virtual camera image being generated based on the first set of disparity maps and a position of the first virtual camera relative to the first camera module and the second camera module; and
constructing a left panoramic image and a right panoramic image associated with the particular time from the first set of image frames captured by the first set of camera modules and the first virtual camera image of the first virtual camera.
2 Assignments
0 Petitions
Accused Products
Abstract
The disclosure includes a system and method for stitching image frames. The system includes a processor and a memory storing instructions that, when executed, cause the system to: receive image frames that are captured by two or more camera modules at a particular time; interpolate a virtual camera between a first set of camera modules from the two or more camera modules; determine a set of disparity maps between the first set of camera modules; generate, based on the set of disparity maps, a virtual camera image associated with the particular time for the virtual camera from a set of image frames captured by the first set of camera modules at the particular time; and construct a left panoramic image and a right panoramic image associated with the particular time from the image frames captured by the two or more camera modules and the virtual camera image.
-
Citations
22 Claims
-
1. A method comprising:
-
receiving a first set of image frames that are captured by a first set of camera modules at a particular time, wherein (1) the first set of camera modules includes a first camera module and a second camera module that have an overlapping field of view, (2) the first set of image frames includes a first image captured by the first camera module and a second image captured by the second camera module, and (3) the first image includes a first sub-image that overlaps with a second sub-image of the second image on the overlapping field of view; interpolating a first virtual camera between the first camera module and the second camera module; determining a first set of disparity maps between the first set of camera modules by; determining a first disparity map that maps disparity of pixels from the first sub-image to the second sub-image; determining a second disparity map that maps disparity of pixels from the second sub-image to the first sub-image; and for each corresponding pixel location in the overlapping field of view; determining similarity scores between a first pixel of the first sub-image at a first pixel location and second pixels of the second sub-image at second pixel locations, wherein a corresponding distance between the first pixel location and each of the second pixel locations is equal to a different disparity value; determining a highest similarity score from the similarity scores; determining a disparity value associated with the highest similarity score; and assigning the disparity value associated with the highest similarity score to the first pixel location in the first disparity map; generating, by a processor-based computing device programmed to perform the generating, a first virtual camera image associated with the particular time for the first virtual camera from the first set of image frames, the first virtual camera image being generated based on the first set of disparity maps and a position of the first virtual camera relative to the first camera module and the second camera module; and constructing a left panoramic image and a right panoramic image associated with the particular time from the first set of image frames captured by the first set of camera modules and the first virtual camera image of the first virtual camera. - View Dependent Claims (2, 3, 4, 5, 6, 7, 8)
-
-
9. A system comprising:
-
one or more processors; and one or more non-transitory tangible computer readable mediums communicatively coupled to the one or more processors and storing executable instructions executable by the one or more processors to perform operations comprising; receiving a first set of image frames that are captured by a first set of camera modules at a particular time, wherein (1) the first set of camera modules includes a first camera module and a second camera module that have an overlapping field of view, (2) the first set of image frames includes a first image captured by the first camera module and a second image captured by the second camera module, and (3) the first image includes a first sub-image that overlaps with a second sub-image of the second image on the overlapping field of view; interpolating a first virtual camera between the first camera module and the second camera module; determining a first set of disparity maps between the first set of camera modules by; determining a first disparity map that maps disparity of pixels from the first sub-image to the second sub-image; determining a second disparity map that maps disparity of pixels from the second sub-image to the first sub-image; and for each corresponding pixel location in the overlapping field of view; determining similarity scores between a first pixel of the first sub-image at a first pixel location and second pixels of the second sub-image at second pixel locations, wherein a corresponding distance between the first pixel location and each of the second pixel locations is equal to a different disparity value; determining a highest similarity score from the similarity scores; determining a disparity value associated with the highest similarity score; and assigning the disparity value associated with the highest similarity score to the first pixel location in the first disparity map; generating, based on the first set of disparity maps and a position of the first virtual camera relative to the first set of camera modules, a first virtual camera image associated with the particular time for the first virtual camera from the first set of image frames, the first virtual camera image being generated based on the first set of disparity maps and a position of the first virtual camera relative to the first camera module and the second camera module; and constructing a left panoramic image and a right panoramic image associated with the particular time from the first set of image frames captured by the first set of camera modules and the first virtual camera image of the first virtual camera. - View Dependent Claims (10, 11, 12, 13, 14, 15)
-
-
16. A computer program product comprising a non-transitory computer-usable medium including a computer-readable program, wherein the computer-readable program when executed on a computer causes the computer to:
-
receive a first set of image frames that are captured by a first set of camera modules at a particular time, wherein (1) the first set of camera modules includes a first camera module and a second camera module that have an overlapping field of view, (2) the first set of image frames includes a first image captured by the first camera module and a second image captured by the second camera module, and (3) the first image includes a first sub-image that overlaps with a second sub-image of the second image on the overlapping field of view; interpolate a first virtual camera between the first camera module and the second camera module; determine a first set of disparity maps between the first set of camera modules by; determining a first disparity map that maps disparity of pixels from the first sub-image to the second sub-image; determining a second disparity map that maps disparity of pixels from the second sub-image to the first sub-image; and for each corresponding pixel location in the overlapping field of view; determining similarity scores between a first pixel of the first sub-image at a first pixel location and second pixels of the second sub-image at second pixel locations, wherein a corresponding distance between the first pixel location and each of the second pixel locations is equal to a different disparity value; determining a highest similarity score from the similarity scores; determining a disparity value associated with the highest similarity score; and assigning the disparity value associated with the highest similarity score to the first pixel location in the first disparity map; generate, based on the first set of disparity maps and a position of the first virtual camera relative to the first set of camera modules, a first virtual camera image associated with the particular time for the first virtual camera from the first set of image frames, the first virtual camera image being generated based on the first set of disparity maps and a position of the first virtual camera relative to the first camera module and the second camera module; and construct a left panoramic image and a right panoramic image associated with the particular time from the set of image frames captured by the first set of camera modules and the first virtual camera image of the first virtual camera. - View Dependent Claims (17, 18, 19, 20, 21, 22)
-
Specification