SYSTEMS AND METHODS FOR SYNTHESIZING HIGH RESOLUTION IMAGES USING SUPER-RESOLUTION PROCESSES
First Claim
1. A method of generating a high resolution image of a scene using an imager array including a plurality of imagers that each capture an image of the scene, and a forward imaging transformation for each imager, the method comprising:
- obtaining input images using the plurality of imagers;
determining an initial estimate of at least a portion of a high resolution image using a plurality of pixels from the input images;
determining a high resolution image that when mapped through the forward imaging transformation matches the input images to within at least one predetermined criterion using the initial estimate of at least a portion of the high resolution image;
wherein each forward imaging transformation corresponds to the manner in which each imager in the imaging array generate the input images; and
wherein the high resolution image has a resolution that is greater than any of the input images.
13 Assignments
0 Petitions
Accused Products
Abstract
Systems and methods in accordance with embodiments of the invention are disclosed that use super-resolution (SR) processes to use information from a plurality of low resolution (LR) images captured by an array camera to produce a synthesized higher resolution image. One embodiment includes obtaining input images using the plurality of imagers, using a microprocessor to determine an initial estimate of at least a portion of a high resolution image using a plurality of pixels from the input images, and using a microprocessor to determine a high resolution image that when mapped through the forward imaging transformation matches the input images to within at least one predetermined criterion using the initial estimate of at least a portion of the high resolution image. In addition, each forward imaging transformation corresponds to the manner in which each imager in the imaging array generate the input images, and the high resolution image synthesized by the microprocessor has a resolution that is greater than any of the input images.
317 Citations
104 Claims
-
1. A method of generating a high resolution image of a scene using an imager array including a plurality of imagers that each capture an image of the scene, and a forward imaging transformation for each imager, the method comprising:
-
obtaining input images using the plurality of imagers; determining an initial estimate of at least a portion of a high resolution image using a plurality of pixels from the input images; determining a high resolution image that when mapped through the forward imaging transformation matches the input images to within at least one predetermined criterion using the initial estimate of at least a portion of the high resolution image; wherein each forward imaging transformation corresponds to the manner in which each imager in the imaging array generate the input images; and wherein the high resolution image has a resolution that is greater than any of the input images. - View Dependent Claims (2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 87, 88, 89, 90, 91, 92, 93)
-
-
19. The method of 18, wherein the neighborhood of pixels varies in size based upon the number of pixels populated onto the high resolution grid in the neighborhood of the identified pixel.
-
54. The method of 53, wherein determining differences between pixels in the initial estimate of at least a portion of the high resolution image and the identified corresponding pixels in at least a portion of at least one input image comprises:
-
determining the difference in value between a pixel in the initial estimate of at least a portion of the high resolution image and each of the identified corresponding pixels in the input images; assigning weights to the determined differences in values; and accumulating a weighted difference using the determined differences in value and the assigned weights. - View Dependent Claims (55, 56, 57, 58, 59, 60, 61, 62)
-
-
94. A method of fusing a plurality of input images, the method comprising:
-
populating a high resolution grid corresponding to the pixel locations of the at least a portion of a fused high resolution image with pixels from the input images using geometric correction information; assigning a depth value for each pixel on the high resolution grid; and interpolating the high resolution grid to obtain filtered pixel values for each pixel in the initial estimate of the high resolution image, where the depth values are used to direct the interpolation of the high resolution grid. - View Dependent Claims (95, 96, 97, 99)
-
-
98. The method of 97, wherein the neighborhood of pixels varies in size based upon the number of pixels populated onto the high resolution grid in the neighborhood of the identified pixel.
-
100. A method of generating a stereo pair of images of a scene using an imager array, comprising:
-
obtaining a plurality of images from the imager array; synthesizing a first high resolution image of the scene with respect to a first field of view using pixels from at least two of the plurality of images; and synthesizing a second high resolution image of the scene with respect to a second field of view using pixels from at least two of the plurality of images; wherein the first and second synthesized images form a stereo pair; and wherein the first and second high resolution images have higher resolution than the resolution of each of the images obtained from the imager array. - View Dependent Claims (101, 102)
-
-
103. An array camera, comprising:
-
an imager array including a plurality of imagers; memory containing parameters defining a forward imaging model for the imager array; and a processor configured to obtain a plurality of input images using the imager array and store the input images in memory; wherein the processor is configured to determine an initial estimate of at least a portion of a high resolution image using a plurality of pixels from the input images; and wherein the processor is configured to determine a high resolution image that when mapped through the forward imaging transformation matches the input images to within at least one predetermined criterion using the initial estimate of at least a portion of the high resolution image.
-
-
104. An array camera configured to capture stereo pairs, comprising:
-
an imager array including a plurality of imagers; memory containing parameters defining a forward imaging model for the imager array; and a processor configured to obtain a plurality of input images using the imager array and store the input images in memory; wherein the processor is configured to synthesizing a stereo pair of high resolution images using the plurality of input images, where each image in the stereo pair is synthesized with respect to a different field of view using pixels from at least two of the plurality of images, and is higher resolution than each of the plurality of input images.
-
Specification