Systems and methods for synthesizing high resolution images using a set of geometrically registered images
First Claim
1. An array camera, comprising:
- an imager array including a plurality of imagers;
a processor; and
memory containing an image processing pipeline software application, an imaging prior including photometric calibration and geometric calibration data, and parameters defining a forward imaging transformation for the imager array;
wherein the image processing pipeline software directs the processor to;
trigger the imagers in the imager array to capture a plurality of images;
photometrically normalize each of the captured images using the photometric calibration data to obtain input images and storing the input images in memory;
applying scene independent geometric corrections to the input images using the geometric calibration data to obtain geometrically registered input images;
determine an initial estimate of at least a portion of a high resolution image from a plurality of pixels from the geometrically registered input images; and
determine a high resolution image that when mapped through the forward imaging transformation matches the geometrically registered input images to within at least one predetermined criterion using the initial estimate of at least a portion of the high resolution image.
13 Assignments
0 Petitions
Accused Products
Abstract
Systems and methods in accordance with embodiments of the invention are disclosed that use super-resolution (SR) processes to use information from a plurality of low resolution (LR) images captured by an array camera to produce a synthesized higher resolution image. One embodiment includes obtaining input images, determining an initial estimate of at least a portion of a high resolution image using a plurality of pixels from the input images, and determining a high resolution image that when mapped through the forward imaging transformation matches the input images to within at least one predetermined criterion using the initial estimate of at least a portion of the high resolution image. In addition, each forward imaging transformation corresponds to the manner in which each imager generates the input images, and the high resolution image has a resolution that is greater than any of the input images.
499 Citations
28 Claims
-
1. An array camera, comprising:
-
an imager array including a plurality of imagers; a processor; and memory containing an image processing pipeline software application, an imaging prior including photometric calibration and geometric calibration data, and parameters defining a forward imaging transformation for the imager array; wherein the image processing pipeline software directs the processor to; trigger the imagers in the imager array to capture a plurality of images; photometrically normalize each of the captured images using the photometric calibration data to obtain input images and storing the input images in memory; applying scene independent geometric corrections to the input images using the geometric calibration data to obtain geometrically registered input images; determine an initial estimate of at least a portion of a high resolution image from a plurality of pixels from the geometrically registered input images; and determine a high resolution image that when mapped through the forward imaging transformation matches the geometrically registered input images to within at least one predetermined criterion using the initial estimate of at least a portion of the high resolution image. - View Dependent Claims (2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27)
-
-
28. An array camera, comprising:
-
an imager array including a plurality of imagers; a processor; and memory containing an image processing pipeline software application, an imaging prior including photometric calibration and geometric calibration data, and parameters defining a forward imaging transformation for the imager array; wherein the image processing pipeline software directs the processor to; trigger the imagers in the imager array to capture a plurality of images, where the captured images capture a scene in which depths of points in the imaged scene vary and each of the input images differs from the other input images due to; scene independent geometric distortions inherent to the optics and manufacturing processes used to fabricate each of the plurality of imagers; and scene dependent geometric displacements due to parallax based upon the depths of the points in the imaged scene; and photometrically normalize each of the captured images using the photometric calibration data to obtain input images and storing the input images in memory; applying scene independent geometric corrections to the input images using the geometric calibration data to obtain geometrically registered input images; determine scene dependent parallax information with respect to the geometrically registered input images based upon disparity relative to a reference point of view resulting from the depths of points in the imaged scene, where the scene dependent parallax information comprises scene dependent geometric transformations; determine an initial estimate of at least a portion of a high resolution image from a plurality of pixels from the geometrically registered input images based upon scene dependent parallax information for each of the plurality of pixels relative to the reference point of view, where the scene dependent parallax information of a given pixel location is the combination of a scene independent geometric correction determined for the given pixel using the scene dependent geometric transformation determined for the given pixel location; and determine a high resolution image that when mapped through the forward imaging transformation matches the geometrically registered input images to within at least one predetermined criterion using the initial estimate of at least a portion of the high resolution image; wherein each forward imaging transformation corresponds to the manner in which each imager in the plurality of images captures the geometrically registered input images and comprises applying geometric transformations related to parallax observed due to the depths of points in the imaged scene; and wherein the high resolution image has a resolution that is greater than any of the input images.
-
Specification