Systems and methods for controlling aliasing in images captured by an array camera for use in super resolution processing using pixel apertures

• US 9,794,476 B2
• Filed: 02/19/2016
• Issued: 10/17/2017
• Est. Priority Date: 09/19/2011
• Status: Active Grant

• Current Assignee: FotoNation Limited (Adeia Inc.)
• Original Assignee: Fotonation Cayman Limited
• Inventors: Nayar, Shree, Venkataraman, Kartik, Pain, Bedabrata, Lelescu, Dan
• Primary Examiner(s): Fosselman, Joel
• Assistant Examiner(s): Kao, Yih-Sien
• Application Number:
  US15/048,369
• Publication Number:
  US 20160316140A1
• Time in Patent Office: 606 Days
• Field of Search: None
• CPC Class Codes: H04N 13/232 using fly-eye lenses, e.g. arrangements of circular lenses
• CPC Class Codes: H04N 13/271 wherein the generated image signals comprise depth maps or disparity maps
• CPC Class Codes: H04N 2013/0081 Depth or disparity estimation from stereoscopic image signals
• CPC Class Codes: H04N 23/16 Optical arrangements associated therewith, e.g. for beam-splitting or for colour correction
• CPC Class Codes: H04N 23/45 for generating image signals from two or more image sensors being of different type or operating in different modes, e.g. with a CMOS sensor for moving images in combination with a charge-coupled device [CCD] for still images
• CPC Class Codes: H04N 23/951 by using two or more images to influence resolution, frame rate or aspect ratio
• CPC Class Codes: H04N 25/41 Extracting pixel data from a plurality of image sensors simultaneously picking up an image, e.g. for increasing the field of view by combining the outputs of a plurality of sensors

Abstract

Imager arrays, array camera modules, and array cameras in accordance with embodiments of the invention utilize pixel apertures to control the amount of aliasing present in captured images of a scene. One embodiment includes a plurality of focal planes, control circuitry configured to control the capture of image information by the pixels within the focal planes, and sampling circuitry configured to convert pixel outputs into digital pixel data. In addition, the pixels in the plurality of focal planes include a pixel stack including a microlens and an active area, where light incident on the surface of the microlens is focused onto the active area by the microlens and the active area samples the incident light to capture image information, and the pixel stack defines a pixel area and includes a pixel aperture, where the size of the pixel apertures is smaller than the pixel area.

Claims

1. An imager array, comprising:

a plurality of focal planes, where each focal plane comprises a two dimensional arrangement of pixels having at least two pixels in each dimension and each focal plane is contained within a region of the imager array that does not contain pixels from another focal plane;

control circuitry configured to control the capture of image information by the pixels within the focal planes; and

sampling circuitry configured to convert pixel outputs into digital pixel data;

wherein each of the pixels in the plurality of focal planes comprises a pixel stack including a microlens and an active area, where light incident on the surface of the microlens is focused onto the active area by the microlens and the active area samples the incident light to capture image information;

wherein the pixel stack of each pixel defines a pixel area and includes a pixel aperture, wherein the light incident on the pixel stack passes through the pixel aperture and is incident on the pixel area, where the size of the pixel aperture is smaller than the pixel area;

wherein aliasing present in the images captured by each focal plane is different so that high frequencies folded back into the lower frequencies of the captured images can be recovered using super resolution processing, wherein the super resolution processing processes the images captured by each focal plane to obtain a synthesized high resolution (HR) image with an increased overall resolution.

2. The imager array of claim 1, wherein the pixel aperture of each pixel stack is formed by a microlens that is smaller than the pixel area of the pixel.

3. The imager array of claim 2, wherein gaps exist between adjacent microlenses in the pixel stacks of adjacent pixels in a focal plane.

4. The imager array of claim 3, wherein light is prevented from entering the pixel stacks through the gaps between the microlenses by a light blocking material.

5. The imager array of claim 4, wherein photoresist is located in the gaps between the microlenses.

6. The imager array of claim 2, wherein the amount of aliasing in an image captured by a focal plane is greater than the amount of aliasing that would be present were the microlens to occupy the entire pixel area defined by the pixel stack.

7. The imager array of claim 2, wherein the pixel stacks further comprise an oxide layer and the microlens sits atop the oxide layer.

8. The imager array of claim 7, wherein the pixel stack includes a color filter.

9. The imager array of claim 8, wherein the color filters in the pixel stacks of the two dimensional arrangement of pixels within a focal plane are the same.

10. The imager array of claim 8, wherein the color filters in the pixel stacks of the two dimensional arrangement of pixels within at least one focal plane form a Bayer filter pattern.

11. The imager array of claim 7, wherein the pixel stack does not include a color filter.

12. The imager array of claim 7, wherein the pixel stack includes a nitride passivation layer and a bottom oxide layer that provides support and isolation for metal interconnects.

13. The imager array of claim 1, wherein the pixel aperture is formed using at least one light blocking material.

14. An imager array, comprising:

a plurality of focal planes, where each focal plane comprises a two dimensional arrangement of pixels having at least two pixels in each dimension and each focal plane is contained within a region of the imager array that does not contain pixels from another focal plane;

control circuitry configured to control the capture of image information by the pixels within the focal planes; and

sampling circuitry configured to convert pixel outputs into digital pixel data;

wherein each of the pixels in the plurality of focal planes comprises a pixel stack including a microlens and an active area, where light incident on the surface of the microlens is focused onto the active area by the microlens and the active area samples the incident light to capture image information;

wherein the pixel stack of each pixel defines a pixel area and the microlens forms a pixel aperture, wherein the light incident on the pixel stack passes through the pixel aperture and is incident on the pixel area, where the size of the pixel aperture is smaller than the pixel area so that the sampling frequency of the pixels introduces aliasing into the images captured by each focal plane;

wherein gaps exist between adjacent microlenses in the pixel stacks of adjacent pixels in a focal plane;

wherein the pixel stack further comprises light blocking material located to prevent light from entering the pixel stacks through the gaps between the microlenses;

wherein the amount of aliasing in images captured by a focal plane is greater than the amount of aliasing that would be present were the microlens in each pixel stack to occupy the entire pixel area defined by the pixel stack; and

wherein the aliasing present in the images captured by each focal plane is different so that high frequencies folded back into the lower frequencies of the captured images can be recovered using super resolution processing, wherein the super resolution processing processes the images captured by each focal plane to obtain a synthesized high resolution (HR) image with an increased overall resolution.

15. An array camera module, comprising:

an imager array configured to capture light field image data, comprising;

a plurality of focal planes, where each focal plane comprises a two dimensional arrangement of pixels having at least two pixels in each dimension and each focal plane is contained within a region of the imager array that does not contain pixels from another focal plane;

control circuitry configured to control the capture of image information by the pixels within the focal planes; and

sampling circuitry configured to convert pixel outputs into digital pixel data;

wherein each of the pixels in the plurality of focal planes comprise a pixel stack including a microlens and an active area, where light incident on the surface of the microlens is focused onto the active area by the microlens and the active area samples the incident light to capture image information;

wherein the pixel stack of each pixel defines a pixel area and includes a pixel aperture, wherein the light incident on the pixel stack passes through the pixel aperture and is incident on the pixel area, where the size of the pixel apertures is smaller than the pixel area so that the sampling frequency of the pixels introduces aliasing into the images captured by each focal plane; and

an optic array of lens stacks,

wherein the aliasing present in the images captured by each focal plane is different so that high frequencies folded back into the lower frequencies of the captured images can be recovered using super resolution processing, wherein the super resolution processing processes the images captured by each focal plane to obtain a synthesized high resolution (HR) image with an increased overall resolution.

16. The array camera module of claim 15, wherein the pixel aperture of each pixel stack is formed by a microlens that is smaller than the pixel area of the pixel.

17. The array camera module of claim 16, wherein gaps exist between adjacent microlenses in the pixel stacks of adjacent pixels in a focal plane.

18. The array camera module of claim 17, wherein light is prevented from entering the pixel stacks through the gaps between the microlenses by a light blocking material.

19. The array camera module of claim 18, wherein photoresist is located in the gaps between the microlenses.

20. The array camera module of claim 16, wherein the amount of aliasing in images captured by a focal plane is greater than the amount of aliasing that would be present were the microlens in each pixel stack to occupy the entire pixel area defined by the pixel stack.