Capturing and processing of images using monolithic camera array with heterogeneous imagers

• US 8,514,491 B2
• Filed: 11/22/2010
• Issued: 08/20/2013
• Est. Priority Date: 11/20/2009
• Status: Active Grant

• Current Assignee: FotoNation Limited (Adeia Inc.)
• Original Assignee: Pelican Imaging Corporation
• Inventors: Duparre, Jacques
• Primary Examiner(s): PINKNEY, DAWAYNE
• Application Number:
  US12/952,106
• Publication Number:
  US 20110122308A1
• Time in Patent Office: 1,002 Days
• Field of Search: 359618-621, 359625-626, 359/443, 359454-455, 264/1.1, 264/1.32, 264/2.7
• US Class Current: 359/618
• CPC Class Codes: G02B 13/001 Miniaturised objectives for electronic devices, e.g. portable telephones, webcams, PDAs, small digital cameras G02B23/243 takes precedence
• CPC Class Codes: G02B 3/0056 arranged along two different directions in a plane, e.g. honeycomb arrangement of lenses G02B3/0043 takes precedence; miniaturised objectives for electronic devices employing wafer level optics G02B13/0085
• CPC Class Codes: G02B 3/0062 Stacked lens arrays, i.e. refractive surfaces arranged in at least two planes, without structurally separate optical elements in-between
• CPC Class Codes: G02B 5/201 in the form of arrays
• CPC Class Codes: H01L 27/14621 Colour filter arrangements
• CPC Class Codes: H01L 27/14625 Optical elements or arrangements associated with the device
• CPC Class Codes: H04N 23/10 for generating image signals from different wavelengths
• CPC Class Codes: H04N 23/16 Optical arrangements associated therewith, e.g. for beam-splitting or for colour correction
• CPC Class Codes: H04N 23/45 for generating image signals from two or more image sensors being of different type or operating in different modes, e.g. with a CMOS sensor for moving images in combination with a charge-coupled device [CCD] for still images
• CPC Class Codes: H04N 23/51 Housings
• CPC Class Codes: H04N 23/54 Mounting of pick-up tubes, electronic image sensors, deviation or focusing coils
• CPC Class Codes: H04N 23/90 Arrangement of cameras or camera modules, e.g. multiple cameras in TV studios or sports stadiums
• CPC Class Codes: H04N 23/951 by using two or more images to influence resolution, frame rate or aspect ratio
• CPC Class Codes: H04N 25/41 Extracting pixel data from a plurality of image sensors simultaneously picking up an image, e.g. for increasing the field of view by combining the outputs of a plurality of sensors
• CPC Class Codes: H04N 25/76 Addressed sensors, e.g. MOS or CMOS sensors

Abstract

Systems and methods for implementing array cameras configured to perform super-resolution processing to generate higher resolution super-resolved images using a plurality of captured images and lens stack arrays that can be utilized in array cameras are disclosed. Lens stack arrays in accordance with many embodiments of the invention include lens elements formed on substrates separated by spacers, where the lens elements, substrates and spacers are configured to form a plurality of optical channels, at least one aperture located within each optical channel, at least one spectral filter located within each optical channel, where each spectral filter is configured to pass a specific spectral band of light, and light blocking materials located within the lens stack array to optically isolate the optical channels.

Claims

1. A lens stack array configured to form multiple images of a scene, comprising:

lens elements formed on substrates separated by spacers, where the lens elements, substrates and spacers are configured to form a plurality of optical channels, where the optical channels are configured to form an image of an object space within a scene from slightly different viewpoints so that the images formed by each of the optical channels include a view of the same object space within the scene and contain non-redundant information about the object space;

at least one aperture located within each optical channel;

at least one spectral filter located within each optical channel, where each spectral filter is configured to pass a specific spectral band of light;

light blocking materials located within the lens stack array to optically isolate the optical channels; and

wherein the shape of at least one lens surface of at least one lens element within at least a first optical channel is a function of a first spectral band of light;

wherein the shape of at least one lens surface of at least one lens element within at least a second optical channel is also a function of the first spectral band of light; and

wherein the shape of at least one lens surface of at least one lens element within at least a third optical channel is a function of a second spectral band of light.

2. The lens stack array of claim 1, wherein the light blocking materials are selected from the group consisting of opaque materials, reflective materials and combinations thereof.

3. The lens stack array of claim 1, wherein each spectral filter is selected from the group consisting of an organic color filter, an absorptive material, a dielectric coating, an interference filter, a multilayer coating, and combinations thereof.

4. The lens stack array of claim 1, further comprising at least one polarizing filter located within each optical channel.

5. The lens stack array of claim 1, wherein the radius of curvature of the shape of the at least one lens surface of the at least one lens element within the first optical channel is a function of the first spectral band of light.

6. The lens stack array of claim 1, wherein the shape of the at least one lens surface of the at least one lens element within the first optical channel is rotationally symmetric and aspheric, and is described by an aspheric lens equation, wherein the coefficients of the aspheric lens equation are a function of the first spectral band of light.

7. The lens stack array of claim 1, wherein the back focal length of each of the at least two optical channels in the lens stack array that have a spectral filter is the same irrespective of the spectral band of light passed by the spectral filter within the optical channel.

8. The lens stack array of claim 1, wherein a combination of high and low Abbe number materials is used in the construction of each of the lens elements in an optical channel to reduce chromatic aberrations.

9. The lens stack array of claim 1, further comprising:

at least one aperture stop located within each optical channel;

wherein each spectral filter is located within each optical channel so that the spectral filter is proximate the aperture stop.

10. The lens stack array of claim 9, wherein each aperture stop is formed by a light blocking material selected from the group consisting of metal materials, oxide materials, black particle filled photoresists and combinations thereof.

11. The lens stack array of claim 1, wherein:

at least one lens surface is selected from the group consisting of diffractive, Fresnel, refractive and combinations thereof.

12. The lens stack array of claim 11, wherein the radii of curvature of at least two lens surfaces differ based upon the specific spectral band of light passed by the spectral filter within each of the at least two optical channels that have spectral filters.

13. The lens stack array of claim 11, wherein each of the optical channels have the same back focal length.

14. The lens stack array of claim 1, wherein at least one of the lens elements in an optical channel is a negative lens element, and the negative lens element is proximate the image formed by the optical channel.

15. The lens stack array of claim 1, wherein at least two of the optical channels have different focal lengths.

16. The lens stack array of claim 1, further comprising a mechanical zoom mechanism within an optical channel configured to smoothly transition between different fields of view.

17. The lens stack array of claim 1, wherein the lens stack array comprises an N×

M array of optical channels, where at least one of N and M is greater than 2.

18. The lens stack array of claim 1, wherein the lens stack array is fabricated using techniques consisting of wafer level optics techniques, injection molding, glass molding and combinations thereof.

19. The lens stack array of claim 1, wherein the light blocking materials located within the lens stack array to optically separate the optical channels comprise at least two opaque surfaces located on a substrate within an optical channel, where the two opaque surfaces have openings arranged in axial alignment with the optical channel.

20. The lens stack array of claim 1, wherein the light blocking materials located within the lens stack array to optically separate the optical channels comprise opaque walls disposed at the boundaries between the optical channels.

21. The lens stack array of claim 20, wherein the opaque walls are cavities between the optical channels filled with light blocking material.

22. The lens stack array of claim 1, wherein a plurality of the spacers are constructed from light blocking materials and are located within the lens stack array to optically separate the optical channels.

23. The lens stack array of claim 1, wherein a plurality of the spacers are coated in light blocking materials and are located within the lens stack array to optically isolate the optical channels.

24. The lens stack array of claim 1, further comprising at least one spectral filter located within a plurality of the optical channels, where each spectral filter is configured to pass a specific spectral band of light, such that a first spectral filter located in the first optical channel defines the first spectral band of light, and a second spectral filter located in the third optical channel defines the second spectral band of light.

25. The lens stack array of claim 24, further comprising a third spectral filter located in a fourth optical channel that defines a third spectral band of light.

26. The lens stack array of claim 1, wherein the substrates are coated with the light blocking materials so as to define at least two aperture stops within at least a first optical channel and two aperture stops within at least an adjacent second optical channel, such that at least the first optical channel and at least the second optical channel are optically isolated from one another.

27. The lens stack array of claim 1, wherein at least one spectral filter is located within the space defined by the substrates separated by spacers.

28. The lens stack array of claim 27, wherein at least one spectral filter is disposed on the surface of a lens element.

29. The lens stack array of claim 1, wherein at least one spectral filter is disposed external to the space defined by the substrates separated by spacers.

30. The lens stack array of claim 1, wherein the lens stack array is configured to be integrated with a sensor that includes an imager for each optical channel to form a camera module assembly, such that each optical channel focuses light onto one imager, wherein at least one spectral filter within an optical channel is located on the surface of a corresponding imager.