MULTI-LENS CAMERA

• US 20130258044A1
• Filed: 03/30/2012
• Published: 10/03/2013
• Est. Priority Date: 03/30/2012
• Status: Abandoned Application

• Current Assignee: Zetta Research and Development LLC - FORC Series
• Original Assignee: Zetta Research and Development LLC - FORC Series
• Inventors: Betts-Lacroix, Jonathan N.
• Application Number:
  US13/435,549
• Publication Number:
  US 20130258044A1
• Time in Patent Office: Days
• US Class Current: 348/36
• CPC Class Codes: H04N 13/243 using three or more 2D image sensors
• CPC Class Codes: H04N 17/002 for television cameras
• CPC Class Codes: H04N 23/45 for generating image signals from two or more image sensors being of different type or operating in different modes, e.g. with a CMOS sensor for moving images in combination with a charge-coupled device [CCD] for still images
• CPC Class Codes: H04N 9/8042 involving data reduction
• CPC Class Codes: Y10T 29/49764 with testing or indicating

Abstract

A camera with multiple lenses and multiple sensors wherein each lens/sensor pair generates a sub-image of a final photograph or video. Different embodiments include: manufacturing all lenses as a single component; manufacturing all sensors as one piece of silicon; different lenses incorporate filters for different wavelengths, including IR and UV; non-circular lenses; different lenses are different focal lengths; different lenses focus at different distances; selection of sharpest sub-image; blurring of selected sub-images; different lens/sensor pairs have different exposures; selection of optimum exposure sub-images; identification of distinct objects based on distance; stereo imaging in more than one axis; and dynamic optical center-line calibration.

Claims

1. A camera comprising a plurality of lens/sensor pairs;

each lens configured to provide a sub-image on the corresponding sensor in the lens/sensor pair;

the corresponding sensor configured to provide a corresponding sub-image data set;

calibration data, for each lens/sensor pair, comprising the relative optical axis of each lens/sensor pair;

software configured to combine sub-image data from a plurality of lens/sensor pairs, responsive to the calibration data, to form a final digital image, wherein the software is in non-transitory memory;

storage means configured to store digital image data.

2. The camera of claim 1 wherein the calibration data additionally comprises a map of bad pixels.

3. The camera of claim 1 wherein a first lens/sensor pair additionally comprises a optical filter that passes light of a first spectra, and wherein a second lens/sensor pair additionally comprises a optical filter that passes light of a second spectra.

4. The camera of claim 1 wherein a first lens/sensor pair additionally comprises a optical filter that passes light of a first spectra;

and wherein a second lens/sensor pair additionally comprises a optical filter that passes light of a second spectra; and

wherein a third lens/sensor pair additionally comprises a optical filter that passes light of a third spectra; and

wherein the first, second and third spectra comprise light of red, green and blue light respectively.

5. The camera of claim 1 wherein a first lens/sensor pair additional comprises a optical filter that passes light of a first spectra, and wherein a second lens/sensor pair additionally comprises a optical filter that passes light of a second spectra;

and wherein the first spectra comprises visible light and the second spectra comprises infrared light; and

wherein the final image comprises data from both the first lens/sensor pair and the second lens/sensor pair.

6. The camera of claim 1 wherein the lens in a first lens/sensor pair comprises a first focal-length and the wherein the lens in a second lens/sensor pair comprises a second focal-length, and the second focal-length is numerically higher than the first focal-length;

and wherein the optical field of view of the second lens is contained with the optical field of view of the first lens; and

wherein the final image comprises data from both the first lens/sensor pair and the second lens/sensor pair.

7. The camera of claim 1 wherein the lens in a first lens/sensor pair comprises a first focal-length and the wherein the lens in a second lens/sensor pair comprises a second focal-length, and the second focal-length is numerically higher than the first focal-length;

and wherein the optical field of view of the second lens is contained with the optical field of view of the first lens; and

wherein the camera further comprises a means for a user of the camera to select a final image whose field of view is substantially similar to the field of view of the first lens or to select a final image whose field of view is substantially similar to the field of view of the second lens.

8. The camera of claim 1 wherein:

a first lens of a first lens/sensor pair has a first field of view;

a second lens in a second lens/sensor pair has a second field of view;

a third lens in a third lens/sensor pair has a third field of view;

the first field of view overlaps the second field of view;

the second field of view overlaps the third field of view;

the first field of view does not overlap the third field of view;

the final image data comprises sub-image data from the first and the second and the third lens/sensor pairs;

the final image data comprises a substantially continuous image.

9. The camera of claim 1 wherein:

a first lens of a first lens/sensor pair has a first focus distance and a first field of view;

a second lens in a second lens/sensor pair has a second focus distance and a second field of view;

the first field of view overlaps the second field of view;

the first and the second focus distances are different;

the final image data comprises sub-image data from the first and the second lens/sensor pairs;

the final image data comprises a substantially continuous image, wherein individual final pixels are selected to come from the first sub-image or the second sub-image responsive to the relative sharpness of the area surrounding the corresponding individual pixels in the first and the second sub-images.

10. The camera of claim 1 wherein:

a first lens/sensor pair additionally comprises a first optical filter that passes a first spectra;

a second lens/sensor pair additionally comprises a second optical filter that passes a second spectra;

the first and second spectra are different;

the first lens/sensor pair generates a first set of sub-image data;

the second lens/sensor pair generates a second set of sub-image data;

both the first lens of the first lens/sensor pair and the second lens of the second lens/sensor pair are free from one or more chromatic aberration correction elements that would be required in an alternative single lens to focus light of both the first spectra and the second spectra to produce an image with the same sharpness as the average sharpness of the images created responsive to the first and second sets of sub-image data.

11. The camera of claim 1 wherein:

a plurality of lens/sensor pairs are aggregated into a panorama set;

each lens/sensor pair in the panorama set comprises an optical axis;

the optical axis of each lens/sensor pair in the panorama set is non-parallel to the optical axis of all other lens/sensor pairs in the panorama set;

the field of view of each lens/sensor pair in the panorama set overlaps with the field of view of at least one other lens/sensor pair in the panorama set;

each lens/sensor pair in the panorama set generates sub-image data from a panorama exposure wherein each panorama exposure occurs at the same time for each lens/sensor pair in the panorama set;

the sub-image data from the panorama exposures are merged responsive to the calibration data for the lens/sensor pairs in the panorama set to create a final continuous panorama image wherein all picture elements in the final continuous panorama image are exposed at the same time.

12. The camera of claim 11 wherein:

the responsive merging of sub-image data from a first number of lens/sensor pairs responsive to perspective variation between the different sub-images wherein the perspective variation of the final continuous panorama image is comparable to that of a merged panorama image created from at least twice as many uncorrected sub-images as the first number.

13. A method of taking a photograph using the camera wherein the camera comprises a plurality of lens/sensor pairs;

and each lens is configured to provide a sub-image on the corresponding sensor in the lens/sensor pair;

the corresponding sensor configured to provide a corresponding sub-image data set;

wherein the steps comprise;

calibrating, comprising storing the relative optical axis of each lens/sensor pair;

photo-taking, wherein a user of the camera initiates a picture taking sequence within the camera;

the photo-taking sequence comprising;

generating an optical sub-image on the sensor in the each lens/pair using the lens in each lens/sensor pair;

generating a digital sub-image data corresponding to the optical sub-image for each lens/sensor pair;

correcting the digital sub-image data of said each sensor responsive to the calibration data stored for lens/sensor pair for that sensor;

combining the corrected digital sub-image data into a final digital image;

storing the final digital image.

14. The method of claim 13 with the further limitation:

the camera further comprises;

a first lens/sensor pair focused at a first distance;

a second lens/sensor pair focused at a second distance;

the first and second lens/sensor pairs comprise overlapping field of views;

the picture-taking step further comprises;

both the first and the second lens/sensor pair take an exposure at the same time;

the second generating step further comprises;

the first lens/sensor pair generates a first set of sub-image data;

the second lens/sensor pair generates a second set of sub-image data;

an additional comparing step between the correcting step and the combining step wherein the sharpness of an image area A in the first set of sub-image data with the sharpness of the same image area A in the second set of sub-image data;

an additional selection step after the comparing step wherein either the image area A from the first set of sub-image data or image area A from the second-set of sub-image data is selected;

the comparing and selection steps are repeated for additional image areas;

the combining step further comprises merging the image areas selected in the comparing and selection steps into a final image data set.

15. The method of claim 14 comprising the additional step, prior to the combining step, of:

performing an image-processing algorithm on the digital sub-image data from at least one lens/sensor pair.

16. The method of claim 15 wherein the image-processing algorithm is a blurring algorithm.

17. A method of manufacturing a camera comprising a plurality of lens/sensor pairs wherein:

each lens is configured to provide a sub-image on the corresponding sensor in the lens/sensor pair;

the corresponding sensor configured to provide a corresponding sub-image data set;

calibration data, for each lens/sensor pair, comprising the relative optical axis of each lens/sensor pair;

software configured to combine sub-image data from a plurality of lens/sensor pairs, responsive to the calibration data, to form a final digital image;

storage means configured to store digital image data;

wherein at least two of the lenses in the at least two lens/sensor pairs are formed as single piece.

18. The method of claim 17 wherein:

the camera further comprises a lens substrate designed to accept at least two insertable lenses of the lens/sensor pairs wherein the substrate is configured to position the insertable lenses in the proper optical position.

19. The method of claim 18 wherein:

the camera further comprises a monolithic sensor sheet further comprising the sensors of at least two lens/sensor pairs.

20. A method of taking a photograph using a camera comprising a plurality of lens/sensor pairs, wherein the camera comprises:

each lens in the lens/sensor pairs configured to provide a sub-image on the corresponding sensor in the lens/sensor pair;

the corresponding sensor configured to provide a corresponding sub-image data set;

calibration data, for each lens/sensor pair, comprising the relative optical axis of each lens/sensor pair;

software configured to combine sub-image data from a plurality of lens/sensor pairs, responsive to the calibration data, to form a final digital image;

storage means configured to store digital image data;

at least one IR lens/sensor pair configured to focus and use light in the infrared spectrum;

wherein the steps of the method comprise;

initiating a picture-taking sequence by the user of the camera;

turning on an infrared illuminator;

creating an exposure using the IR lens/sensor pair using the light generated by the infrared illuminator;

wherein the IR lens/sensor pair generates an IR sub-image data set;

while at the same time creating an exposure from a second lens/sensor pair using visible light;

wherein the second lens/sensor pair generates a visible sub-image data set;

turning off the infrared illuminator;

correcting the IR sub-image data set responsive to the calibration data stored for IR lens/sensor pair and correcting the visible light sub-image data set responsive to the calibration data stored for the visible lens/sensor pair;

combining the corrected IR sub-image data set with the corrected visible light sub-image data set into a final continuous color image;

storing the final continuous color image.