HIGH RESOLUTION THIN MULTI-APERTURE IMAGING SYSTEMS

• US 20150085174A1
• Filed: 11/23/2013
• Published: 03/26/2015
• Est. Priority Date: 11/28/2012
• Status: Active Grant

• Current Assignee: Corephotonics Ltd. (Samsung Electronics Co. Ltd.)
• Original Assignee: Corephotonics Ltd. (Samsung Electronics Co. Ltd.)
• Inventors: Shabtay, Gal, Cohen, Noy, Goldenberg, Ephraim, Gigushinski, Oded
• Granted Patent: US 9,538,152 B2
• Time in Patent Office: Days
• US Class Current: 348/336
• CPC Class Codes: G01J 3/0208 using focussing or collimating elements, e.g. lenses or mirrors; performing aberration correction
• CPC Class Codes: G01J 3/0229 using masks, aperture plates, spatial light modulators or spatial filters, e.g. reflective filters
• CPC Class Codes: G01J 3/0248 using a sighting port, e.g. camera or human eye
• CPC Class Codes: G01J 3/18 using diffraction elements, e.g. grating gratings per se G02B
• CPC Class Codes: G01J 3/2823 Imaging spectrometer
• CPC Class Codes: G01J 3/36 Investigating two or more bands of a spectrum by separate detectors
• CPC Class Codes: G02B 5/1814 structurally combined with one or more further optical elements, e.g. lenses, mirrors, prisms or other diffraction gratings G02B5/189 takes precedence
• CPC Class Codes: G02B 5/1842 Gratings for image generation G02B5/1847 takes precedence
• CPC Class Codes: G06T 11/60 Editing figures and text; Combining figures or text
• CPC Class Codes: G06T 2207/20221 Image fusion; Image merging
• CPC Class Codes: G06T 5/20 using local operators
• CPC Class Codes: G06T 7/00 Image analysis
• CPC Class Codes: G06T 7/33 using feature-based methods
• CPC Class Codes: G06T 7/337 involving reference images or patches
• CPC Class Codes: H04N 2209/045 using mosaic colour filter
• CPC Class Codes: H04N 23/00 Cameras or camera modules comprising electronic image sensors; Control thereof
• CPC Class Codes: H04N 23/10 for generating image signals from different wavelengths
• CPC Class Codes: H04N 23/11 for generating image signals from visible and infrared light wavelengths
• CPC Class Codes: H04N 23/13 with multiple sensors
• CPC Class Codes: H04N 23/16 Optical arrangements associated therewith, e.g. for beam-splitting or for colour correction
• CPC Class Codes: H04N 23/45 : for generating image signals from two or more image sensors being of different type or operating in different modes, e.g. with a CMOS sensor for moving images in combination with a charge-coupled device [CCD] for still images
• CPC Class Codes: H04N 23/60 : Control of cameras or camera modules
• CPC Class Codes: H04N 23/69 : Control of means for changing angle of the field of view, e.g. optical zoom objectives or electronic zooming
• CPC Class Codes: H04N 23/698 : for achieving an enlarged field of view, e.g. panoramic image capture
• CPC Class Codes: H04N 23/843 : Demosaicing, e.g. interpolating colour pixel values
• CPC Class Codes: H04N 23/88 : for colour balance, e.g. white-balance circuits or colour temperature control
• CPC Class Codes: H04N 23/951 : by using two or more images to influence resolution, frame rate or aspect ratio
• CPC Class Codes: H04N 25/133 : including elements passing panchromatic light, e.g. filters passing white light
• CPC Class Codes: H04N 25/134 : based on three different wavelength filter elements
• CPC Class Codes: H04N 25/135 : based on four or more different wavelength filter elements

Abstract

A multi-aperture imaging system comprising a first camera with a first sensor that captures a first image and a second camera with a second sensor that captures a second image, the two cameras having either identical or different FOVs. The first sensor may have a standard color filter array (CFA) covering one sensor section and a non-standard color CFA covering another. The second sensor may have either Clear or standard CFA covered sections. Either image may be chosen to be a primary or an auxiliary image, based on a zoom factor. An output image with a point of view determined by the primary image is obtained by registering the auxiliary image to the primary image.

Claims

1. A multi-aperture imaging system comprising:

a) a first camera subset that provides a first image, the first camera subset having a first sensor with a first plurality of sensor pixels covered at least in part with a non-standard color filter array (CFA), the non-standard CFA used to increase a specific color sampling rate relative to a same color sampling rate in a standard CFA;

b) a second camera subset that provides a second image, the second camera subset having a second sensor with a second plurality of sensor pixels, the second plurality of sensor pixels being either Clear or covered with a standard CFA; and

c) a processor configured to process the first and second images into a combined output image.

4. The imaging system of claim 1, wherein the non-standard CFA includes a repetition of a 4×

4 micro-cell in which a color filter order is BBRR-RBBR-RRBB-BRRB.

10. The imaging system of claim 1, wherein the processor is further configured to register respective first and second Luma images obtained from the first and second images during the processing of the first and second images into a combined output image, the registered first and second Luma images used together with color information to form the combined output image.

13. The imaging system of claim 1, wherein the first camera subset has a first field of view (FOV), wherein the second camera subset has a second, smaller FOV than the first FOV, and wherein the non-standard CFA covers substantially only an overlap area on the first sensor that captures the second FOV, thereby providing both optical zoom and increased color resolution.

2. (canceled)

3. (canceled)

5-9. -9. (canceled)

11. (canceled)

12. (canceled)

14-31. -31. (canceled)

32. A multi-aperture imaging system comprising:

a) a first camera subset that provides a first image, the first camera subset having a first field of view (FOV) and first sensor with a first plurality of sensor pixels covered at least in part with a standard CFA;

b) a second camera subset that provides a second image, the second camera subset having a second, smaller FOV than the first FOV and a second sensor with a second plurality of sensor pixels covered with a standard CFA; and

c) a processor configured to, for a zoom factor input that defines an FOV equal to or smaller than the second FOV, form an output image based on the second image.

33. A multi-aperture imaging system comprising:

a) a first camera subset that provides a first image, the first camera subset having a first field of view (FOV₁) and a first sensor with a first plurality of sensor pixels covered at least in part with a standard color filter array (CFA);

b) a second camera subset that provides a second image, the second camera subset having a second field of view (FOV₂) such that FOV₂<

FOV₁ and a second sensor with a second plurality of sensor pixels, the second plurality of sensor pixels being either Clear or covered with a standard CFA; and

c) a processor configured to provide an output image from a point of view of either the first or the second camera subset based on a zoom factor (ZF) input that defines a respective field of view (FOV_ZF).

34. The imaging system of claim 33, wherein the processor is further configured to form the output image by processing the first and second images into a combined output image.

35. The imaging system of claim 34, wherein if FOV₂<

FOV_ZF≦

FOV₁, then the point of view of the output image is that of the first camera and if FOV₂≧

FOV_ZF, then the point of view of the output image is that of the second camera.

36. The imaging system of claim 34, wherein the processor is further configured to, during the processing of the first and second images into a combined output image and based on the ZF input, register respective first and second Luma images obtained from the first and second images, the registered first and second Luma images used together with color information to form the combined output image.

37. The imaging system of claim 36, wherein if FOV₂<

FOV_ZF≦

FOV₁ then the registration includes finding a corresponding pixel in the second Luma image for each pixel in the first Luma image and the processor is further configured to form the combined output image by transferring information from the second image to the first image, and if FOV₂≧

FOV_ZF then the registration includes finding a corresponding pixel in the first Luma image for each pixel in the second Luma image and the processor is further configured to form the combined output image by transferring information from the first image to the second image.

38. The imaging system of claim 33, wherein the first plurality of sensor pixels is covered at least in part with a non-standard CFA, the non-standard CFA used to increase a specific color sampling rate relative to a same color sampling rate in a standard CFA.

39. The imaging system of claim 38, wherein if FOV₂<

FOV_ZF≦

FOV₁, then the point of view of the output image is that of the first camera and if FOV₂≧

FOV_ZF, then the point of view of the output image is that of the second camera.

40. The imaging system of claim 38, wherein the processor is further configured to form the output image by processing the first and second images into a combined output image.

41. The imaging system of claim 40, wherein the processor is further configured to, during the processing of the first and second images into a combined output image and based on the ZF input, register respective first and second Luma images obtained from the first and second images, the registered first and second Luma images used together with color information to form the combined output image.

42. The imaging system of claim 41, wherein if FOV₂<

FOV_ZF≦

FOV₁ then the registration includes finding a corresponding pixel in the second Luma image for each pixel in the first Luma image and the processor is further configured to form the combined output image by transferring information from the second image to the first image, and if FOV₂≧

FOV_ZF then the registration includes finding a corresponding pixel in the first Luma image for each pixel in the second Luma image and the processor is further configured to form the combined output image by transferring information from the first image to the second image.

43. The imaging system of claim 38, wherein the non-standard CFA covers substantially only an overlap area on the first sensor that captures the second FOV, thereby providing both optical zoom and increased color resolution.