Plenoptic color imaging system with enhanced resolution

• US 9,706,116 B2
• Filed: 10/31/2013
• Issued: 07/11/2017
• Est. Priority Date: 10/31/2013
• Status: Expired due to Fees

• Current Assignee: Ricoh Company Limited
• Original Assignee: Ricoh Company Limited
• Inventors: Meng, Lingfei, Berkner, Kathrin
• Primary Examiner(s): Tissire, Abdelaaziz
• Application Number:
  US14/069,242
• Publication Number:
  US 20150116526A1
• Time in Patent Office: 1,349 Days
• Field of Search: 3482181, 348340
• CPC Class Codes: G06T 2207/10052 Images from lightfield camera
• CPC Class Codes: G06T 3/4053 based on super-resolution, i.e. the output image resolution being higher than the sensor resolution
• CPC Class Codes: G06T 5/50 using two or more images, e.g. averaging or subtraction
• CPC Class Codes: H04N 13/282 for generating image signals corresponding to three or more geometrical viewpoints, e.g. multi-view systems
• CPC Class Codes: H04N 23/16 Optical arrangements associated therewith, e.g. for beam-splitting or for colour correction
• CPC Class Codes: H04N 23/45 for generating image signals from two or more image sensors being of different type or operating in different modes, e.g. with a CMOS sensor for moving images in combination with a charge-coupled device [CCD] for still images
• CPC Class Codes: H04N 23/60 Control of cameras or camera modules
• CPC Class Codes: H04N 23/951 by using two or more images to influence resolution, frame rate or aspect ratio
• CPC Class Codes: H04N 25/134 based on three different wavelength filter elements

Abstract

Color plenoptic images captured by a spectrally-coded plenoptic imaging system are combined with higher resolution higher resolution images captured by a conventional imaging system, resulting in color images with higher resolution than those captured by the plenoptic imaging system alone.

Claims

1. An enhanced-resolution, plenoptic color imaging system comprising:

a spectrally-coded plenoptic imaging system for capturing a color plenoptic image of an object, the spectrally-coded plenoptic imaging system comprising;

first imaging optics that forms a first optical image of the object at an image plane of the first imaging optics, the first imaging optics having a pupil plane;

a plenoptic sensor module having a secondary imaging array and a first sensor array;

the secondary imaging array positioned at the image plane of the first imaging optics and the first sensor array positioned at a conjugate of the pupil plane; and

a color filter module comprising a plurality of different color filters including RGB filters, the color filter module positioned at a conjugate to the first sensor array; and

a higher resolution imaging system for capturing a conventional grayscale image of the object;

wherein the higher resolution imaging system and the spectrally-coded plenoptic imaging system are optically aligned to capture images of the same object, and the captured conventional grayscale image has a higher resolution than the captured color plenoptic image; and

a processing module that combines the captured higher resolution image and the captured color plenoptic image into a color image of the object based on a luminance component of the captured higher resolution image and on a luminance component of the captured color plenoptic image, said color image having a higher resolution than the captured color plenoptic image;

wherein the processing module converts RGB color components of the captured color plenoptic image to an HSV color space, performs histogram equalization between a luminance of the captured color plenoptic image converted to the HSV color space and the grayscale image, replaces the histogram-equalized luminance of the captured color plenoptic image converted to the HSV color space with the grayscale image, and converts from the HSV color space back to an RGB color space.

2. The enhanced-resolution, plenoptic color imaging system of claim 1 wherein:

the higher resolution imaging system and the spectrally-coded plenoptic imaging system share a front aperture;

the higher resolution imaging system comprises second imaging optics and a second sensor array, the second imaging optics forming a second optical image of the object and the second sensor array capturing said second optical image;

wherein the first imaging optics and the second imaging optics share the front aperture;

but the second sensor array and the plenoptic sensor module are implemented by different physical components.

3. The enhanced-resolution, plenoptic color imaging system of claim 2 wherein the first imaging optics and the second imaging optics are physically implemented by a shared imaging optics.

4. The enhanced-resolution, plenoptic color imaging system of claim 3 further comprising:

a beamsplitting device positioned downstream of the shared imaging optics, the beamsplitting device splitting light from the shared imaging optics between the second sensor array and the plenoptic sensor module.

5. The enhanced-resolution, plenoptic color imaging system of claim 4 wherein the beamsplitting device is a neutral density beamsplitter.

6. The enhanced-resolution, plenoptic color imaging system of claim 4 wherein the beamsplitting device is a dichroic beamsplitter.

7. The enhanced-resolution, plenoptic color imaging system of claim 4 wherein the beamsplitting device is a polarization beamsplitter.

8. The enhanced-resolution, plenoptic color imaging system of claim 3 further comprising:

a time-multiplexing device positioned downstream of the shared imaging optics, the time-multiplexing device directing light from the shared imaging optics alternately to the second sensor array and to the plenoptic sensor module.

9. The enhanced-resolution, plenoptic color imaging system of claim 8 wherein the time-multiplexing device is a chopper.

10. The enhanced-resolution, plenoptic color imaging system of claim 8 wherein the time-multiplexing device is a moveable mirror that can be moved to direct light from the shared imaging optics alternately to the second sensor array and to the plenoptic sensor module.

11. The enhanced-resolution, plenoptic color imaging system of claim 3 further comprising:

a mechanical mechanism for alternately positioning the second sensor array and the plenoptic sensor module to receive light from the shared imaging optics.

12. The enhanced-resolution, plenoptic color imaging system of claim 1 wherein:

the higher resolution imaging system comprises second imaging optics and a second sensor array, the second imaging optics forming a second optical image of the object and the second sensor array capturing said second optical image;

wherein the first imaging optics and the second imaging optics are physically implemented by a shared imaging optics; and

the first sensor array and the second sensor array are physically implemented by a shared sensor array.

13. The enhanced-resolution, plenoptic color imaging system of claim 12 further comprising:

a mechanical mechanism for alternately positioning and removing the secondary imaging array from between the shared imaging optics and the shared sensor array;

wherein the shared sensor array captures the spectrally-coded plenoptic image when the secondary imaging array is positioned between the shared imaging optics and the shared sensor array and the shared sensor array captures the optical image when the secondary imaging array is removed from between the shared imaging optics and the shared sensor array.

14. The enhanced-resolution, plenoptic color imaging system of claim 13 wherein the mechanical mechanism flips the secondary imaging array into and out of a position between the shared imaging optics and the shared sensor array.