Plenoptic Color Imaging System with Enhanced Resolution

• US 20150116526A1
• Filed: 10/31/2013
• Published: 04/30/2015
• Est. Priority Date: 10/31/2013
• Status: Active Grant

• Current Assignee: Ricoh Company Limited
• Original Assignee: Ricoh Company Limited
• Inventors: Meng, Lingfei, Berkner, Kathrin
• Granted Patent: US 9,706,116 B2
• Time in Patent Office: Days
• US Class Current: 348/218.1
• CPC Class Codes: G06T 2207/10052 Images from lightfield camera
• CPC Class Codes: G06T 3/4053 based on super-resolution, i.e. the output image resolution being higher than the sensor resolution
• CPC Class Codes: G06T 5/50 using two or more images, e.g. averaging or subtraction
• CPC Class Codes: H04N 13/282 for generating image signals corresponding to three or more geometrical viewpoints, e.g. multi-view systems
• CPC Class Codes: H04N 23/16 Optical arrangements associated therewith, e.g. for beam-splitting or for colour correction
• CPC Class Codes: H04N 23/45 for generating image signals from two or more image sensors being of different type or operating in different modes, e.g. with a CMOS sensor for moving images in combination with a charge-coupled device [CCD] for still images
• CPC Class Codes: H04N 23/60 Control of cameras or camera modules
• CPC Class Codes: H04N 23/951 by using two or more images to influence resolution, frame rate or aspect ratio
• CPC Class Codes: H04N 25/134 based on three different wavelength filter elements

Abstract

Color plenoptic images captured by a spectrally-coded plenoptic imaging system are combined with higher resolution higher resolution images captured by a conventional imaging system, resulting in color images with higher resolution than those captured by the plenoptic imaging system alone.

Claims

1. An enhanced-resolution, plenoptic color imaging system comprising:

a spectrally-coded plenoptic imaging system for capturing a color plenoptic image of an object;

a higher resolution imaging system for capturing a conventional image of the object;

wherein the higher resolution imaging system and the spectrally-coded plenoptic imaging system are optically aligned to capture images of the same object, and the captured conventional image has a higher resolution than the captured color plenoptic image; and

a processing module that combines the captured higher resolution image and the captured color plenoptic image into a color image of the object, said color image having a higher resolution than the captured color plenoptic image.

2. The enhanced-resolution, plenoptic color imaging system of claim 1 wherein:

the higher resolution imaging system and the spectrally-coded plenoptic imaging system share a front aperture;

the higher resolution imaging system comprises first imaging optics and a first sensor array, the first imaging optics forming an optical image of the object and the first sensor array capturing said optical image;

the spectrally-coded plenoptic imaging system comprises second imaging optics, a color filter module, and a plenoptic sensor module having a secondary imaging array and a second sensor array;

the second imaging optics, color filter module and secondary imaging array forming a spectrally-coded plenoptic image of the object and the second sensor array capturing said spectrally-coded plenoptic image;

wherein the first imaging optics and the second imaging optics share the front aperture;

but the first sensor array and the plenoptic sensor module are implemented by different physical components.

3. The enhanced-resolution, plenoptic color imaging system of claim 2 wherein the first imaging optics and the second imaging optics are physically implemented by a shared imaging optics.

4. The enhanced-resolution, plenoptic color imaging system of claim 3 further comprising:

a beamsplitting device positioned downstream of the shared imaging optics, the beamsplitting device splitting light from the shared imaging optics between the first sensor array and the plenoptic sensor module.

5. The enhanced-resolution, plenoptic color imaging system of claim 4 wherein the beamsplitting device is a neutral density beamsplitter.

6. The enhanced-resolution, plenoptic color imaging system of claim 4 wherein the beamsplitting device is a dichroic beamsplitter.

7. The enhanced-resolution, plenoptic color imaging system of claim 4 wherein the beamsplitting device is a polarization beamsplitter.

8. The enhanced-resolution, plenoptic color imaging system of claim 3 further comprising:

a time-multiplexing device positioned downstream of the shared imaging optics, the time-multiplexing device directing light from the shared imaging optics alternately to the first sensor array and to the plenoptic sensor module.

9. The enhanced-resolution, plenoptic color imaging system of claim 8 wherein the time-multiplexing device is a chopper.

10. The enhanced-resolution, plenoptic color imaging system of claim 8 wherein the time-multiplexing device is a moveable mirror that can be moved to direct light from the shared imaging optics alternately to the first sensor array and to the plenoptic sensor module.

11. The enhanced-resolution, plenoptic color imaging system of claim 3 further comprising:

a mechanical mechanism for alternately positioning the first sensor array and the plenoptic sensor module to receive light from the shared imaging optics.

12. The enhanced-resolution, plenoptic color imaging system of claim 1 wherein:

the higher resolution imaging system comprises first imaging optics and a first sensor array, the first imaging optics forming an optical image of the object and the first sensor array capturing said optical image;

the spectrally-coded plenoptic imaging system comprises second imaging optics, a color filter module, and a plenoptic sensor module having a secondary imaging array and a second sensor array;

the second imaging optics, color filter module and secondary imaging array forming a spectrally-coded plenoptic image of the object and the second sensor array capturing said spectrally-coded plenoptic image;

wherein the first imaging optics and the second imaging optics are physically implemented by a shared imaging optics; and

the first sensor array and the second sensor array are physically implemented by a shared sensor array.

13. The enhanced-resolution, plenoptic color imaging system of claim 12 further comprising:

a mechanical mechanism for alternately positioning and removing the secondary imaging array from between the shared imaging optics and the shared sensor array;

wherein the shared sensor array captures the spectrally-coded plenoptic image when the secondary imaging array is positioned between the shared imaging optics and the shared sensor array and the shared sensor array captures the optical image when the secondary imaging array is removed from between the shared imaging optics and the shared sensor array.

14. The enhanced-resolution, plenoptic color imaging system of claim 13 wherein the mechanical mechanism flips the secondary imaging array into and out of a position between the shared imaging optics and the shared sensor array.

15. The enhanced-resolution, plenoptic color imaging system of claim 13 further comprising:

a second mechanical mechanism for alternately positioning and removing the color filter module from a position in an optical path of the shared imaging optics.

16. The enhanced-resolution, plenoptic color imaging system of claim 1 wherein the color filter module comprises at least two color filters and at least one filter selected from a group consisting of a clear filter, a neutral density filter and a polarization filter.

17. The enhanced-resolution, plenoptic color imaging system of claim 1 wherein:

the higher resolution imaging system comprises first imaging optics and a first sensor array, the first imaging optics forming an optical image of the object and the first sensor array capturing said optical image;

the spectrally-coded plenoptic imaging system comprises second imaging optics, a color filter module, and a plenoptic sensor module having a secondary imaging array and a second sensor array;

the second imaging optics, color filter module and secondary imaging array forming a spectrally-coded plenoptic image of the object and the second sensor array capturing said spectrally-coded plenoptic image;

wherein the higher resolution imaging system and the spectrally-coded plenoptic imaging system are optically aligned upstream of a front aperture of each imaging system, but the first imaging optics and the second imaging optics are implemented by different physical components.

18. The enhanced-resolution, plenoptic color imaging system of claim 1 wherein the conventional image of the object is a grayscale image of the object.

19. The enhanced-resolution, plenoptic color imaging system of claim 1 wherein the conventional image of the object is a color image of the object, and the processing module combines a luminance component of the captured color image and the captured color plenoptic image.

20. A method for creating a color image with enhanced resolution, the method comprising:

capturing a color plenoptic image of an object;

capturing a conventional image of the object;

wherein the captured conventional image has a higher resolution than the captured color plenoptic image; and

combining the captured higher resolution image and the captured color plenoptic image into a color image of the object, said color image having a higher resolution than the captured color plenoptic image.