3-D light field camera and photography method

• US 10,397,545 B2
• Filed: 12/23/2014
• Issued: 08/27/2019
• Est. Priority Date: 12/23/2013
• Status: Active Grant

• Current Assignee: University of Delaware
• Original Assignee: University of Delaware
• Inventors: Yu, Jingyi, Guo, Xinqing, Yu, Zhan
• Primary Examiner(s): Haskins, Twyler L
• Assistant Examiner(s): Tejano, Dwight Alex C
• Application Number:
  US15/107,661
• Publication Number:
  US 20160330432A1
• Time in Patent Office: 1,708 Days
• Field of Search: None
• CPC Class Codes: G02B 3/005 arranged along a single direction only, e.g. lenticular sheets G02B3/0043 takes precedence
• CPC Class Codes: G06T 15/205 Image-based rendering
• CPC Class Codes: G06T 2200/04 involving 3D image data
• CPC Class Codes: G06T 2200/28 involving image processing hardware
• CPC Class Codes: G06T 2207/10012 Stereo images
• CPC Class Codes: H04N 13/111 Transformation of image signals corresponding to virtual viewpoints, e.g. spatial image interpolation
• CPC Class Codes: H04N 13/128 Adjusting depth or disparity
• CPC Class Codes: H04N 13/207 using a single 2D image sensor
• CPC Class Codes: H04N 13/232 using fly-eye lenses, e.g. arrangements of circular lenses
• CPC Class Codes: H04N 13/305 using lenticular lenses, e.g. arrangements of cylindrical lenses

Abstract

Methods and systems for generating three-dimensional (3D) images, 3D light field (LF) cameras and 3D photographs are provided. Light representing a scene is directed through a lens module coupled to an imaging sensor. The lens module includes: a surface having a slit-shaped aperture and a cylindrical lens array positioned along an optical axis of the imaging sensor. A longitudinal direction of the slit-shaped aperture is arranged orthogonal to a cylindrical axis of the cylindrical lens array. The light directed through the lens module is captured by the imaging sensor to form a 3D LF image. A 3D photograph includes a 3D LF printed image of the scene and a cylindrical lens array disposed on the printed image, such that the combination of 3D LF printed image and the cylindrical lens array forms a 3D stereoscopic image.

Claims

1. A method of generating an image of a scene, the method comprising:

directing light representing the scene through a lens module coupled to an imaging sensor, the lens module including;

a surface having a slit-shaped aperture and a cylindrical lens array positioned along an optical axis of the imaging sensor, a longitudinal direction of the slit-shaped aperture being arranged orthogonal to a cylindrical axis of the cylindrical lens array;

capturing, by the imaging sensor, the light directed through the lens module to form a three-dimensional (3D) light field (LF) image; and

processing, by a processor, the 3D LF image to form a rendered image based on pitch size, focal length and an image center of each lens in the cylindrical lens array, wherein the processing includes at least one of;

(a) refocusing the 3D LF image to a predetermined focus depth, including prior to refocusing the 3D LF image, locating the image center of each lens in the cylindrical lens array, based on a reference image, (b) adjusting a perspective of the 3D LF image based on a predetermined viewpoint, or (c) generating a 3D stereoscopic view image from the 3D LF image.

2. The method according to claim 1, wherein the directing of the light includes directing the light through the slit-shaped aperture onto a lens of the lens module to form aperture-shaped light and passing the aperture-shaped light from the lens through the cylindrical lens array onto the imaging sensor.

3. The method according to claim 1, wherein the reference image is a 3D LF image captured of a reference scene.

4. The method according to claim 1, the method further including displaying at least one of the 3D LF image or the rendered image.

7. The method according to claim 1, further comprising repeating the directing of the light and the capturing of the directed light to capture a plurality of 3D LF images of the scene.

5. A method of generating an image of a scene, the method comprising:

directing light representing the scene through a lens module coupled to an imaging sensor, the lens module including;

a surface having a slit-shaped aperture and a cylindrical lens array positioned along an optical axis of the imaging sensor, a longitudinal direction of the slit-shaped aperture being arranged orthogonal to a cylindrical axis of the cylindrical lens array; and

capturing, by the imaging sensor, the light directed through the lens module to form a three dimensional (3D) light field (LF) image;

processing, by a processor, the 3D LF image to form a rendered image, wherein the processing includes at least one of refocusing the 3D LF image to a predetermined focus depth, adjusting a perspective of the 3D LF image based on a predetermined viewpoint or generating a 3D stereoscopic view image from the 3D LF image; and

printing at least one of the 3D LF image or the rendered image to form a printed image.

6. The method according to claim 5, the method further including disposing the printed image on a further cylindrical lens array to form a 3D photograph.

8. A system for generating an image of a scene, comprising:

a three-dimensional (3D) light field (LF) camera comprising;

a surface having a slit-shaped aperture mounted on a lens;

an imaging sensor configured to capture at least one 3D LF image of a scene; and

a cylindrical lens array disposed between the imaging sensor and the lens, along an optical axis of the imaging sensor, a longitudinal direction of the slit-shaped aperture being arranged orthogonal to a cylindrical axis of the cylindrical lens array; and

a processor configured for;

processing the 3D LF image to form a rendered image based on pitch size, focal length and an image center of each lens in the cylindrical lens array, wherein the processing includes at least one of;

(a) refocusing the 3D LF image to a predetermined focus depth, including prior to refocusing the 3D LF image, locating the image center of each lens in the cylindrical lens array, based on a reference image, (b) adjusting a perspective of the 3D LF image based on a predetermined viewpoint, or (c) generating a 3D stereoscopic view image from the 3D LF image.

9. The 3D LF camera according to claim 8, wherein:

the surface, the lens and the cylindrical lens array are disposed in a lens module, and

the lens module is coupled to a camera including the imaging sensor.

10. The 3D LF camera according to claim 9, wherein the lens module is configured to be detachably coupled to the camera.

11. The 3D LF camera according to claim 9, wherein the camera includes a digital single-lens reflex (DSLR) camera.

12. The 3D LF camera according to claim 8, wherein the cylindrical lens array includes a plurality of cylindrical lens arranged to extend in a vertical direction.

13. The 3D LF camera according to claim 8, wherein the at least one 3D LF image includes a plurality of 3D LF images of the scene.

14. The 3D LF camera according to claim 8, wherein the cylindrical lens array is configured to provide higher resolution in a first direction corresponding to the cylindrical axis and higher angular information in a second direction opposite the first direction.

15. The 3D LF camera according to claim 8, wherein the slit-shaped aperture is configured to reduce defocus blurring.

16. A three-dimensional (3D) photograph comprising:

a 3D light field printed image of a scene; and

a first cylindrical lens array disposed on the 3D light field printed image, such that the combination of the 3D light field printed image and the first cylindrical lens array forms a 3D stereoscopic image;

wherein the 3D light field printed image comprises an image rendered by;

directing light representing the scene through a lens module coupled to an imaging sensor, the lens module including;

a surface having a slit-shaped aperture and a second cylindrical lens array positioned along an optical axis of the imaging sensor, a longitudinal direction of the slit-shaped aperture arranged orthogonal to a cylindrical axis of the second cylindrical lens array;

capturing, by the imaging sensor, the light directed through the lens module to form a three-dimensional (3D) light field (LF) image;

processing, by a processor, the 3D LF image by at least one of refocusing the 3D LF image to a predetermined focus depth, adjusting a perspective of the 3D LF image based on a predetermined viewpoint or generating a 3D stereoscopic view image from the 3D LF image.

17. The 3D photograph according to claim 16, wherein the 3D photograph is an autostereoscopic 3D display.

18. The 3D photograph according to claim 16, wherein the 3D light field printed image is captured via a camera comprising the imaging sensor and the lens module, and wherein the 3D light field printed image includes a set of sub-aperture images corresponding to the second cylindrical lens array.

19. The 3D photograph according to claim 18, wherein the 3D light field printed image is disposed on the first cylindrical lens array such that the set of sub-aperture images are arranged parallel to a cylindrical axis of the first cylindrical lens array.