Integrated extended depth of field (EDOF) and light field photography

• US 9,516,200 B2
• Filed: 12/29/2014
• Issued: 12/06/2016
• Est. Priority Date: 12/29/2014
• Status: Expired due to Fees

• Current Assignee: Samsung Electronics Co. Ltd.
• Original Assignee: Samsung Electronics Co. Ltd.
• Inventors: Small, Eran
• Primary Examiner(s): Do, Anh
• Application Number:
  US14/583,995
• Publication Number:
  US 20160191897A1
• Time in Patent Office: 708 Days
• Field of Search: 382/162, 382/167, 382/199, 348/187, 348/135, 348/266, 348/269, 348/272, 348/222.1, 348/223.1, 600/109, 600/129, 600/176, 600/111
• US Class Current: 1/1
• CPC Class Codes: G06T 2200/21 involving computational photography
• CPC Class Codes: G06T 2207/10052 Images from lightfield camera
• CPC Class Codes: G06T 2207/10148 Varying focus
• CPC Class Codes: G06T 5/50 using two or more images, e.g. averaging or subtraction
• CPC Class Codes: H04N 23/00 Cameras or camera modules comprising electronic image sensors; Control thereof
• CPC Class Codes: H04N 23/676 Bracketing for image capture at varying focusing conditions

Abstract

A method for light field acquisition, comprising: acquiring an image by an optical arrangement with chromatic aberrations; separating the image to a plurality of monochromatic images, each having a different color and a different point of focus according to the acquiring; transporting sharpness from each of the plurality of monochromatic images to others of the plurality of monochromatic images to construct a plurality of color images, each color image of the plurality of color images having a different point of focus; and reconstructing a light field by combining the plurality of color images.

Claims

1. A method for light field acquisition by a device including a processor, comprising:

acquiring, by the processor, an image from an optical arrangement with chromatic aberrations;

separating by the processor, said image into a plurality of monochromatic images, each having a different color and a different point of focus according to said acquiring;

transporting, by the processor, sharpness from each of said plurality of monochromatic images to others of said plurality of monochromatic images to construct a plurality of color images, each color image of said plurality of color images having a different point of focus; and

reconstructing, by the processor, a light field by combining said plurality of color images.

2. The method of claim 1, wherein said combining includes inversing, by the processor, the focusing operator of stack of said plurality of color images.

3. The method of claim 1, wherein said reconstructing comprises:

performing, by the processor, a two-dimensional Fourier transform on each of said plurality of color images to create a plurality of transformed color images;

calculating, by the processor, an inverted light field by pseudo-inversing each frequency for each of said plurality of color transformed images; and

performing, by the processor, a two-dimensional inverse Fourier transform on said inverted light field to create a true light field.

4. The method of claim 1, further comprising, after said reconstructing:

performing, by the processor, a re-focusing algorithm on said reconstructed light field.

5. The method of claim 1, further comprising, after said reconstructing:

producing, by the processor, an all in focus image from said reconstructed light field.

6. The method of claim 1, further comprising, after said reconstructing:

constructing, by the processor, a three-dimensional model of a scene of the image according to said reconstructed light field.

7. The method of claim 1, wherein said plurality of monochromatic images is of colors red, green and blue.

8. The method of claim 1, wherein said plurality of monochromatic images is of colors cyan, magenta and yellow.

9. A tangible computer readable storage medium comprising computer executable instructions adapted to perform the method of claim 1.

10. A method for light field reconstruction acquisition by a device including a processor, comprising:

acquiring, by the processor, a plurality of color images, each having a different point of focus;

performing, by the processor, a two-dimensional Fourier transform on each of said plurality of color images to create a plurality of transformed color images;

calculating, by the processor, an inverted light field by pseudo-inversing each frequency for each of said plurality of transformed color images; and

performing, by the processor, a two-dimensional inverse Fourier transform on said inverted light field to create a true light field.

11. The method of claim 10, further comprising, before said performing of said two-dimensional Fourier transform:

re-scaling, by the processor, each of said plurality of color images.

12. The method of claim 10, further comprising, before said performing of said two-dimensional Fourier transform:

separating, by the processor, each of said plurality of color images to a plurality of parts.

13. The method of claim 10, wherein at least one of said performing a two-dimensional Fourier transform, said calculating an inverted light field and said performing, by the processor, a two-dimensional inverse Fourier transform is done simultaneously using parallel computing.

14. The method of claim 10, wherein the calculating of the inverted light field comprises, pseudo-inversing, by the processor, each frequency by a Moore-Penrose algorithm.

15. An apparatus for light field acquisition, comprising:

an optical arrangement with chromatic aberrations, configured to acquire an image;

a storage configured to store instructions therein;

a processor that, when executing the instructions, is configured to;

separate said image to a plurality of monochromatic images, each having a different color and a different point of focus according to said acquiring,

construct a plurality of color images, each color image of said plurality of color images having a different point of focus, by transporting sharpness from of said plurality of monochromatic images to others of said plurality of monochromatic images, and

reconstruct the light field by combining said plurality of color images.

16. The apparatus of claim 15, wherein said processor is further configured to inverse the focusing operator of stack of said plurality of color images.

17. The apparatus of claim 15, wherein said processor is further configured to:

perform a two-dimensional Fourier transform on each of said plurality of color images to create a plurality of transformed color images,

calculate an inverted light field by pseudo-inversing each frequency for each of said plurality of color transformed images, and

perform a two-dimensional inverse Fourier transform on said inverted light field to create a true light field.

18. The apparatus of claim 15, wherein said optical arrangement is a camera that includes a lens having different focal plane for each color.

19. The apparatus of claim 15, wherein said processor is further configured to perform a re-focusing algorithm on said reconstructed light field.

20. The apparatus of claim 15, wherein said processor is further configured to produce an all in focus image from said reconstructed light field.

21. The apparatus of claim 15, wherein said processor is further configured to construct a three-dimensional model of a scene of the image according to said reconstructed light field.