Systems and methods for generating compressed light field representation data using captured light fields, array geometry, and parallax information

• US 10,009,538 B2
• Filed: 05/19/2017
• Issued: 06/26/2018
• Est. Priority Date: 02/21/2013
• Status: Active Grant

• Current Assignee: FotoNation Limited (Adeia Inc.)
• Original Assignee: Fotonation Cayman Limited
• Inventors: Venkataraman, Kartik, Lelescu, Dan, Molina, Gabriel
• Primary Examiner(s): Adams, Eileen M
• Application Number:
  US15/599,900
• Publication Number:
  US 20170257562A1
• Time in Patent Office: 403 Days
• Field of Search: None
• CPC Class Codes: G06T 7/50 Depth or shape recovery
• CPC Class Codes: G06T 7/60 Analysis of geometric attributes
• CPC Class Codes: H04N 13/271 wherein the generated image signals comprise depth maps or disparity maps
• CPC Class Codes: H04N 23/00 Cameras or camera modules comprising electronic image sensors; Control thereof
• CPC Class Codes: H04N 23/64 Computer-aided capture of images, e.g. transfer from script file into camera, check of taken image quality, advice or proposal for image composition or decision on when to take image

Abstract

Systems and methods for the generating compressed light field representation data using captured light fields in accordance embodiments of the invention are disclosed. In one embodiment, an array camera includes a processor and a memory connected configured to store an image processing application, wherein the image processing application configures the processor to obtain image data, wherein the image data includes a set of images including a reference image and at least one alternate view image, generate a depth map based on the image data, determine at least one prediction image based on the reference image and the depth map, compute prediction error data based on the at least one prediction image and the at least one alternate view image, and generate compressed light field representation data based on the reference image, the prediction error data, and the depth map.

Claims

1. A method for decoding compressed light field representation data comprising:

obtaining, using a processor, compressed light field representation data, wherein the compressed light field representation data contains a compressed representation of a plurality of images of a scene captured from multiple points of view comprising a compressed representation of a reference image, a depth map, and prediction error data;

decompressing a reference image, a depth map, and prediction error data from the compressed light field representation data;

forming at least one alternate view image by;

computing at least one prediction image using the reference image and the depth map; and

applying the prediction error data to modify the pixels of the at least one computed prediction image to generate the at least one alternate view image;

wherein the reference image and the at least one alternative view image form a light field comprising a plurality of images captured from multiple points of view.

2. The method of claim 1, wherein the reference image corresponds to a viewpoint image in the compressed light field representation data.

3. The method of claim 2, wherein the viewpoint image corresponds to a focal plane in an imager array.

4. The method of claim 1, wherein the reference image corresponds to a virtual viewpoint that does not correspond to a focal plane in an imager array.

5. The method of claim 1, wherein the at least one prediction image is computed using metadata describing interpolation techniques utilized in the creation of the compressed light field representation data.

6. The method of claim 1, wherein the light field further comprises the depth map, the prediction error data, and other metadata included in the compressed light field representation data.

7. The method of claim 1, wherein the compressed light field representation data comprises a plurality of reference images.

8. The method of claim 7, further comprising recursively forming alternate view images from the viewpoint of each reference image.

9. The method of claim 1, wherein the compressed light field representation data comprises the at least one alternate view image.

10. The method of claim 1, wherein the compressed light field representation data comprises metadata and prediction error data providing information that compensates for information lost during a lossy compression.

11. The method of claim 10, wherein determining the reference image comprises applying the metadata and the prediction error data to a compressed image in order to reconstruct an original, uncompressed image.

12. The method of claim 1, wherein the depth map describes the distance from the viewpoint of the reference image with reference to objects imaged by pixels within the reference image.

13. The method of claim 1, wherein a portion of the prediction error data describes a difference in photometric information between a pixel in a prediction image and a pixel in at least one alternate view image corresponding to the prediction image.

14. The method of claim 1, wherein the reference image is a virtual image interpolated from a virtual viewpoint within the image data.