Method and apparatus for block-based compression of light-field images

• US 8,401,316 B2
• Filed: 03/23/2012
• Issued: 03/19/2013
• Est. Priority Date: 04/29/2008
• Status: Active Grant

• Current Assignee: Adobe Inc.
• Original Assignee: Adobe Systems Incorporated (Adobe Inc.)
• Inventors: Babacan, Sevket Derin, Georgiev, Todor G.
• Primary Examiner(s): Liu, Li
• Application Number:
  US13/429,226
• Publication Number:
  US 20120183232A1
• Time in Patent Office: 361 Days
• Field of Search: None
• US Class Current: 382/232
• CPC Class Codes: G02B 27/0075 with means for altering, e.g. increasing, the depth of field or depth of focus
• CPC Class Codes: H04N 19/176 the region being a block, e.g. a macroblock
• CPC Class Codes: H04N 19/597 specially adapted for multi-view video sequence encoding
• CPC Class Codes: H04N 19/60 using transform coding
• CPC Class Codes: H04N 19/90 using coding techniques not provided for in groups H04N19/10-H04N19/85, e.g. fractals
• CPC Class Codes: H04N 5/772 the recording apparatus and the television camera being placed in the same enclosure
• CPC Class Codes: H04N 5/91 Television signal processing therefor
• CPC Class Codes: H04N 9/8042 involving data reduction

Abstract

A method and apparatus for the block-based compression of light-field images. Light-field images may be preprocessed by a preprocessing module into a format that is compatible with the blocking scheme of a block-based compression technique, for example JPEG. The compression technique is then used to compress the preprocessed light-field images. The light-field preprocessing module reshapes the angular data in a captured light-field image into shapes compatible with the blocking scheme of the compression technique so that blocking artifacts of block-based compression are not introduced in the final compressed image. Embodiments may produce compressed 2D images for which no specific light-field image viewer is needed to preview the full light-field image. Full light-field information is contained in one compressed 2D image.

Claims

1. A method, comprising:

processing, by one or more computing devices, a light-field image to generate a processed light-field image, wherein the light-field image comprises a plurality of microimages, wherein dimensions of the microimages are different than blocking dimensions used by an image processing technique, and wherein said processing comprises;

estimating a microimage period for the light field image, wherein the microimage period indicates vertical and horizontal distances between the microimages in the light field image;

locating each microimage in the light field image according to the microimage period;

at each located microimage, extracting a microimage block from the light field image according to the microimage period;

resizing each extracted microimage block according to the blocking dimensions; and

storing each resized microimage block to a respective location in the processed light-field image according to the blocking dimensions.

2. The method as recited in claim 1, wherein the light-field image is captured with a plenoptic camera comprising an objective lens, a photosensor, and a microlens array comprising a plurality of microlenses located between the objective lens and the photosensor, wherein the plurality of microimages each correspond to one of the plurality of microlenses in the microlens array.

3. The method as recited in claim 1, wherein extracting a microimage block from the light field image according to the microimage period comprises cropping the respective located microimage to generate the microimage block, wherein vertical and horizontal dimensions of the microimage block are less than the vertical and horizontal dimensions of the microimage period.

4. The method as recited in claim 1, further comprising normalizing each microimage according to a corresponding region of a calibration image prior to said resizing.

5. The method as recited in claim 1, further comprising applying an interpolation technique to each resized microimage block.

6. The method as recited in claim 1, wherein said estimating a microimage period for the light field image comprises:

determining an average of pixel values in vertical and horizontal directions in the light-field image;

determining an autocorrelation of the average pixel values in the vertical and horizontal directions; and

determining the vertical and the horizontal distances between the microimages in the light field image according to the autocorrelations.

7. The method as recited in claim 1, wherein the image processing technique is a block-based compression technique, and wherein the method further comprises compressing the processed light-field image in accordance with the block-based compression technique to generate a compressed light-field image.

8. The method as recited in claim 7, wherein the block-based compression technique compresses the preprocessed light-field image in accordance with the JPEG (Joint Photographic Experts Group) compression standard.

9. The method as recited in claim 7, wherein the compressed light-field image is viewable using any application that is capable of accessing files compressed using the block-based compression technique.

10. The method as recited in claim 1, wherein the dimensions of the blocks in the processed light-field image are eight pixels by eight pixels.

11. The method as recited in claim 1, further comprising, prior to said processing the light-field image to generate a processed light-field image, normalizing the light-field image in accordance with a calibration image.

12. A system, comprising:

one or more processors; and

a memory comprising program instructions, wherein the program instructions are executable by at least one of the one or more processors to process a light-field image to generate a processed light-field image, wherein the light-field image comprises a plurality of microimages, wherein dimensions of the microimages are different than blocking dimensions used by an image processing technique, wherein, to process the light-field image, the program instructions are executable by at least one of the one or more processors to;

estimate a microimage period for the light field image, wherein the microimage period indicates vertical and horizontal distances between the microimages in the light field image;

locate each microimage in the light field image according to the microimage period;

at each located microimage, extract a microimage block from the light field image according to the microimage period;

resize each extracted microimage block according to the blocking dimensions; and

store each resized microimage block to a respective location in the processed light-field image according to the blocking dimensions.

13. The system as recited in claim 12, wherein, to extract a microimage block from the light field image according to the microimage period, the program instructions are executable by at least one of the one or more processors to crop the respective located microimage to generate the microimage block, wherein vertical and horizontal dimensions of the microimage block are less than the vertical and horizontal dimensions of the microimage period.

14. The system as recited in claim 12, wherein the program instructions are executable by at least one of the one or more processors to normalize each microimage according to a corresponding region of a calibration image prior to said resizing.

15. The system as recited in claim 12, wherein, to estimate a microimage period for the light field image, the program instructions are executable by at least one of the one or more processors to:

determine an average of pixel values in vertical and horizontal directions in the light-field image;

determine an autocorrelation of the average pixel values in the vertical and horizontal directions; and

determine the vertical and the horizontal distances between the microimages in the light field image according to the autocorrelations.

16. The system as recited in claim 12, wherein the image processing technique is a block-based compression technique, and wherein the program instructions are executable by at least one of the one or more processors to compress the processed light-field image in accordance with the block-based compression technique to generate a compressed light-field image.

17. A non-transitory computer-readable storage medium storing program instructions, wherein the program instructions are computer-executable to implement:

processing a light-field image to generate a processed light-field image, wherein the light-field image comprises a plurality of microimages, wherein dimensions of the microimages are different than blocking dimensions used by an image processing technique, and wherein, in said processing the light-field image, the program instructions are computer-executable to implement;

estimating a microimage period for the light field image, wherein the microimage period indicates vertical and horizontal distances between the microimages in the light field image;

locating each microimage in the light field image according to the microimage period;

at each located microimage, extracting a microimage block from the light field image according to the microimage period;

resizing each extracted microimage block according to the blocking dimensions; and

storing each resized microimage block to a respective location in the processed light-field image according to the blocking dimensions.

18. The non-transitory computer-readable storage medium as recited in claim 17, wherein, in said extracting a microimage block from the light field image according to the microimage period, the program instructions are computer-executable to implement cropping the respective located microimage to generate the microimage block, wherein vertical and horizontal dimensions of the microimage block are less than the vertical and horizontal dimensions of the microimage period.

19. The non-transitory computer-readable storage medium as recited in claim 17, wherein, in said estimating a microimage period for the light field image, the program instructions are computer-executable to implement:

determining an average of pixel values in vertical and horizontal directions in the light-field image;

determining an autocorrelation of the average pixel values in the vertical and horizontal directions; and

determining the vertical and the horizontal distances between the microimages in the light field image according to the autocorrelations.

20. The non-transitory computer-readable storage medium as recited in claim 17, wherein the image processing technique is a block-based compression technique, and wherein the program instructions are computer-executable to implement compressing the processed light-field image in accordance with the block-based compression technique to generate a compressed light-field image.