Methods and apparatus for stereoscopic video compression, encoding, transmission, decoding and/or decompression

• US 8,451,320 B1
• Filed: 01/25/2010
• Issued: 05/28/2013
• Est. Priority Date: 01/23/2009
• Status: Active Grant

• Current Assignee: Nevermind Capital LLC
• Original Assignee: Next3D, Inc
• Inventors: Cole, David Michael, Moss, Alan McKay, Roller, David Jr.
• Primary Examiner(s): VO, TUNG T
• Application Number:
  US12/693,421
• Time in Patent Office: 1,219 Days
• Field of Search: 37524001-24026, 348 42- 43, 348 51- 52
• US Class Current: 348/43
• CPC Class Codes: H04N 13/161 Encoding, multiplexing or demultiplexing different image signal components for multi-view video sequence encoding H04N19/597
• CPC Class Codes: H04N 19/597 specially adapted for multi-view video sequence encoding

Abstract

Methods and apparatus for performing stereoscopic image encoding and decoding are described. Left and right eye images are generated. Image difference information is generated, e.g., a set of pixel values resulting from XORing the pixel values of the left and right eye images. One of the left and right eye images is compressed along with the difference map. The compressed image and compressed difference map is stored and/or transmitted. Stereoscopic images are generated by decompressing and using the received compressed image and compressed difference information. Prior to generation of the difference map the left and right eye images may be subject to a transposition operation to minimize the differences between the images and thus the size of the difference map. When transposition is applied, transposition information is stored and communicated in addition to the compressed image data so that the transposition can be reversed during the stereoscopic image generation process.

Claims

1. A method of encoding images, comprising:

tessellating a left eye image and a right eye image, said left eye image and right eye image corresponding to a left and right eye image pair;

performing transposition operations on said tessellated left and right eye images using transposition offsets which maximize redundancy between transposed tessellated left and right eye images generated by said transposition operations;

generating, from the transposed tessellated left and right eye images, a differential map, said differential map providing information indicating the difference between a reference one of the transposed tessellated left and right eye images and a non-reference one of the transposed tessellated left and right eye images;

performing a first compression operation on the reference one of the tessellated left and right eye images to generate a compressed reference image;

performing a second compression operation on the differential map to generate a compressed differential map; and

performing at least one of;

i) storing the compressed reference image and compressed differential map and ii) transmitting the compressed reference image and compressed differential map.

2. The method of claim 1, further comprising:

iteratively transposing said tessellated left and right images to determine transpositions which maximize redundancy between the resulting transposed tessellated left and right eye images.

3. The method of claim 2,

wherein said tessellated left eye image and tessellated right eye image are bitmap images including a plurality of tiles;

wherein maximum redundancy between the resulting transposed tessellated left and right eye images is determined based on the number of redundant tiles between said transposed tessellated left and right eye images; and

wherein said differential map is a bit map including non-zero values for tiles of the transposed tessellated left and right eye images which do not match.

4. The method of claim 3, wherein each of said non-zero values is a difference between a bitmap value in the reference image and a bitmap value in the non-reference image.

5. The method of claim 3, further comprising:

performing said tessellating, transposing, generating, first compression operation and second compression operation for each of a plurality of left and right eye image pairs in a sequence of left and right eye image pairs to thereby generate a sequence of compressed reference images and corresponding compressed differential maps.

6. The method of claim 3, wherein performing a first compression operation and performing a second compression operation each include performing compression in accordance with a compression standard, the same compression standard being used to compress the reference image and the corresponding differential map.

7. The method of claim 6, wherein said compression standard is one of:

i) a JPEG compression standard, ii) an MPEG compression standard and iii) a wavelet based compression standard.

8. The method of claim 1, further comprising:

prior to generating, from the transposed tessellated left and right eye images, a differential map;

storing information indicating an X offset and Y offset used to transpose one of the left eye image and right eye image to maximize the redundancy between the transposed tessellated left and right eye images prior to generating said differential map.

9. The method of claim 8, wherein the non-reference one of the transposed tessellated left eye image and right eye image is transposed using said indicated X and Y offsets prior to determination of said differential map.

10. The method of claim 8, wherein performing at least one of:

i) storing the compressed reference image and compressed differential map and ii) transmitting the compressed reference image and compressed differential map includes storing the compressed reference image and compressed differential map in a file along with said X offset and said Y offset.

11. A method of processing compressed image data, the method comprising:

decompressing a compressed reference image to generate a first previously transposed one of a left eye image and a right eye image;

performing a first transposition on said first previously transposed one of said left eye image and said right eye image to shift said first previously transposed one of said left eye image and said right eye image by an amount indicated by transposition information included in a file with said compressed reference image to generate a first processed image;

decompressing a compressed differential map to generate a decompressed differential map; and

generating a second previously transposed one of the left eye image and right eye image from said first previously transposed one of said left eye image and said right eye image and said decompressed differential map; and

performing a second transposition on said generated second previously transposed one of said left eye image and said right eye image to shift said second previously transposed one of said left eye image and right eye image by an amount indicated by said transposition information to generate a second processed image.

12. The method of claim 11, wherein performing a first transposition includes performing an image shift in a direction which is opposite the direction of an image shift performed by said second transposition.

13. The method of claim 12, wherein each of said first and second transposition reverses a transposition performed during encoding prior to generation of said differential map.

14. The method of claim 11, further comprising:

retrieving said compressed image and said compressed differential map from a file.

15. The method of claim 14, wherein said file includes transposition information in the form of an X offset and a Y offset to be applied to at least one of said left and right eye images.

16. The method of claim 15, wherein decompressing a compressed reference image includes performing one of:

i) a JPEG decompression operation, ii) an MPEG decompression operation and iii) a wavelet based decompression operation.

17. The method of claim 16, wherein decompressing a compressed differential map includes using the same decompression method used to decompress said compressed reference image to decompress said compressed differential map.

18. The method of claim 11 further comprising:

repeating, for each of a plurality of left and right image eye pairs for which image data is included in a file being processed, said steps of;

decompressing a compressed reference image to generate a first previously transposed one of a left eye image and a right eye image;

performing a first transposition on said first previously transposed one of said left eye image and said right eye image to shift said first previously transposed one of said left eye image and said right eye image by an amount indicated by transposition information included in a file with said compressed reference image to generate a first processed image;

decompressing a compressed differential map to generate a decompressed differential map;

generating a second previously transposed one of the left eye image and right eye image from said generated first previously transposed one of said left eye image and said right eye image and said decompressed differential map; and

performing a second transposition on said generated second previously transposed one of said left eye image and said right eye image to shift said second previously transposed one of said left eye image and right eye image by an amount indicated by said transposition information to generate a second processed image.

19. The method of claim 11, further comprising:

outputting said first processed image and said second processed image to a graphical processing unit;

operating the graphical processing unit to generate a single image from said first processed image and said second processed image, said graphical processing unit generating said single image as a function of convergence information provided by a user to whom the generated single image is to be displayed; and

displaying the generated single image on a display device.

20. A system for performing image compression, the system comprising:

a tessellating module for tessellating a left eye image and a right eye image, said left eye image and right eye image corresponding to a left and right eye image pair;

a processor configured to perform transposition operations on said tessellated left and right eye images using transposition offsets which maximize redundancy between transposed tessellated left and right eye images generated by said transposition operations;

a differential map generation module for generating, from the transposed tessellated left and right eye images, a differential map, said differential map providing information indicating the difference between a reference one of the transposed tessellated left and right eye images and a non-reference one of the transposed tessellated left and right eye images;

a compression module for performing a first compression operation on the reference one of the transposed tessellated left and right eye images to generate a compressed reference image, and for performing a second compression operation on the differential map to generate a compressed differential map; and

a storage device for storing the compressed reference image.

21. A system for processing compressed image data, comprising:

a storage device for storing a compressed reference image with a corresponding differential map;

a decompression module for decompressing said compressed reference image to generate a first previously transposed one of a left eye image and a right eye image and decompressing a compressed differential map to generate a decompressed differential map;

a reconstruction module for generating a second previously transposed one of the left eye image and right eye image from said generated first previously transposed one of said left eye image and said right eye image and said decompressed differential map; and

a processor configured to perform;

i) a first transposition on said first previously transposed one of said left eye image and said right eye image to shift said first previously transposed one of said left eye image and said right eye image by an amount indicated by image transposition information included in a file with said compressed reference image to generate a first output eye image, and ii) a second transposition on said generated second previously transposed one of said left eye image and said right eye image to shift said second previously transposed one of said left eye image and right eye image by an amount indicated by said transposition information.

22. The system of claim 21,

wherein said image transposition information is stored in said storage device;

and

wherein said each of said first transposition and said second transposition performed by said processor reverses a transposition performed during encoding prior to generation of said differential map.