STEREOSCOPIC VIDEO ENCODING AND DECODING METHODS AND APPARATUS

• US 20160219305A1
• Filed: 04/04/2016
• Published: 07/28/2016
• Est. Priority Date: 08/24/2009
• Status: Active Grant

• Current Assignee: Nevermind Capital LLC
• Original Assignee: NextVR, Inc. (Apple Inc.)
• Inventors: ROLLER, David Jr., Cole, David, Moss, Alan McKay
• Granted Patent: US 9,912,965 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: H04N 13/122 Improving the 3D impression of stereoscopic images by modifying image signal contents, e.g. by filtering or adding monoscopic depth cues H04N13/128 takes precedence
• CPC Class Codes: H04N 13/161 Encoding, multiplexing or demultiplexing different image signal components for multi-view video sequence encoding H04N19/597
• CPC Class Codes: H04N 19/00 Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
• CPC Class Codes: H04N 19/583 Motion compensation with overlapping blocks
• CPC Class Codes: H04N 19/597 specially adapted for multi-view video sequence encoding
• CPC Class Codes: H04N 19/82 involving filtering within a prediction loop
• CPC Class Codes: H04N 19/91 Entropy coding, e.g. variable length coding [VLC] or arithmetic coding
• CPC Class Codes: H04N 2213/007 Aspects relating to detection of stereoscopic image format, e.g. for adaptation to the display format

Abstract

Methods and apparatus for stereoscopic image encoding and decoding are described. Left and right eye images are encoded following an entropy reduction operation being applied to one of the eye images when there is a difference between the left and right images of an image pair. Information about regions of negative parallax within the entropy reduced image of an image pair is encoded along with the images. Upon decoding a sharpening filter is applied to the image in an image pair which was subjected to the entropy reduction operation. In addition edge enhancement filtering is performed on the regions of the recovered entropy reduced image which are identified in the encoded image data as regions of negative parallax. Interleaving of left and right eye images at the input of the encoder combined with entropy reduction allows for efficient encoding, storage, and transmission of 3D images.

Claims

1. A method of operating an encoder apparatus to encode stereoscopic image data representing pairs of first and second eye images, each first eye image in an image pair corresponding to one of a left eye image and a right eye image, the second eye image of each image pair corresponding to the other one of the left and right eye images, the method comprising:

determining an amount of difference between a first eye image and a second eye image of a first image pair;

when it is determined that there is a non-zero amount of difference between the first eye image and the second eye image of said first image pair, performing the steps of;

i) performing an entropy reduction operation on second eye image data representing the second eye image of the first image pair to produce entropy reduced second eye image data;

ii) encoding first eye image data representing the first one of the eye images of said first image pair to generate compressed first eye image data; and

iii) encoding said entropy reduced second eye image data to generate compressed second eye image data; and

storing or transmitting the compressed first eye image data and compressed second eye image data.

2. The method of claim 1, wherein said encoding said entropy reduced second eye image data includes supplying said entropy reduced second eye image data to an encoder which performs inter-frame motion compensated encoding, data corresponding to said first and second eye images being processed as data corresponding to sequential frames.

3. The method of claim 1, wherein said entropy reduction operation is a function of the determined amount of difference between said first eye image and said second eye image, the amount of entropy reduction being greater when the amount of difference is a first amount than when it is a second amount, said second amount being greater than said first amount.

4. The method of claim 1, wherein said entropy reduction operation includes performing at least one of an FFT based filtering operation or a neighborhood filtering operation on said second eye image data to smooth the second eye image of said first image pair.

5. The method of claim 1, further comprising:

storing information indicating said amount of difference in said file with said compressed first eye image data and compressed second eye image data.

6. The method of claim 1, further comprising:

when it is determined that there is a zero amount of difference between the first eye image and the second eye image of said first image pair performing, prior to performing said storing or transmitting step, the steps of;

ii) encoding said first eye image data to generate compressed first eye image data; and

iii) encoding said second eye image data to generate compressed second eye image data.

7. The method of claim 6, wherein said encoding said second eye image data generates compressed second eye image data indicating that the second eye image is the same as the first eye image.

8. The method of claim 1, further comprising:

processing a second image pair including a left eye image and a right eye image.

9. An apparatus to encode stereoscopic image data representing pairs of first and second eye images, each first eye image in an image pair corresponding to one of a left eye image and a right eye image, the second eye image of each image pair corresponding to the other one of the left and right eye images, the apparatus comprising:

a determination module for determining an amount of difference between a first eye image and a second eye image of a first image pair;

an entropy reduction filtering module for performing an entropy reduction operation on second eye image data representing the second eye image of the first image pair to produce entropy reduced second eye image data, when it is determined that there is a non-zero amount of difference between the first eye image and the second eye image of said first image pair;

a compression module for encoding first eye image data representing the first one of the eye images of said first image pair to generate compressed first eye image data, and for encoding said entropy reduced second eye image data to generate compressed second eye image data; and

a storage module for storing the compressed first eye image data and compressed second eye image data.

10. The apparatus of claim 9, further comprising:

an interface for transmitting the compressed first eye image data and compressed second eye image data.

11. The apparatus of claim 12, wherein said compression module for encoding said entropy reduced second eye image data includes an encoder which performs inter-frame motion compensated encoding, data corresponding to said first and second eye images being processed as data corresponding to sequential frames.

12. The apparatus of claim 9, wherein said entropy reduction operation is a function of the determined amount of difference between said first eye image and said second eye image, the amount of entropy reduction being greater when the amount of difference is a first amount than when it is a second amount, said second amount being greater than said first amount.

13. The apparatus of claim 9, wherein said entropy reduction filtering module includes a module for performing at least one of an FFT based filtering operation or a neighborhood filtering operation on said second eye image data to smooth the second eye image of said first image pair.

14. The apparatus of claim 9, wherein said storage module further stores information indicating said amount of difference in said file with said compressed first eye image data and compressed second eye image data.