IMAGE PROCESSING AND ENCODING

• US 20160212403A1
• Filed: 01/20/2016
• Published: 07/21/2016
• Est. Priority Date: 01/21/2015
• Status: Active Grant

• Current Assignee: Nevermind Capital LLC
• Original Assignee: NextVR, Inc. (Apple Inc.)
• Inventors: Cole, David, Moss, Alan McKay
• Granted Patent: US 11,218,682 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: H04N 13/128 Adjusting depth or disparity
• CPC Class Codes: H04N 13/144 for flicker reduction
• CPC Class Codes: H04N 13/161 Encoding, multiplexing or demultiplexing different image signal components for multi-view video sequence encoding H04N19/597
• CPC Class Codes: H04N 2013/0081 Depth or disparity estimation from stereoscopic image signals

Abstract

Encoding and streaming methods and apparatus are described. Objects in negative parallax in frame pairs, e.g., pairs of left and right eye images forming a stereoscopic image, are identified. An amount of negative parallax reduction implemented depends, in some embodiments, on the data rate being used for encoding and/or the amount of negative parallax detected in the frame pair to be encoded. The lower the supported data rate the greater the reduction in negative parallax in some embodiments. In some, but not all, embodiments objects in positive parallax, e.g., objects appearing to go into the page, are not subject to parallax reduction. When a lowest supported data rate is used mono encoding is used and parallax reduction steps are skipped. The same frame pair is encoded multiple times at different data rates. Different amounts of negative parallax reduction are performed for at least some of the different supported data rates.

Claims

1. A method of encoding stereoscopic image data, the method comprising:

determining if a frame pair to be encoded includes an object in negative parallax; and

determining, when a frame pair to be encoded includes an object in negative parallax, an amount of negative parallax reduction to be performed as a function of an encoding data rate to be used for encoding.

2. The method of claim 1, wherein said determining an amount of negative parallax reduction determines a greater amount of negative parallax reduction when said encoding data rate to be used is a first encoding rate than when a second encoding data rate which is higher than said first data rate is to be used.

3. The method of claim 2, further comprising:

identifying a closest object in the frame pair with negative parallax; and

determining a parallax difference in the identified closest object in the frame pair with negative parallax.

4. The method of claim 3, further comprising:

wherein determining the amount of negative parallax reduction to be performed includes;

determining the amount of parallax reduction to be performed, based on the determined parallax difference in the identified closest object in the frame pair with negative parallax as well as the encoding data rate to be used.

5. The method of claim 4, further comprising:

performing the determined amount of negative parallax reduction on the frame pair to produced a processed frame pair with less negative parallax than said frame pair;

and

encoding the processed frame pair at the encoding data rate to be used for encoding the frame pair.

6. The method of claim 5, further comprising:

processing the frame pair to be encoded using the second encoding data rate which is higher than said first encoding data rate;

wherein processing the frame pair to be encoded at the second data rate includes performing a second amount of negative parallax reduction which is less than said first amount of parallax reduction; and

encoding the frame pair at said second encoding data rate.

7. The method of claim 6, further comprising:

encoding said frame pair at a lowest supported data rate using a mono encoding method in which a single image is encoded for the frame pair which included left and right eye images.

8. An image processing system (300) for processing stereoscopic image data, said image processing system (300) comprising:

a processor (312) configured to;

determine if a frame pair to be encoded includes an object in negative parallax; and

determine, when a frame pair to be encoded includes an object in negative parallax, an amount of negative parallax reduction to be performed as a function of an encoding data rate to be used for encoding; and

memory (316) coupled to said processor (312).

9. The image processing system (300) of claim 8, wherein said processor (312) is configured to:

determine a greater amount of negative parallax reduction when said encoding data rate to be used is a first encoding rate than when a second encoding data rate which is higher than said first data rate is to be used, as part of being configured to determine an amount of negative parallax reduction.

10. The image processing system (300) of claim 9, wherein said processor (312) is further configured to:

identify a closest object in the frame pair with negative parallax; and

determine a parallax difference in the identified closest object in the frame pair with negative parallax.

11. The image processing system (300) of claim 10, wherein said processor (312) is configured to:

determine the amount of parallax reduction to be performed, based on the determined parallax difference in the identified closest object in the frame pair with negative parallax as well as the encoding data rate to be used, as part of being configured to determine the amount of negative parallax reduction to be performed.

12. The image processing system (300) of claim 11, wherein said processor (312) is further configured to:

perform the determined amount of negative parallax reduction on the frame pair to produced a processed frame pair with less negative parallax than said frame pair; and

encode the processed frame pair at the encoding data rate to be used for encoding the frame pair.

13. The image processing system (300) of claim 12, wherein said processor (312) is further configured to:

process the frame pair to be encoded using the second encoding data rate which is higher than said first encoding data rate, wherein processing the frame pair to be encoded at the second data rate includes performing a second amount of negative parallax reduction which is less than said first amount of parallax reduction; and

encode the frame pair at said second encoding data rate.

14. The image processing system (300) of claim 13, wherein said processor (312) is further configured to:

encode said frame pair at a lowest supported data rate using a mono encoding method in which a single image is encoded for the frame pair which included left and right eye images.

15. A non-transitory computer readable medium including processor executable instructions which when executed by a processor control an image processing system to:

determine if a frame pair to be encoded includes an object in negative parallax; and

determine, when a frame pair to be encoded includes an object in negative parallax, an amount of negative parallax reduction to be performed as a function of an encoding data rate to be used for encoding.