Stereoscopic image processing methods and apparatus

• US 9,930,318 B2
• Filed: 11/30/2015
• Issued: 03/27/2018
• Est. Priority Date: 01/17/2012
• Status: Active Grant

• Current Assignee: Nevermind Capital LLC
• Original Assignee: NextVR, Inc. (Apple Inc.)
• Inventors: Cole, David, Moss, Alan McKay
• Primary Examiner(s): Lee, Michael
• Application Number:
  US14/954,590
• Publication Number:
  US 20160080728A1
• Time in Patent Office: 848 Days
• Field of Search: 348 46
• CPC Class Codes: H04N 13/128 Adjusting depth or disparity
• CPC Class Codes: H04N 13/161 Encoding, multiplexing or demultiplexing different image signal components for multi-view video sequence encoding H04N19/597
• CPC Class Codes: H04N 13/20 Image signal generators
• CPC Class Codes: H04N 13/239 using two 2D image sensors having a relative position equal to or related to the interocular distance H04N13/243 takes precedence
• CPC Class Codes: H04N 13/257 Colour aspects
• CPC Class Codes: H04N 13/271 wherein the generated image signals comprise depth maps or disparity maps
• CPC Class Codes: H04N 2013/0081 Depth or disparity estimation from stereoscopic image signals

Abstract

Stereoscopic image processing methods and apparatus are described. Left and right eye images of a stereoscopic frame are examined to determine if one or more difference reduction operations designed to reduce the luminance and/or chrominance differences between the left and right frames is within a range used to trigger a difference reduction operation. A difference reduction operation may involve assigning portions of the left and right frames to different depth regions and/or other region categories. A decision on whether or not to perform a difference reduction operation is then performed on a per regions basis with at the difference between the left and right eye portions of at least one region being reduced when a difference reduction operation is to be performed. The difference reduction process may, and in some embodiments is, performed in a precoder which processes left and right eye images of stereoscopic frames prior to stereoscopic encoding.

Claims

1. A method of processing a pair of images including a left eye image and a right eye image, each of said left and right eye images including image values of a first type, the method comprising:

generating a first overall disparity value indicating a first overall level of difference between values of said first type in said left eye image and values of said first type in said right eye image;

determining if said first overall disparity value is in a first difference range used to control performing of a first difference reduction operation, said first difference reduction operation being an operation on values of said first type included in at least one of said left eye image or said right eye image to reduce the overall difference between values of said first type in said left eye image and values of said first type in said right eye image; and

performing said first difference reduction operation to reduce the first overall level of difference between values of said first type in said left eye image and values of said first type in said right eye image when said first overall disparity value is determined to be in said first difference range;

wherein values of said first type are luminance values;

wherein performing said first difference reduction operation to reduce the first overall level of difference between values of said first type includes performing a difference reduction operation on luminance values;

the method further comprising;

performing a second difference reduction operation to reduce a level of difference between values of a second type included in said left eye image and said right eye image, said values of the second type being chrominance values, said step of performing a second difference reduction operation including performing a chrominance difference reduction operation which performs a different amount of difference reduction than is applied by said first difference reduction operation; and

wherein performing the second difference reduction operation includes making a maximum amount of change in chrominance values as part of a chrominance reduction operation which is lower than a maximum amount of change in luminance values performed during said first difference reduction operation.

2. The method of claim 1,

wherein determining if said first overall disparity value is in the first difference range used to control performing of the first difference reduction operation further includes:

comparing said first overall disparity value to a first threshold corresponding to a lower boundary of said first difference range; and

comparing said first overall disparity value to a second threshold corresponding to an upper boundary of said first difference range.

3. The method of claim 2, wherein determining if said first overall disparity value is in the first difference range used to control performing of the first difference reduction operation further includes:

determining that said overall disparity value is in the first difference range when said comparing steps indicate that said first overall disparity value is above said first threshold and below said second threshold.

4. The method of claim 1, wherein performing said first difference reduction operation includes:

modifying values of the first type in a first one of said left and right eye images to reduce an overall disparity between the values of the first type in a first region of the first one of said left and right eye images and the values of the first type included in a second one of said left and right eye images, said second one of said left and right eye images being different from said first one of said left and right eye images.

5. The method of claim 4, wherein performing said first difference reduction operation further includes:

generating a depth map for each of said left and right eye images; and

identifying different regions in said left and right eye images based on depths to which the regions are determined to correspond, said depths including at least a first image depth and a second image depth.

6. The method of claim 5, wherein said first image depth corresponds to a first distance range from a camera to objects in an image region identified as corresponding to said first image depth and a second image depth corresponds to a second distance range from the camera to objects in an image region identified as corresponding to said second image depth, said first and second image depths being different.

7. The method of claim 5, wherein said first and second image depths correspond to foreground and background image regions, respectively.

8. The method of claim 1, further comprising:

performing a third difference reduction operation to reduce a level of difference between values of a third type included in said left eye image and said right eye image, said values of the third type being chrominance values of a different type than said second type.

9. The method of claim 1, wherein generating a first overall disparity value includes generating a sum, the generated sum being the first overall disparity value.

10. The method of claim 9, wherein said sum is a sum of luminance difference values.

11. The method of claim 9, wherein said sum is equal to the difference between a sum of luminance values in said left eye image and a sum of luminance values in said right eye image.

12. The method of claim 1, wherein performing said first difference reduction operation to reduce the first overall level of difference between values of said first type in said left and right eye images is not performed on a first region of the left and right eye images while said first difference reduction operation is performed on another region of at least one of said left and right eye images.

13. The method of claim 12, wherein the first region of the left and right eye images on which the first difference reduction operation is not performed is a foreground image region;

and

wherein said another region of at least one of said left and right eye images upon which said first difference reduction operation is performed is a background region.