Method for distinguishing a 3D image from a 2D image and for identifying the presence of a 3D image format by feature correspondence determination

• US 8,773,430 B2
• Filed: 12/09/2009
• Issued: 07/08/2014
• Est. Priority Date: 12/09/2009
• Status: Active Grant

• Current Assignee: InterDigital Madison Patent Holdings (InterDigital, Inc.)
• Original Assignee: Thomson Licensing (Vantiva SA)
• Inventors: Zhang, Tao
• Primary Examiner(s): Welch, David T
• Application Number:
  US13/514,622
• Publication Number:
  US 20120235996A1
• Time in Patent Office: 1,672 Days
• Field of Search: 345/419
• US Class Current: 345/419
• CPC Class Codes: G06T 19/00 Manipulating 3D models or images for computer graphics
• CPC Class Codes: G06T 2207/10012 Stereo images
• CPC Class Codes: G06T 7/593 from stereo images
• CPC Class Codes: H04N 13/139 Format conversion, e.g. of frame-rate or size
• CPC Class Codes: H04N 13/156 Mixing image signals
• CPC Class Codes: H04N 13/221 using the relative movement between cameras and objects
• CPC Class Codes: H04N 2213/007 Aspects relating to detection of stereoscopic image format, e.g. for adaptation to the display format

Abstract

A method identifies the presence of a three-dimensional (3D) image format in received image through the use of feature matching and correspondence. The received image is sampled using a candidate 3D format to generate two sub-images from the received image. Initially, these sub-images are compared to determine whether these sub-images are similar with respect to structure. If the sub-images are not similar, a new 3D format is selected and the method is repeated. If the sub-images are similar, features are detected in the two sub-images and a correspondence is formed between features that match in the two sub-images. Positional differences are computed between corresponding features. The amount and uniformity of the positional differences are then used to determine whether the format is 2D or 3D and, if 3D, which of the 3D formats was used for the received image.

Claims

1. A method for identifying the presence of a three-dimensional (3D) image format in a received image, the method comprising:

generating first and second sub-images from the received image using a sampling method corresponding to a candidate 3D format selected from a plurality of 3D formats;

detecting at least one feature in the first and second sub-images;

determining a correspondence between the at least one feature in the first sub-image and the at least one feature in the second sub-image;

computing positional differences between the at least one feature in the first sub-image that correspond to the at least one feature in the second sub-image;

comparing said first and second sub-images to determine whether the first and second sub-images are similar with respect to structure;

wherein, if said positional differences are determined to be uniform and less than or equal to a threshold, determining whether each 3D format of the plurality of 3D formats has been selected as a candidate 3D format;

wherein if the first and second sub-images are determined to be not similar in structure from said comparing, determining whether each 3D format of the plurality of 3D formats has been selected as a candidate 3D format; and

if all 3D formats in the plurality of 3D formats have been selected, identifying the received image as a two-dimensional (2D) format; and

if all 3D formats in the plurality of 3D formats have not been selected, selecting as the candidate 3D format a 3D format not previously selected from the plurality of 3D formats and repeating the generating and computing steps using the not previously selected 3D format; and

wherein, if said positional differences are determined not to be uniform and are determined to be greater than the threshold, identifying the received image as being formatted in a candidate 3D format used to make such determination; and

wherein, if the first and second sub-images are determined to be similar in structure from said comparing, executing the step of detecting one or more features in each of the first and second sub-images.

2. The method as defined in claim 1 wherein said determining a correspondence comprises feature tracking of the features detected in one sub-image with features detected in the other sub-image.

3. The method as defined in claim 2 wherein said feature tracking includes a Kanade-Lucas-Tomasi (KLT) tracking method.

4. The method as defined in claim 1 wherein said determining a correspondence comprises feature matching and feature correspondence.

5. The method as defined in claim 1 wherein said plurality of 3D formats includes formats selected from the group consisting of blended 3D formats, non-blended 3D formats, and color-based 3D formats.

6. The method as defined in claim 1 wherein said structure includes both structure and content.

7. The method as defined in claim 1 further including processing for display said received image according to the identified format.

8. The method as defined in claim 1 wherein said generating first and second sub-images further comprises filtering each of the first and second sub-images to be in a single channel of a plurality of channels so that said single channel of said first sub-image is identical to said single channel of said second sub-image.

9. The method as defined in claim 1 wherein said comparing said first and second sub-images to determine whether the first and second sub-images are similar with respect to structure comprises comparing at least one feature point in the first sub-image with at least a corresponding one feature point in the second sub-image.

10. The method as defined in claim 9 wherein said comparing said first and second sub-images to determine whether the first and second sub-images are similar with respect to structure further comprises detecting one or more features in each of said first and second sub-images.

11. The method as defined in claim 1 wherein said comparing said first and second sub-images to determine whether the first and second sub-images are similar with respect to structure further comprises evaluating a ratio of non-blank pixels in an edge map to a total number of pixels in said edge map as a measure of structure similarity.

12. An apparatus for identifying the presence of a three-dimensional (3D) image format in a received image, comprising:

generating first and second sub-images from the received image using a sampling method corresponding to a candidate 3D format selected from a plurality of 3D formats;

means for detecting at least one feature in the first and second sub-images;

means for determining a correspondence between the at least one feature in the first sub-image and the at least one feature in the second sub-image;

means for computing positional differences between the at least one feature in the first sub-image that correspond to the at least one feature in the second sub-image;

means for comparing said first and second sub-images to determine whether the first and second sub-images are similar with respect to structure;

wherein, if said positional differences are determined to be uniform and less than or equal to a threshold, determining whether each 3D format of the plurality of 3D formats has been selected as a candidate 3D format;

wherein if the first and second sub-images are determined to be not similar in structure from said comparing, determining whether each 3D format of the plurality of 3D formats has been selected as a candidate 3D format; and

if all 3D formats in the plurality of 3D formats have been selected, identifying the received image as a two-dimensional (2D) format; and

if all 3D formats in the plurality of 3D formats have not been selected, selecting as the candidate 3D format a 3D format not previously selected from the plurality of 3D formats and repeating the generating and computing steps using the not previously selected 3D format; and

wherein, if said positional differences are determined not to be uniform and are determined to be greater than the threshold, identifying the received image as being formatted in a candidate 3D format used to make such determination; and

wherein, if the first and second sub-images are determined to be similar in structure from said comparing, executing the detecting of one or more features in each of the first and second sub-images.

13. A non-transitory computer-readable medium having computer-executable instructions for execution by a processing system, the computer-executable instructions for identifying the presence of a three-dimensional (3D) image format in a received image, when executed, cause the processing system to:

generate first and second sub-images from the received image using a sampling method corresponding to a candidate 3D format selected from a plurality of 3D formats;

detect at least one feature in the first and second sub-images;

determine a correspondence between the at least one feature in the first sub-image and the at least one feature in the second sub-image;

compute positional differences between the at least one feature in the first sub-image that correspond to the at least one feature in the second sub-image;

compare said first and second sub-images to determine whether the first and second sub-images are similar with respect to structure;

wherein, if said positional differences are determined to be uniform and less than or equal to a threshold, determine whether each 3D format of the plurality of 3D formats has been selected as a candidate 3D format;

wherein if the first and second sub-images are determined to be not similar in structure from said comparing, determine whether each 3D format of the plurality of 3D formats has been selected as a candidate 3D format; and

if all 3D formats in the plurality of 3D formats have been selected, identify the received image as a two-dimensional (2D) format; and

if all 3D formats in the plurality of 3D formats have not been selected, select as the candidate 3D format a 3D format not previously selected from the plurality of 3D formats and repeat the generate and compute steps using the not previously selected 3D format; and

wherein, if said positional differences are determined not to be uniform and are determined to be greater than the threshold, identify the received image as being formatted in a candidate 3D format used to make such determination; and

wherein, if the first and second sub-images are determined to be similar in structure from said comparing, execute the detection of one or more features in each of the first and second sub-images.