Two-dimensional to stereoscopic conversion systems and methods

• US 8,995,755 B2
• Filed: 07/24/2013
• Issued: 03/31/2015
• Est. Priority Date: 09/30/2011
• Status: Active Grant

• Current Assignee: Cyberlink Corporation
• Original Assignee: Cyberlink Corporation
• Inventors: Lu, Pei-Ying, Kuo, Chia-Chen, Cheng, Chih-Yu
• Primary Examiner(s): Fitzpatrick, Atiba O
• Application Number:
  US13/949,372
• Publication Number:
  US 20130336577A1
• Time in Patent Office: 615 Days
• Field of Search: None
• US Class Current: 382/154
• CPC Class Codes: G06F 18/00 Pattern recognition
• CPC Class Codes: G06T 2207/10024 Color image
• CPC Class Codes: G06T 7/194 involving foreground-background segmentation
• CPC Class Codes: G06T 7/215 Motion-based segmentation
• CPC Class Codes: G06T 7/579 from motion
• CPC Class Codes: H04N 13/15 for colour aspects of image signals
• CPC Class Codes: H04N 13/261 with monoscopic-to-stereoscopic image conversion
• CPC Class Codes: H04N 13/264 using the relative movement of objects in two video frames or fields
• CPC Class Codes: H04N 2013/0085 Motion estimation from stereoscopic image signals

Abstract

In one embodiment, a two-dimensional to stereoscopic conversion method, comprising: estimating a local motion region in a first image relative to one or more second images, the first and the one or more second images comprising two-dimensional images; generating a color model based on the local motion region; calculating a similarity value for each of at least one image pixel selected from the first image based on the color model; and assigning a depth value for each of the at least one image pixel selected from the first image based on the calculated similarity value to generate a stereoscopic image, the method performed by one or more processors.

Claims

1. A two-dimensional to stereoscopic conversion method, comprising:

estimating a local motion region in a first image relative to one or more second images, the first and the one or more second images comprising two-dimensional images;

generating a color model based on the local motion region;

calculating a similarity value for each of at least one image pixel selected from the first image based on the color model; and

assigning a depth value for each of the at least one image pixel selected from the first image based on the calculated similarity value to generate a stereoscopic image, the method performed by one or more processors.

2. The method of claim 1, wherein the one or more second images comprise one or more neighboring images.

3. The method of claim 2, wherein the one or more neighboring images comprise a forward image, a prior image, or a combination of both.

4. The method of claim 2, wherein the one or more neighboring images comprise an adjacent image, a non-adjacent image, or a combination of both.

5. The method of claim 1, wherein estimating further comprises:

using for the one or more second images or one or more neighboring images as a reference image or reference images to estimate motion of every region in the first image;

using the estimated motion to calculate global motion; and

obtaining the local motion by subtracting the global motion from the estimated motion.

6. The method of claim 5, wherein estimating motion of every region in the first region comprises estimating motion by a pixel-based method or a feature-based method.

7. The method of claim 6, wherein the pixel-based method comprises block-matching, optical flow or a phase correlation algorithm.

8. The method of claim 6, wherein the feature-based method comprises steps of detecting features and applying a statistical function.

9. The method of claim 6, wherein the feature-based method comprises a particle filter method.

10. The method of claim 5, wherein using the estimated motion to calculate global motion is according to an average of estimated motion of every region.

11. The method of claim 5, wherein using the estimated motion to calculate global motion is according to a mean-shift algorithm.

12. The method of claim 1, wherein generating further comprises determining one or more prime colors from image pixel colors based on a statistical clustering method.

13. The method of claim 1, wherein responsive to the similarity value being greater than a defined value, the assigning further comprises assigning a relatively high depth value to the at least one image pixel, wherein the high depth value corresponds to an object that is closer to the viewer than if the depth value was lower.

14. The method of claim 1, wherein responsive to the similarity value being less than a defined value, the assigning further comprises assigning a relatively low depth value to the at least one image pixel, wherein the low depth value corresponds to an object that is farther from the viewer than if the depth value was higher.

15. The method of claim 1, further comprising performing post-processing on the assigned depth value, wherein the post-processing comprises erosion processing, smooth processing, temporal processing, or spatial processing, or a combination of two or more of the post-processing.

16. The method of claim 1, wherein generating further comprises training the color model, wherein the training comprises collecting candidates in pixel regions of the first image with higher local motion, and classifying the candidates based on prime colors of the pixel regions.

17. A two-dimensional to stereoscopic conversion method, comprising:

estimating local motion in a first image relative to a neighboring image;

generating a color model based on the estimated local motion in the first image relative to the neighboring image;

computing a similarity value between plural pixels of the first image and the color model; and

adjusting a depth map based on the computed similarity value, the method performed by one or more processors.

18. The method of claim 17, wherein generating further comprises:

determining which of the plural pixels of the first image corresponds to a motion relative to the neighboring image that is greater than a predefined motion threshold; and

determining one or more prime colors from image pixel colors based on a statistical clustering method.

19. The method of claim 18, wherein determining the one or more prime colors further comprises performing a weighted average.

20. A two-dimensional to stereoscopic conversion method, comprising:

estimating a local motion region in a first image relative to one or more second images, the first and the one or more second images comprising two-dimensional images;

generating a color model based on the local motion region;

calculating a similarity value for each of at least one image pixel selected from the first image based on the color model;

estimating a local motion for each of the at least one image pixel selected from the first image, wherein the local motion comprises one or both of local motion distance or local motion direction; and

assigning a depth value for each of the at least one image pixel selected from the first image based on the calculated similarity value and the estimated local motion to generate a stereoscopic image, the method performed by one or more processors.

21. The method of claim 20, wherein responsive to the similarity value being greater than a defined value and the local motion distance of the estimated local motion being greater than a defined value, the assigning further comprises assigning a relatively high depth value to the at least one image pixel, wherein the high depth value corresponds to an object that is closer to the viewer than if the depth value was lower.

22. The method of claim 20, wherein responsive to the similarity value being greater than a defined value and the local motion direction of the estimated local motion being downward, the assigning further comprises assigning a relatively high depth value to the at least one image pixel, wherein the high depth value corresponds to an object that is closer to the viewer than if the depth value was lower.

23. A two-dimensional to stereoscopic conversion system, comprising:

one or more processors configured to;

execute an estimating module to estimate local motion in a first image relative to one or more second images;

execute a color model identifier module to generate a color model based on the estimated local motion;

compute a similarity value between plural pixels of the first image and the color model; and

execute a depth adjuster module to adjust a depth map based on the computed similarity value.

24. The system of claim 23, wherein the one or more second images include a forward image, a prior image, a non-adjacent image, or any combination thereof.

25. The system of claim 23, wherein the one or more processors are further configured to execute the estimating module to:

use the one or more second images as a reference image or reference images to estimate motion of every region in the first image;

use the estimated motion to calculate global motion; and

obtain the local motion by subtracting the global motion from the estimated motion.

26. The system of claim 23, wherein the one or more processors are further configured to execute the color model identifier module to generate by:

determining which of the plural pixels of the first image corresponds to a motion relative to the one or more second images that exceeds a predetermined motion threshold; and

classifying each of the plural pixels exceeding the predetermined motion threshold according to a defined feature, the defined feature comprising color.

27. The system of claim 23, wherein the one or more processors are further configured to compute by:

computing a similarity value in color between each of the plural pixels and each color classifier of the color model.

28. The system of claim 23, wherein responsive to the similarity value for one of the plural pixels being:

greater than a defined value, the one or more processors are further configured to execute the depth adjuster module to adjust by assigning a relatively high depth value to the one of the plural pixels, wherein the high depth value corresponds to an object that is closer to the viewer than if the depth value was lower; and

less than a defined value, the one or more processors are further configured to execute the depth adjuster module to adjust by assigning a relatively low depth value to the one of the plural pixels, wherein the low depth value corresponds to an object that is further from the viewer than if the depth value was higher.

29. The system of claim 23, wherein the computing is performed by the one or more processors executing a similarity value computation module.