DEPTH ESTIMATION BASED ON GLOBAL MOTION

• US 20120127270A1
• Filed: 11/21/2011
• Published: 05/24/2012
• Est. Priority Date: 11/23/2010
• Status: Active Grant

• Current Assignee: Qualcomm, Inc.
• Original Assignee: Qualcomm, Inc.
• Inventors: Zhang, Rong, Chen, Ying, Karczewicz, Marta
• Granted Patent: US 9,171,372 B2
• Time in Patent Office: Days
• US Class Current: 348/43
• CPC Class Codes: G06T 7/579 from motion

Abstract

This disclosure describes techniques for estimating a depth of image objects for a two-dimensional (2D) view of a video presentation. For example, a plurality of feature points may be determined for a 2D view. The plurality of feature points may be used to estimate global motion, e.g., motion of an observer (e.g., camera), of the 2D view. For example, the plurality of feature points may be used to generate a global motion frame difference. The global motion frame difference may be used to create a depth map for the 2D view, which may be used to generate an alternative view of the video presentation that may be used to display a three-dimensional (3D) video presentation.

Claims

1. A method of determining a depth map for a two-dimensional view of a video presentation, comprising:

determining, via an image processing module of a computing device for a two-dimensional (2D) view of a video presentation, a plurality of feature points for a frame of a video presentation;

determining global motion for the frame based on the plurality of feature points; and

generating a depth map for the frame based on the determined global motion.

2. The method of claim 1, further comprising:

determining a global motion compensated frame difference for the frame; and

generating the depth map for the frame using the global motion compensated frame difference.

3. The method of claim 2, wherein generating the depth map comprises classifying at least one pixel of the frame as a foreground pixel or a background pixel based on the global motion compensated frame difference.

4. The method of claim 3, wherein classifying at least one pixel of the frame as a foreground pixel or a background pixel comprises comparing an intensity of the global motion compensated frame difference for the at least one pixel to a predetermined threshold.

5. The method of claim 1, wherein determining the plurality of feature points comprises determining whether one or more blocks or pixels of the frame are in a center region of the frame.

6. The method of claim 5, further comprising:

if the one or more blocks or pixels of the frame are in the center region of the frame, using pixel information from at least one previous frame to generate the depth map for the one or more blocks or pixels in the center region of the frame.

7. The method of claim 1, wherein determining the plurality of feature points comprises:

identifying pixels of one or more corner regions of the frame.

8. The method of claim 7, wherein the identifying the pixels of the one or more corner regions comprises using Harris Corner Detection to identify the pixels of the one or more corner regions.

9. The method of claim 1, wherein determining the plurality of feature points comprises:

determining whether a pixel includes local motion.

10. The method of claim 1, wherein determining the plurality of feature points comprises:

determining whether a pixel is located in a homogenous region of the frame.

20. The device of claim 1, wherein the image processing module is further configured to:

determine the plurality of feature points based on whether a pixel is located in a homogenous region of the frame.

11. A device configured to determine a depth map for a two-dimensional (2D) view of a video presentation, comprising:

an image processing module configured to;

determine a plurality of feature points for a frame of a video presentation;

determine global motion for the frame based on the plurality of feature points; and

generate a depth map for the frame based on the determined global motion.

12. The device of claim 11, wherein the image processing module is further configured to:

determine a global motion compensated frame difference for the frame; and

generate the depth map for the frame using the global motion compensated frame difference.

13. The device of claim 12, wherein the image processing module is further configured to:

generate the depth map based on classifying at least one pixel of the frame as a foreground or a background pixel based on the global motion compensated frame difference.

14. The device of claim 13, wherein the image processing module is further configured to:

classify the at least one pixel of the frame as foreground or background pixels based on comparing an intensity of the global motion compensated frame difference for the at least one pixel to a predetermined threshold.

15. The device of claim 11, wherein the image processing module is further configured to:

determine the plurality of feature points based on whether one or more blocks or pixels of the frame are in a center region of the frame.

16. The device of claim 15, wherein the image processing module is further configured to:

if the one or more blocks or pixels of the frame are in the center region of the frame, use pixel information from at least one previous frame to generate the depth map for the one or more blocks or pixels in the center region of the frame.

17. The device of claim 11, wherein the image processing module is further configured to:

determine the plurality of feature points based on identifying pixels of one or more corner regions of the frame.

18. The device of claim 17, wherein the image processing module is further configured to:

identify the pixels of one or more corner regions of the frame using Harris Corner Detection to identify the pixels of the one or more corner regions.

19. The device of claim 11, wherein the image processing module is further configured to:

determine the plurality of feature points based on whether a pixel includes local motion.

21. A device configured to determine a depth map for a two-dimensional (2D) view of a video presentation, comprising:

means for determining a plurality of feature points for a frame of a video presentation;

means for determining global motion for the frame based on the plurality of feature points; and

means for generating a depth map for the frame based on the determined global motion.

22. The device of claim 21, further comprising:

means for determining a global motion compensated frame difference for the frame; and

means for generating the depth map for the frame using the global motion compensated frame difference.

23. The device of claim 22, wherein the means for generating the depth map further comprise:

means for classifying at least one pixel of the frame as a foreground pixel or a background pixel based on the global motion compensated frame difference.

24. The device of claim 23, wherein the means for classifying the at least one pixel of the frame as a foreground pixel or a background pixel comprise:

means for comparing an intensity of the global motion compensated frame difference for the at least one pixel to a predetermined threshold.

25. The device of claim 21, wherein the means for determining the plurality of feature points comprise:

means for determining whether one or more blocks or pixels of the frame are in a center region of the frame.

26. The device of claim 25, wherein the means for determining the plurality of feature points further comprise:

means for, if the one or more blocks or pixels of the frame are in the center region of the frame, using pixel information from at least one previous frame to generate the depth map for the one or more blocks or pixels in the center region of the frame.

27. The device of claim 21, wherein the means for determining the plurality of feature points comprise:

means for identifying pixels of one or more corner regions of the frame.

28. The device of claim 27, wherein the means for determining the plurality of feature points comprise:

means for using Harris Corner Detection to identify the pixels of the corner regions.

29. The device of claim 21, wherein the means for determining the plurality of feature points comprise:

means for determining whether a pixel includes local motion.

30. The device of claim 21, wherein the means for determining the plurality of feature points comprise:

means for determining whether a pixel is located in a homogenous region of the frame.

31. A computer-readable storage medium comprising instructions that upon execution cause one or more processors to:

determine a plurality of feature points for a frame of a video presentation;

determine global motion for the frame based on the plurality of feature points; and

generate a depth map for the frame based on the determined global motion.

32. The computer-readable storage medium of claim 31, wherein the instructions further cause the processor to:

determine a global motion compensated frame difference for the frame; and

generate the depth map for the frame using the global motion compensated frame difference.

33. The computer-readable storage medium of claim 32, wherein the instructions further cause the processor to:

generate the depth map based on classifying at least one pixel of the frame as a foreground pixel or a background pixel based on the global motion compensated frame difference.

34. The computer-readable storage medium of claim 33, wherein the instructions further cause the processor to:

classify at least one pixel of the frame as a foreground pixel or a background pixel based on comparing an intensity of the global motion compensated frame difference for the at least one pixel to a predetermined threshold.

35. The computer-readable storage medium of claim 31, wherein the instructions further cause the processor to:

determine the plurality of feature points based on whether one or more blocks or pixels of the frame are in a center region of the frame.

36. The computer-readable storage medium of claim 35, wherein the instructions further cause the processor to:

if the one or more blocks or pixels of the frame are in the center region of the frame, use pixel information from at least one previous frame to generate the depth map for the one or more blocks or pixels in the center region of the frame.

37. The computer-readable storage medium of claim 31, wherein the instructions further cause the processor to:

determine the plurality of feature points based on identifying pixels of one or more corner regions of the frame.

38. The computer-readable storage medium of claim 37, wherein the instructions further cause the processor to:

identify the pixels of the one or more corner regions comprises using Harris Corner Detection to identify the pixels of the one or more corner regions.

39. The computer-readable storage medium of claim 31, wherein the instructions further cause the processor to:

determine the plurality of feature points based on whether a pixel includes local motion.

40. The computer-readable storage medium of claim 31, wherein the instructions further cause the processor to:

determine the plurality of feature points based on whether a pixel is located in a homogenous region of the frame.