Depth estimation based on global motion

• US 9,171,372 B2
• Filed: 11/21/2011
• Issued: 10/27/2015
• Est. Priority Date: 11/23/2010
• Status: Expired due to Fees

• Current Assignee: Qualcomm, Inc.
• Original Assignee: Qualcomm, Inc.
• Inventors: Zhang, Rong, Chen, Ying, Karczewicz, Marta
• Primary Examiner(s): PRINCE, JESSICA MARIE
• Application Number:
  US13/301,225
• Publication Number:
  US 20120127270A1
• Time in Patent Office: 1,436 Days
• Field of Search: 348/43
• US Class Current: 1/1
• CPC Class Codes: G06T 7/579 from motion

Abstract

This disclosure describes techniques for estimating a depth of image objects for a two-dimensional (2D) view of a video presentation. For example, a plurality of feature points may be determined for a 2D view. The plurality of feature points may be used to estimate global motion, e.g., motion of an observer (e.g., camera), of the 2D view. For example, the plurality of feature points may be used to generate a global motion frame difference. The global motion frame difference may be used to create a depth map for the 2D view, which may be used to generate an alternative view of the video presentation that may be used to display a three-dimensional (3D) video presentation.

Claims

1. A method of determining a depth map for a two-dimensional view of a video presentation, comprising:

determining, via an image processing module of a computing device for a two-dimensional (2D) view of a video presentation, a plurality of feature points for a frame of a video presentation by determining whether one or more blocks or pixels of the frame are in a center region of the frame;

determining global motion for the frame based on the plurality of feature points;

determining a global motion compensated frame difference for the frame, the global motion compensated frame difference indicating a difference between the frame and a global motion compensated frame; and

generating a depth map for the frame based on the determined global motion and the determined global motion compensated frame difference for the frame, and if the one or more blocks or pixels of the frame are in the center region of the frame, using pixel information from at least one previous frame to generate the depth map for the one or more blocks or pixels in the center region of the frame.

2. The method of claim 1, wherein generating the depth map comprises classifying at least one pixel of the frame as a foreground pixel or a background pixel based on the global motion compensated frame difference.

3. The method of claim 2, wherein classifying at least one pixel of the frame as a foreground pixel or a background pixel comprises comparing an intensity of the global motion compensated frame difference for the at least one pixel to a predetermined threshold.

4. The method of claim 1, wherein determining the plurality of feature points comprises:

identifying pixels of one or more corner regions of the frame.

5. The method of claim 4, wherein the identifying the pixels of the one or more corner regions comprises using Harris corner Detection to identify the pixels of the one or more corner regions.

6. The method of claim 1, wherein determining the plurality of feature points comprises:

determining whether a pixel includes local motion.

7. The method of claim 1, wherein determining the plurality of feature points comprises:

determining whether a pixel is located in a homogenous region of the frame; and

selecting the pixel for determining the plurality of feature points based on the determination.

8. A device configured to determine a depth map for a two-dimensional (2D) view of a video presentation, comprising:

an image processing module configured to;

determine a plurality of feature points for a frame of a video presentation based on whether one or more blocks or pixels of the frame are in a center region of the frame;

determine global motion for the frame based on the plurality of feature points;

determine a global motion compensated frame difference for the frame, the global motion compensated frame difference indicating a difference between the frame and a global motion compensated frame; and

generate a depth map for the frame based on the determined global motion and the determined global motion compensated frame difference for the frame, and if the one or more blocks or pixels of the frame are in the center region of the frame, use pixel information from at least one previous frame to generate the depth map for the one or more blocks or pixels in the center region of the frame.

9. The device of claim 8, wherein the image processing module is further configured to:

generate the depth map based on classifying at least one pixel of the frame as a foreground or a background pixel based on the global motion compensated frame difference.

10. The device of claim 9, wherein the image processing module is further configured to:

classify the at least one pixel of the frame as foreground or background pixels based on comparing an intensity of the global motion compensated frame difference for the at least one pixel to a predetermined threshold.

11. The device of claim 8, wherein the image processing module is further configured to:

determine the plurality of feature points based on identifying pixels of one or more corner regions of the frame.

12. The device of claim 11, wherein the image processing module is further configured to:

identify the pixels of one or more corner regions of the frame using Harris corner Detection to identify the pixels of the one or more corner regions.

13. The device of claim 8, wherein the image processing module is further configured to:

determine the plurality of feature points based on whether a pixel includes local motion.

14. The device of claim 8, wherein the image processing module is further configured to:

determine the plurality of feature points based on whether a pixel is located in a homogenous region of the frame.

15. A device configured to determine a depth map for a two-dimensional (2D) view of a video presentation, comprising:

means for determining a plurality of feature points for a frame of a video presentation via a means for determining whether one or more blocks or pixels of the frame are in a center region of the frame;

means for determining global motion for the frame based on the plurality of feature points;

means for determining a global motion compensated frame difference for the frame, the global motion compensated frame difference indicating a difference between the frame and a global motion compensated frame; and

means for generating a depth map for the frame based on the determined global motion and the determined global motion compensated frame difference for the frame, and if the one or more blocks or pixels of the frame are in the center region of the frame, means for using pixel information from at least one previous frame to generate the depth map for the one or more blocks or pixels in the center region of the frame.

16. The device of claim 15, wherein the means for generating the depth map further comprise:

means for classifying at least one pixel of the frame as a foreground pixel or a background pixel based on the global motion compensated frame difference.

17. The device of claim 16, wherein the means for classifying the at least one pixel of the frame as a foreground pixel or a background pixel comprise:

means for comparing an intensity of the global motion compensated frame difference for the at least one pixel to a predetermined threshold.

18. The device of claim 15, wherein the means for determining the plurality of feature points comprise:

means for identifying pixels of one or more corner regions of the frame.

19. The device of claim 18, wherein the means for determining the plurality of feature points comprise:

means for using Harris Corner Detection to identify the pixels of the corner regions.

20. The device of claim 15, wherein the means for determining the plurality of feature points comprise:

means for determining whether a pixel includes local motion.

21. The device of claim 15, wherein the means for determining the plurality of feature points comprise:

means for determining whether a pixel is located in a homogenous region of the frame and for selecting the pixel for determining the plurality of feature points based on the determination.

22. A non-transitory computer-readable storage medium comprising instructions that upon execution cause one or more processors to:

determine a plurality of feature points for a frame of a video presentation based on whether one or more blocks or pixels of the frame are in a center region of the frame;

determine global motion for the frame based on the plurality of feature points;

determine a global motion compensated frame difference for the frame, the global motion compensated frame difference indicating a difference between the frame and a global motion compensated frame; and

generate a depth map for the frame based on the determined global motion, and the determined global motion compensated frame difference for the frame, and if the one or more blocks or pixels of the frame are in the center region of the frame, use pixel information from at least one previous frame to generate the depth map for the one or more blocks or pixels in the center region of the frame.

23. The non-transitory computer-readable storage medium of claim 22, wherein the instructions further cause the processor to:

generate the depth map based on classifying at least one pixel of the frame as a foreground pixel or a background pixel based on the global motion compensated frame difference.

24. The non-transitory computer-readable storage medium of claim 23, wherein the instructions further cause the processor to:

classify at least one pixel of the frame as a foreground pixel or a background pixel based on comparing an intensity of the global motion compensated frame difference for the at least one pixel to a predetermined threshold.

25. The non-transitory computer-readable storage medium of claim 22, wherein the instructions further cause the processor to:

determine the plurality of feature points based on identifying pixels of one or more corner regions of the frame.

26. The non-transitory computer-readable storage medium of claim 25, wherein the instructions further cause the processor to:

identify the pixels of the one or more corner regions comprises using Harris corner Detection to identify the pixels of the one or more corner regions.

27. The non-transitory computer-readable storage medium of claim 22, wherein the instructions further cause the processor to:

determine the plurality of feature points based on whether a pixel includes local motion.

28. The non-transitory computer-readable storage medium of claim 22, wherein the instructions further cause the processor to:

determine the plurality of feature points based on whether a pixel is located in a homogenous region of the frame.