Method and apparatus for real-time conversion of 2-dimensional content to 3-dimensional content

• US 9,483,836 B2
• Filed: 06/06/2013
• Issued: 11/01/2016
• Est. Priority Date: 02/28/2011
• Status: Active Grant

• Current Assignee: Sony Corporation (Sony Group Corp.)
• Original Assignee: Sony Corporation (Sony Group Corp.)
• Inventors: Usikov, Daniel
• Primary Examiner(s): Johnson, M Good
• Application Number:
  US13/911,389
• Publication Number:
  US 20140363100A1
• Time in Patent Office: 1,244 Days
• US Class Current: 1/1
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 7/20 Analysis of motion motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51
• CPC Class Codes: G06T 7/579 from motion
• CPC Class Codes: H04N 13/261 with monoscopic-to-stereoscopic image conversion

Abstract

Various aspects of a method and apparatus for video processing may include a computing device communicably coupled to an external device. The computing device may be operable to determine an average vertical velocity and an average horizontal velocity of a subset of pixels in an image frame and determine a depth value for each pixel of the subset of pixels based on calculated motion vectors of the pixel of the subset of pixels, the average vertical velocity of the subset of pixels and the average horizontal velocity of the subset of pixels.

Claims

1. A method for video processing, said method comprising:

in a computing device;

determining a depth value for each pixel of a subset of pixels in an image frame based on calculated motion vectors of said each pixel of said subset of pixels, an average vertical pixel velocity of said image frame and an average horizontal pixel velocity of said image frame,

wherein said average vertical pixel velocity and said average horizontal pixel velocity are determined based on pixels with predetermined motion vectors in said image frame,

wherein said determination of said depth value of a pixel of said subset of pixels in said image frame is based on a first difference and a second difference,

wherein said first difference is a difference between a vertical velocity of said pixel in said image frame and said average vertical pixel velocity of said image frame, and said second difference is a difference between a horizontal velocity of said pixel in said image frame and said average horizontal pixel velocity of said image frame; and

generating a stereoscopic image pair for said image frame based on said determined depth value for each pixel in said image frame.

2. The method of claim 1, wherein said determination of said depth value of said pixel of said subset of pixels in said image frame is based on summation of an absolute value of the first difference and an absolute value of the second difference.

3. The method of claim 1,

wherein said average vertical pixel velocity and said average horizontal pixel velocity are determined based on background pixels of said image frame,

wherein said background pixels are pixels that corresponds to a background in said image frame.

4. A method for video processing, said method comprising:

in a computing device;

determining an average vertical velocity and/or an average horizontal velocity of a subset of pixels in an image frame;

determining a depth value for each pixel of said subset of pixels based on calculated motion vectors of said each pixel of said subset of pixels, said average vertical velocity and said average horizontal velocity,

wherein said subset of pixels comprises pixels with predetermined motion vectors,

wherein said determination of said depth value of a pixel of said subset of pixels in said image frame is based on a first difference and a second difference,

wherein said first difference is a difference between a vertical velocity of said pixel in said image frame and said average vertical velocity of said subset of pixels, and said second difference is a difference between a horizontal velocity of said pixel in said image frame and said average horizontal velocity of said subset of pixels; and

generating a stereoscopic image pair for said image frame based on said determined depth value for each pixel in said image frame.

5. The method of claim 4, wherein said generated stereoscopic image pair comprises:

a left image in which said each pixel in said image frame is shifted left by a particular value; and

a right image in which said each pixel in said image frame is shifted right by said particular value,

wherein said particular value is proportional to said determined depth value associated with said each pixel in said image frame.

6. The method of claim 4, wherein said subset of pixels comprises one or more rows of pixels in said image frame.

7. The method of claim 4, wherein said calculated motion vectors of said each pixel of said subset of pixels comprises velocities of said each pixel in X and/or Y directions.

8. The method of claim 4, wherein said image frame is a two-dimensional (2D) image frame.

9. The method of claim 4, wherein said image frame is a two-dimensional (2D) image frame with a downsized pixel resolution.

10. The method of claim 4, comprising receiving said image frame from one or both of:

an external device and/or a local memory.

11. The method of claim 4, wherein said computing device comprises one or more of:

a television, a smart phone, a tablet, a laptop, and/or a desktop computer.

12. The method of claim 4, comprising receiving said image frame from one or more of:

a video player, a set-top-box, a personal video recorder, a video camera, and/or a cloud server.

13. The method of claim 4, wherein said calculated motion vectors provide information regarding said horizontal velocity of said pixel and said vertical velocity of said pixel.

14. A computing device for video processing, said computing device comprising:

one or more processors and/or circuits that are operable to;

determine an average vertical velocity and/or an average horizontal velocity of a subset of pixels in an image frame;

determine a depth value for each pixel of said subset of pixels based on calculated motion vectors of said each pixel of said subset of pixels, said average vertical velocity and said average horizontal velocity,

wherein said subset of pixels comprises pixels with predetermined motion vectors,

wherein said determination of said depth value of a pixel of said subset of pixels in said image frame is based on a first difference and a second difference,

wherein said first difference is a difference between a vertical velocity of said pixel in said image frame and said average vertical velocity of said subset of pixels, and said second difference is a difference between a horizontal velocity of said pixel in said image frame and said average horizontal velocity of said subset of pixels; and

generate a stereoscopic image pair for said image frame based on said determined depth value for each pixel in said image frame.

15. The computing device of claim 14,

wherein said one or more processors and/or circuits are operable to generate said stereoscopic image pair based on generating a left image and a right image,

wherein said left image is generated by shifting left by a particular value, said each pixel in said image frame,

wherein said right image is generated by shifting right by said particular value, said each pixel in said image frame, and

wherein said particular value is proportional to said determined depth value of said each pixel of said image frame.

16. The computing device of claim 14, wherein said one or more processors and/or circuits are operable to calculate a depth map.

17. The computing device of claim 14, wherein said subset of pixels comprises one or more rows of pixels in said image frame.

18. The computing device of claim 14, wherein said calculated motion vectors of said each pixel of said subset of pixels comprises velocities of said each pixel in X and/or Y directions.

19. The computing device of claim 14, wherein said one or more processors and/or circuits are operable to receive said image frame from one or both of:

an external device and/or a local memory.

20. The computing device of claim 19, wherein said external device comprises one or more of:

a video player, a set-top-box, a personal video recorder, a video camera, and/or a cloud server.

21. The computing device of claim 14, wherein said computing device comprises one or more of:

a television, a smart phone, a tablet, a laptop, and/or a desktop computer.