Conversion of monoscopic visual content to stereoscopic 3D

• US 9,769,460 B1
• Filed: 07/21/2015
• Issued: 09/19/2017
• Est. Priority Date: 02/10/2012
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Mukherjee, Debargha, Wang, Meng, Wu, Chen
• Primary Examiner(s): CHEN, FRANK S
• Application Number:
  US14/805,326
• Time in Patent Office: 791 Days
• Field of Search: None
• CPC Class Codes: G06T 15/00 3D [Three Dimensional] image rendering
• CPC Class Codes: G06T 2207/10028 Range image; Depth image; 3D point clouds
• CPC Class Codes: G06T 7/40 Analysis of texture depth or shape recovery from texture G06T7/529
• CPC Class Codes: H04N 13/261 with monoscopic-to-stereoscopic image conversion
• CPC Class Codes: H04N 2013/0081 Depth or disparity estimation from stereoscopic image signals

Abstract

A request to convert a two-dimensional image to a three-dimensional image may be received. A feature-to-depth mapping function associated with another three-dimensional image that shares a characteristic with the two-dimensional image that is to be converted to the three-dimensional image may be identified. A depth value for a plurality of pixels of the two-dimensional image may be determined based on the feature-to-depth mapping function associated with the other three-dimensional image. The three-dimensional image may be generated based on the depth value for the plurality of pixels of the two-dimensional image.

Claims

1. A method comprising:

receiving a request to convert a two-dimensional image to a three-dimensional image;

identifying a first feature-to-depth mapping function associated with another three-dimensional image that shares a characteristic with a first region of the two-dimensional image that is to be converted to the three-dimensional image;

identifying a second feature-to-depth mapping function associated with another three-dimensional image that shares the characteristic with a second region of the two-dimensional image;

determining, by a processing device, a depth value for a plurality of pixels of the two-dimensional image based on a combination of the first and second feature-to-depth mapping functions, the combination assigning a respective depth value to a particular pixel of the plurality of pixels based on a distance of the particular pixel from another pixel in the first region of the two-dimensional image and another distance of the particular pixel from another pixel in the second region of the two-dimensional image; and

generating the three-dimensional image based on the depth value for the plurality of pixels of the two-dimensional image.

2. The method of claim 1, wherein the characteristic that is shared corresponds to a visual similarity between the other three-dimensional image and the two-dimensional image.

3. The method of claim 1, wherein the determining of the depth value for the plurality of pixels of the two-dimensional image based on the combination of the first and second feature-to-depth mapping functions is further based on a linear combination in view of a distance of the particular pixel from a center of the first or second region.

4. The method of claim 1, wherein the first and second feature-to-depth mapping functions correspond to a particular depth value based on locations of pixels from the respective other three-dimensional image.

5. The method of claim 1, wherein the first and second feature-to-depth mapping functions correspond to a particular depth value based on shapes of the respective other three-dimensional image.

6. The method of claim 1, wherein the first and second feature-to-depth mapping functions correspond to a particular depth value based on a color value of a pixel from the respective other three-dimensional image.

7. The method of claim 1, wherein the another pixel in the first region is at a center of the first region and the another pixel in the second region is at a center of the second region.

8. The method of claim 1, wherein the combination of the first feature-to-depth mapping function and the second feature-to-depth mapping function is further based on an average of a first depth value associated with the first feature-to-depth mapping function and a second depth value associated with the second feature-to-depth mapping function.

9. A non-transitory machine-readable storage medium storing instructions which, when executed by a processing device, cause the processing device to perform operations comprising:

receiving a request to convert a two-dimensional image to a three-dimensional image;

identifying a first feature-to-depth mapping function associated with another three-dimensional image that shares a characteristic with a first region of the two-dimensional image that is to be converted to the three-dimensional image;

identifying a second feature-to-depth mapping function associated with another three-dimensional image that shares the characteristic with a second region of the two-dimensional image;

determining, by a processing device, a depth value for a plurality of pixels of the two-dimensional image based on a combination of the first and second feature-to-depth mapping functions, the combination assigning a respective depth value to a particular pixel of the plurality of pixels based on a distance of the particular pixel from another pixel in the first region of the two-dimensional image and another distance of the particular pixel from another pixel in the second region of the two-dimensional image; and

generating the three-dimensional image based on the depth value for the plurality of pixels of the two-dimensional image.

10. The non-transitory machine-readable storage medium of claim 9, wherein the characteristic that is shared corresponds to a visual similarity between the other three-dimensional image and the two-dimensional image.

11. The non-transitory machine-readable storage medium of claim 9, wherein the determining of the depth value for the plurality of pixels of the two-dimensional image based on the combination of the first and second feature-to-depth mapping functions is further based on a linear combination in view of a distance of the particular pixel from a center of the first or second region.

12. The non-transitory machine-readable storage medium of claim 9, wherein the first and second feature-to-depth mapping functions correspond to a particular depth value based on locations of pixels from the respective other three-dimensional image.

13. The non-transitory machine-readable storage medium of claim 9, wherein the first and second feature-to-depth mapping functions correspond to a particular depth value based on shapes of the respective other three-dimensional image.

14. The non-transitory machine-readable storage medium of claim 9, wherein the first and second feature-to-depth mapping functions correspond to a particular depth value based on a color value of a pixel from the respective other three-dimensional image.

15. A system comprising:

a memory; and

a processing device, operatively coupled with the memory, to;

receive a request to convert a two-dimensional image to a three-dimensional image;

identify a first feature-to-depth mapping function associated with another three-dimensional image that shares a characteristic with a first region of the two-dimensional image that is to be converted to the three-dimensional image;

identify a second feature-to-depth mapping function associated with another three-dimensional image that shares the characteristic with a second region of the two-dimensional image;

determine a depth value for a plurality of pixels of the two-dimensional image based on a combination of the first and second feature-to-depth mapping functions, the combination assigning a respective depth value to a particular pixel of the plurality of pixels based on a distance of the particular pixel from another pixel in the first region of the two-dimensional image and another distance of the particular pixel from another pixel in the second region of the two-dimensional image; and

generate the three-dimensional image based on the depth value for the plurality of pixels of the two-dimensional image.

16. The system of claim 15, wherein the characteristic that is shared corresponds to a visual similarity between the other three-dimensional image and the two-dimensional image.

17. The system of claim 15, wherein the determining of the depth value for the plurality of pixels of the two-dimensional image based on the combination of the first and second feature-to-depth mapping functions is further based on a linear combination in view of a distance of the particular pixel from a center of the first or second region.

18. The system of claim 15, wherein the first and second feature-to-depth mapping functions correspond to a particular depth value based on locations of pixels or a shape from the respective other three-dimensional image.

19. The system of claim 15, wherein the first and second feature-to-depth mapping functions correspond to a particular depth value based on a color value of a pixel from of the respective other three-dimensional image.