External depth map transformation method for conversion of two-dimensional images to stereoscopic images

• US 9,241,147 B2
• Filed: 05/01/2013
• Issued: 01/19/2016
• Est. Priority Date: 05/01/2013
• Status: Expired due to Fees

• Current Assignee: Legend3D, Inc.
• Original Assignee: Legend3D, Inc.
• Inventors: Schad, Thomas, Jones, Scott, DeJohn, Matthew
• Primary Examiner(s): Patel, Jayesh A
• Application Number:
  US13/874,625
• Publication Number:
  US 20140327736A1
• Time in Patent Office: 993 Days
• Field of Search: None
• US Class Current: 1/1
• CPC Class Codes: H04N 13/122 Improving the 3D impression of stereoscopic images by modifying image signal contents, e.g. by filtering or adding monoscopic depth cues H04N13/128 takes precedence
• CPC Class Codes: H04N 13/128 Adjusting depth or disparity
• CPC Class Codes: H04N 13/261 with monoscopic-to-stereoscopic image conversion

Abstract

External depth map transformation method for conversion of two-dimensional images to stereoscopic images that provides increased artistic and technical flexibility and rapid conversion of movies for stereoscopic viewing. Embodiments of the invention convert a large set of highly granular depth information inherent in a depth map associated with a two-dimensional image to a smaller set of rotated planes associated with masked areas in the image. This enables the planes to be manipulated independently or as part of a group and eliminates many problems associated with importing external depth maps including minimization of errors that frequently exist in external depth maps.

Claims

1. An external depth map transformation method for conversion of two-dimensional images to stereoscopic images comprising:

obtaining an external depth map associated with a two-dimensional image;

obtaining at least one mask associated with at least one area within the two-dimensional image;

calculating a best fit for a plane using a computer based on depth associated with the at least one area associated with each of the at least one mask;

applying depth associated with the plane having the best fit to the at least one area to shift pixels in the two-dimensional image horizontally to produce a stereoscopic image or stereoscopic image pair; and

,

altering automatically using said computer, any combination of position, orientation, shape, depth or curve of the plane in order to fit edges or corners of the plane with another plane.

2. The method of claim 1

wherein the obtaining of the external depth map associated with a two-dimensional image comprises obtaining a disparity map.

3. The method of claim 1

wherein the obtaining of the external depth map associated with a two-dimensional image comprises obtaining a depth map of lower resolution than the two-dimensional image from a pair of witness cameras.

4. The method of claim 1

wherein the obtaining of the external depth map associated with a two-dimensional image comprises obtaining a depth map from time-of-flight or light-field system.

5. The method of claim 1

wherein the obtaining of the external depth map associated with a two-dimensional image comprises obtaining a depth map from a triangulation system.

6. The method of claim 1

wherein the obtaining at least one mask associated with at least one area within the two-dimensional image comprises automatically generating the at least one mask comprising the at least one area wherein the at least one area is over a predefined size and within a predefined depth range.

7. The method of claim 1

wherein the obtaining at least one mask associated with at least one area within the two-dimensional image comprises automatically generating the at least one mask comprising the at least one area wherein the at least one area comprises a boundary having a difference in luminance values over a predefined threshold.

8. The method of claim 1 wherein the calculating the best fit for the plane using a computer based on depth associated with the at least one area associated with each of the at least one mask comprises calculating a normal vector for the plane.

9. The method of claim 1 wherein the calculating the best fit for the plane using a computer based on depth associated with the at least one area associated with each of the at least one mask comprises calculating a regression fit for the plane.

10. The method of claim 1 wherein the calculating the best fit for the plane using a computer based on depth associated with the at least one area associated with each of the at least one mask comprises calculating an orthogonal distance regression fit for the plane.

11. The method of claim 1 further comprising:

grouping two or more of said plane and at least one other plane.

12. The method of claim 1 further comprising:

grouping two or more of said plane and at least one other plane wherein each of said two or more of said plane and at least one other plane are grouped with a link of a predefined minimum and maximum distance.

13. The method of claim 1 further comprising:

grouping two or more of said plane and at least one other plane wherein each of said two or more of said plane and at least one other plane are grouped with a link having a spring constant.

14. The method of claim 1 further comprising accepting an input to alter any combination of position, orientation, shape, depth or curve of said plane.

15. An external depth map transformation method for conversion of two-dimensional images to stereoscopic images comprising:

obtaining an external depth map associated with a two-dimensional image;

obtaining at least one mask associated with at least one area within the two-dimensional image by automatically generating the at least one mask comprising the at least one area wherein the at least one area is over a predefined size and within a predefined depth range or wherein the at least one area comprises a boundary having a difference in luminance values over a predefined threshold or wherein the at least one area is over the predefined size and within the predefined depth range and comprises the boundary having the difference in luminance values over the predefined threshold;

calculating a best fit for a plane using a computer based on depth associated with the at least one area associated with each of the at least one mask;

altering automatically using said computer, any combination of position, orientation, shape, depth or curve of the plane in order to fit edges or corners of the plane with another plane; and

,

applying depth associated with the plane having the best fit to the at least one area to shift pixels in the two-dimensional image horizontally to produce a stereoscopic image or stereoscopic image pair.

16. The method of claim 15

wherein the obtaining of the external depth map associated with a two-dimensional image comprises obtaining a disparity map, or obtaining a depth map of lower resolution than the two-dimensional image from a pair of witness cameras, or obtaining a depth map from time-of-flight system, or obtaining a depth map from a triangulation system.

17. The method of claim 15 wherein the calculating the best fit for the plane using a computer based on depth associated with the at least one area associated with each of the at least one mask comprises calculating a normal vector for the plane, or a regression fit for the plane, or an orthogonal distance regression fit for the plane.