System and process for transforming two-dimensional images into three-dimensional images

• US 8,922,628 B2
• Filed: 09/01/2010
• Issued: 12/30/2014
• Est. Priority Date: 09/01/2009
• Status: Expired due to Fees

• Current Assignee: Prime Focus Creative Services Canada Incorporated
• Original Assignee: Prime Focus Vfx Services II Inc.
• Inventors: Bond, Chris
• Primary Examiner(s): Vaughn, Jr., William C.
• Assistant Examiner(s): PEREZ FUENTES, LUIS M
• Application Number:
  US12/874,190
• Publication Number:
  US 20110050864A1
• Time in Patent Office: 1,581 Days
• Field of Search: 348/51
• US Class Current: 348/51
• CPC Class Codes: G06T 7/579 from motion
• CPC Class Codes: G06T 7/593 from stereo images
• CPC Class Codes: H04N 13/261 with monoscopic-to-stereoscopic image conversion
• CPC Class Codes: H04N 13/305 using lenticular lenses, e.g. arrangements of cylindrical lenses
• CPC Class Codes: H04N 13/395 with depth sampling, i.e. the volume being constructed from a stack or sequence of 2D image planes

Abstract

A system and process is provided for the conversion of a stream of two-dimensional images into a pair of streams of images for providing the perception of a stream of three-dimensional images. Each complimentary image in the image stream undergoes the application of a selection and remapping process to independently alter portions of the image, so that the remappings shift the image elements in a manner which produces a stereo depth effect when the images are viewed through the appropriate viewing device.

Claims

1. A process for creating a three dimensional media projection from a two-dimensional image, comprising:

copying a two-dimensional image to create a working image of the two dimensional image;

creating a depth selection mask associated with at least one characteristic of the working image and associated with at least a portion of the working image, including;

selecting a set of image features, and

determining a real value in a normalized range for the set of selected image features for each pixel location in the portion of the working image;

determining a vector field of each pixel of the portion of the working image;

applying the depth selection mask to the vector field of the portion of the working image to create a weighted displacement mask having a displacement vector associated with each pixel of the portion of the working image, the displacement vector having a magnitude and direction;

displacing each pixel of the portion of the working image in accordance with the magnitude of the vector associated with that pixel to create a modified working image; and

creating a stereo pair of images by pairing the two dimensional image with the modified working image.

2. The process of claim 1, further comprising:

displaying on a video display the modified working image as part of the pair of stereo images.

3. The process of claim 1, further comprising:

wherein the the displacement has a magnitude that is relative to a perceived depth of the portion and includes a maximum magnitude of displacement being assigned to each pixel location having a first selected depth value and a minimum magnitude of displacement being assigned to each pixel location having a second selected depth value and an intermediate magnitude of displacement being assigned to a pixel location having a third selected depth value, the third selected depth value selected from a range between the first and second depth values.

4. The process of claim 3, further comprising:

creating a boundary on at least one part of the image mask so as to preclude assigning the depth values to non-selected pixel locations outside the boundary.

5. The process of claim 1, wherein creating the depth selection mask associated with at least one characteristic characteristic of the working image includes:

selecting an image characteristic from a group consisting of hue, luminance, saturation, and color; and

generating an image representation of a weighted distribution of the image characteristic in the working image,

wherein the real values are assigned to each pixel location in accordance with the weighted distribution.

6. The process of claim 1, further comprising:

comparing a portion of an image feature present in the working image with a portion of the depth selection mask and a selected range of depth values; and

changing the depth value at a selected pixel of the depth selection mask based on a location of the selected pixel relative to the image feature.

7. The process of claim 1, further comprising:

applying the depth selection mask to a second image to assign each depth value to a corresponding pixel location in the second image;

displacing by distortion of the second image a portion of the second image in a direction opposite the displacement of the working image; and

replacing a selected image in a stereo pair of images with the second image.

8. The process of claim 1, further comprising:

providing a second image associated with the working image;

creating a depth selection mask associated with at least one characteristic of the second image and associated with at least a portion of the second image, including;

selecting a set of image features, and

determining a real value in a normalized range for the set of selected image features for each pixel location in the portion of the second image;

determining a vector field of each pixel of the portion of the second image;

applying the depth selection mask to the vector field of the portion of the second image to create a weighted displacement mask having a displacement vector associated with each pixel of the portion of the second image, the displacement vector having a magnitude and direction;

displacing each pixel of the portion of the second image in accordance with the magnitude of the vector associated with that pixel to create a modified second image; and

replacing the second image in a stereo pair of images with the modified second image.

9. The process of claim 8, wherein the three-dimensional media projection is created when the modified working image and the modified second image are viewed as a pair of stereo images through a stereoscopic viewing device.

10. The process of claim 8, wherein the displacement of the second modified image is in a direction opposite the displacement of the modified working image.

11. The process of claim 10, wherein the magnitude of the displacement of the modified second image is substantially equal to the magnitude of the displacement of the modified working image.

12. A system for creating a three-dimensional media projection from a two-dimensional digitized image, comprising:

a video monitor configured to display a stereo pair of images in response to a video signal;

an input device configured to generate input signals, and configured to select a working image from a group of images, the working image being a copy of an image selected from the group of images, and configured to select characteristics of the working image;

a control configured to generate control signals, and configured to assign a value to a characteristic selected by the input device; and

a processor in operable communication with the monitor, the input device and the control, the processor configured to displace a portion of the working image by

creating a depth selection mask associated with at least one characteristic of the working image and associated with at least a portion of the working image, including

selecting a set of image features and determining a real value in a normalized range for the set of selected image features for each pixel location in the portion of the working image, determining a vector field of each pixel of the portion of the working image,

applying the depth selection mask to the vector field of the portion of the working image to create a weighted displacement mask having a displacement vector associated with each pixel of the portion of the working image, the displacement vector having a magnitude and direction,

displacing each pixel of the portion of the working image in accordance with the magnitude of the vector associated with that pixel to create a modified working image; and

wherein the video monitor, in response to a second input signal, displays the modified working image as part of the stereo pair of images.

13. A system of claim 12, further comprising:

the processor further configured to create the depth selection mask associated with characteristics of the working image and associated with at least a portion of the working image in response to a second input signal, and to

assign a depth value to each pixel location of the working image that is associated with the depth selection mask, the depth value being chosen from a specified range of values,

wherein the magnitude of the displacement being relative to a value received by the control includes a control value received by the control multiplied by a scaling factor determinable at each pixel location according to the depth value assigned to each pixel location.

14. A system of claim 13, further comprising:

the processor further configured to, in response to a second input signal, apply the depth selection mask to a second image to assign each depth value to a corresponding pixel location in the second image;

the processor further configured to displace in a direction opposite the displacement of the working image a portion of the second image by distortion of the second image; and

the processor further configured to replace a selected image in a stereo pair of images with the second image.

15. The system of claim 12, further comprising:

the processor being further configured, on receiving a command from the input device, to distort at least part of a selected image from the stereo pair of images by applying a distortion to the selected image.

16. The system of claim 12, further comprising:

the processor being further configured, on receiving a command from the input device, to remove a distortion from at least part of a selected image from the stereo pair of images by applying a distortion to the selected image.

17. The system of claim 12, wherein the processor is configured, on receiving first commands from the input device, to select a first and second portion of the working image as candidates for displacement, and to assign the first and second portions to first and second controls, wherein the first control is operational to displace the first portion and the second control is operational to displace the second portion.

18. The system of claim 12, further comprising:

a stereoscopic viewing device; and

an image storage for storing an image stream of multiple working images,

wherein the processor is configured to, in response to a second input signal, receive the image stream from the storage and display the image stream on the video monitor to generate a three-dimensional motion picture when viewed through the stereoscopic viewing device.