Methods, systems, devices and associated processing logic for generating stereoscopic images and video

• US 8,928,654 B2
• Filed: 01/23/2012
• Issued: 01/06/2015
• Est. Priority Date: 07/30/2004
• Status: Active Grant

• Current Assignee: Jungo Connectivity Ltd. (Cisco Systems, Inc.)
• Original Assignee: Extreme Reality Ltd.
• Inventors: Givon, Dor
• Primary Examiner(s): Nguyen, Kimbinh T
• Application Number:
  US13/355,643
• Publication Number:
  US 20120176477A1
• Time in Patent Office: 1,079 Days
• Field of Search: 345/419, 345/420, 348/46, 348/E13.074
• US Class Current: 345/419
• CPC Class Codes: G06T 17/00 Three dimensional [3D] modelling, e.g. data description of 3D objects
• CPC Class Codes: H04N 13/111 Transformation of image signals corresponding to virtual viewpoints, e.g. spatial image interpolation
• CPC Class Codes: H04N 13/261 with monoscopic-to-stereoscopic image conversion

Abstract

The present invention includes methods, systems, devices and associated processing logic for generating stereoscopic 3-Dimensional images and/or video from 2-Dimensional images or video. There may be provided a stereoscopic 3D generating system to extrapolate and render 2D complementary images and or video from a first 2D image and/or video. The complementary images and/or video, when combined with the first image or video, or a second complementary image or video, form a stereoscopic image of the scene captured in the first image or video. The stereoscopic 3D generation system may generate a complementary image or images, such that when a viewer views the first image or a second complementary image (shifted in the other direction from the first complementary image) with one eye and the complementary image with the other eye, an illusion of depth in the image is created (e.g. a stereoscopic 3D image).

Claims

1. A method for generating a stereoscopic image comprising:

receiving, by receiving circuitry, a first 2-dimensional (2D) image data set representing a 2D image of a scene;

automatically identifying, by use of processing circuitry, a first non-rigid object within the first 2D image data set;

automatically identifying, by use of the processing circuitry, one or more elements of the first non-rigid object, within the first 2D image data set;

automatically correlating and matching, by use of the processing circuitry, a 3-Dimensional (3D) skeletal model with the first non-rigid object, by automatically correlating, by use of the processing circuitry, elements of the 3D skeletal model with the identified elements of the first non-rigid object;

automatically determining, by use of processing circuitry, depth data relating to the first non-rigid object based at least partially on the correlated skeletal model; and

automatically calculating, based on the determined depth data relating to the first non-rigid object, shifts of position, within the scene, of the identified elements of the first non-rigid object, which shifts correspond to a second viewing angle of the scene, which second viewing angle approximates a viewing angle of a second human eye viewing the scene in 3-Dimensions; and

automatically generating, by use of processing circuitry, a stereoscopic image of the scene by generating a complementary 2D image data set depicting the scene from the second viewing angle, wherein said generating a complementary 2D image data set comprises replicating the 2D image data set while shifting positions of objects in the scene according to their depth to account for the difference in viewing angle, including shifting positions of the identified elements of the first non-rigid object, within the scene, according to the calculated shifts.

2. The method according to claim 1, further comprising performing the steps of the method according to claim 1 upon a series of images, which series of images comprise a video sequence.

3. The method according to claim 1, further comprising creating a stereoscopic image of the scene by combining the complementary 2D image data set and the first 2D image set.

4. The method according to claim 1, further comprising generating a stereoscopic image of the scene by combining the complementary 2D image data set and a second complementary 2D image set generated, according to the steps of the method of claim 1, from the first 2D image data set.

5. The method according to claim 1, further comprising modifying, based at least partially on the 3D skeletal model, the appearance of the first non-rigid object, such as to represent the appearance of the first non-rigid object from the second viewing angle.

6. The method according to claim 1, further comprising identifying a rigid object within the first 2D image set and estimating a depth of the rigid object based on a spatial relationship between the rigid object and the first non-rigid object.

7. The method according to claim 1, further comprising modifying the appearance of a rigid object within the first image data set to correspond to a second viewing angle by copying the texture of one or more pixels along the edge of the rigid object, to neighboring pixels.

8. The method according to claim 1, wherein said method is performed in real time.

9. A system for generating a stereoscopic image comprising:

a receiving module comprising processing circuitry for receiving a first 2-dimensional (2D) image data set representing a 2D image of a scene;

a discrete object identifying module comprising processing circuitry functionally associated with said receiving module for automatically identifying a first non-rigid object within the first 2D image data set;

a feature identifying module comprising processing circuitry for automatically identifying one or more elements of the first non-rigid object, within the first 2D image data set;

a reference model correlator comprising processing circuitry functionally associated with said identifying module and said feature identification module for automatically correlating and matching a 3-Dimensional (3D) skeletal model with the first non-rigid object, by automatically correlating elements of the 3D skeletal model with the identified features of the first non-rigid object;

an object depth estimating module comprising processing circuitry for automatically determining depth data relating to the first non-rigid object based at least partially on the correlated skeletal model; and

a horizontal shift calculator comprising processing circuitry for automatically calculating, based on the determined depth data relating to the first non-rigid object, shifts of position, within the scene, of the identified elements of the first non-rigid object, which shifts correspond to a second viewing angle of the scene, which second viewing angle approximates a viewing angle of a second human eye viewing the scene in 3-Dimensions; and

a rendering module comprising processing circuitry for automatically generating a stereoscopic image of the scene by generating a complementary 2D image data set depicting the scene from the second viewing angle, wherein said generating a complementary 2D image data set comprises replicating the 2D image data set while shifting positions of objects in the scene according to their depth to account for the difference in viewing angle, including shifting positions of the identified elements of the first non-rigid object, within the scene, according to the calculated shifts.

10. The system according to claim 9, further comprising a modified angle of view generator comprising processing circuitry for modifying, based at least partially on the 3D skeletal model, the appearance of the first non-rigid object, such as to represent the appearance of the first non-rigid object from the second viewing angle.

11. The system according to claim 9, further comprising a modified angle of view generator comprising processing circuitry for modifying the appearance of a rigid object within the first image data set to correspond to the second viewing angle by copying the texture of one or more pixels along the edge of the rigid object, to neighboring pixels.