Systems and methods for simulating three-dimensional virtual interactions from two-dimensional camera images

• US 8,624,962 B2
• Filed: 02/02/2009
• Issued: 01/07/2014
• Est. Priority Date: 02/02/2009
• Status: Active Grant

• Current Assignee: YDreams - Informatica S.A.
• Original Assignee: Ydreams-Informatica SA Ydreams
• Inventors: de Almeida, Andre Rui Soares Pereira, Franco, Ivan de Almeida Soares, da Silva Frazao, Joao Pedro Gomes, de Almada, Antao Bastos Carrico Vaz, Cardoso, Nuno Ricardo Sequeira, Lopes, Goncalo Cardoso
• Primary Examiner(s): DENNISON, JERRY B
• Application Number:
  US12/364,122
• Publication Number:
  US 20100194863A1
• Time in Patent Office: 1,800 Days
• Field of Search: 348/50
• US Class Current: 348/50
• CPC Class Codes: G06T 19/00 Manipulating 3D models or images for computer graphics
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/30196 Human being; Person
• CPC Class Codes: G06T 2207/30204 Marker
• CPC Class Codes: G06T 7/12 Edge-based segmentation
• CPC Class Codes: G06T 7/536 from perspective effects, e.g. by using vanishing points

Abstract

A computer implemented method for incorporating a representation of a participant into a virtual 3D environment substantially in real-time is provided. An image including a participant is captured by a camera. A contour of the participant is automatically determined. Depth data is automatically associated with the participant contour. A first virtual 3D representation of the participant is automatically generated by extruding the participant contour based on the associated depth data. An interaction between the first virtual 3D representation of the participant and a second virtual 3D representation of a second object is displayed.

Claims

1. A computer implemented method for incorporating a representation of a participant into a virtual 3D environment, comprising:

receiving an image captured by a camera, the image including a participant;

automatically determining a contour of the participant;

automatically associating depth data with the contour of the participant;

automatically generating a first virtual 3D representation of the participant by extruding the contour based on the associated depth data;

determining a physics interaction between the first virtual 3D representation of the participant and a second virtual 3D representation of a second object based at least on the extruded contour, wherein the physics interaction comprises a force vector;

and automatically causing a display of the interaction between the first virtual 3D representation of the participant and the second virtual 3D representation of a second object based at least on the extruded contour, wherein the computer implemented method is performed substantially in real-time.

2. The computer implemented method of claim 1 wherein determining the physics interaction between the first virtual 3D representation of the participant and the second virtual 3D representation of the second object based at least on the extruded contour is performed at least in part by a real-time physics engine.

3. The computer implemented method of claim 1 wherein the physics interaction between the first virtual 3D representation of the participant and the second virtual 3D representation of the second object comprises a collision or contact between the first virtual 3D representation of the participant and the second virtual 3D representation of the second object.

4. A computing system comprising:

a processor;

a memory coupled to the processor; and

an interactive media subsystem enabled to;

receive an image captured by a camera, the image including a participant;

automatically determine a contour of the participant;

automatically associate depth data with the contour of the participant;

automatically generate a first virtual 3D representation of the participant by extruding the contour based on the associated depth data;

determine a physics interaction between the first virtual 3D representation of the participant and a second virtual 3D representation of a second object based at least on the extruded contour, wherein the physics interaction comprises a force vector; and

automatically cause a display of the interaction between the first virtual 3D representation of the participant and the second virtual 3D representation of a second object based at least on the extruded contour;

wherein the interactive media subsystem operates substantially in real-time.