DYNAMIC THREE-DIMENSIONAL OBJECT MAPPING FOR USER-DEFINED CONTROL DEVICE

• US 20090158220A1
• Filed: 12/15/2008
• Published: 06/18/2009
• Est. Priority Date: 12/17/2007
• Status: Active Grant

• Current Assignee: Sony Interactive Entertainment LLC (Sony Group Corp.)
• Original Assignee: Sony Computer Entertainment America (Sony Group Corp.), Sony Computer Entertainment Europe Limited (Sony Group Corp.)
• Inventors: Gillo, Tomas, Zalewski, Gary
• Granted Patent: US 8,542,907 B2
• Time in Patent Office: Days
• US Class Current: 715/863
• CPC Class Codes: A63F 13/213 comprising photodetecting means, e.g. cameras, photodiodes or infrared cells A63F13/219, A63F13/655 take precedence
• CPC Class Codes: A63F 13/42 by mapping the input signals into game commands, e.g. mapping the displacement of a stylus on a touch screen to the steering angle of a virtual vehicle
• CPC Class Codes: A63F 13/45 Controlling the progress of the video game
• CPC Class Codes: A63F 13/812 Ball games, e.g. soccer or baseball
• CPC Class Codes: A63F 2300/1093 using visible light
• CPC Class Codes: A63F 2300/6045 for mapping control signals received from the input arrangement into game commands
• CPC Class Codes: A63F 2300/8011 Ball
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0304 Detection arrangements using opto-electronic means constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042
• CPC Class Codes: G06F 3/0346 with detection of the device orientation or free movement in a 3D space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors
• CPC Class Codes: G06F 3/04815 Interaction with a metaphor-based environment or interaction object displayed as three-dimensional, e.g. changing the user viewpoint with respect to the environment or object
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: G06T 7/20 Analysis of motion motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51
• CPC Class Codes: G06T 7/60 Analysis of geometric attributes
• CPC Class Codes: G06T 7/70 Determining position or orientation of objects or cameras camera calibration G06T7/80

Abstract

A computer-implemented method is provided to interactively capture and utilize a three-dimensional object as a controlling device for a computer system. One operation of the method is capturing depth data of the three-dimensional object. In another operation, the depth data of the three-dimensional object undergoes processing to create geometric defining parameters for the three-dimensional object. The method can also include defining correlations between particular actions performed with the three-dimensional object and particular actions to be performed by the computer system. The method also includes an operation to save the geometric defining parameters of the three-dimensional object to a recognized object database. In another operation, the correlations between particular actions performed with the three-dimensional object and particular actions to be performed by the computer system in response to recognizing the particular actions are also saved to the recognized object database.

Claims

1. A computer implemented method to interactively capture and utilize a three-dimensional object to be a controlling device for a computer system, comprising:

(a) capturing depth data of the three-dimensional object;

(b) processing captured depth data of the three-dimensional object to create geometric defining parameters for the three-dimensional object;

(c) defining correlations between particular actions performed with the three-dimensional object and particular actions to be performed by the computer system; and

(d) saving to a recognized object database the geometric defining parameters of the three-dimensional object and correlations between particular actions performed with the three-dimensional object and particular actions to be performed by the computer system in response to recognizing the particular actions.

2. The computer implemented method as recited in claim 1, further comprising:

(e) activating an interactive session processed by the computer system, the interactive session including;

(i) capturing depth data and analyzing the depth data, the depth data being compared to the geometric defining parameters stored in the recognized object database; and

(ii) triggering activation of actions during the interactive session in response to actions performed with the three-dimensional object, as defined by the correlations between particular actions performed with the three-dimensional object and particular actions to be performed by the computer system

3. The computer implemented method as recited in claim 1, wherein depth data for the three-dimensional object is captured from a plurality of angles, the plurality of angles allowing the computer system to process the depth data and create the geometric parameters for the three-dimensional object.

4. The computer implemented method as recited in claim 1, wherein the correlations are defined by assigning a particular action performed with the three-dimensional object to a pre-defined particular action to be performed by the computer system.

5. The computer implemented method as recited in claim 1, wherein defining the correlations by having the user interactively move the three-dimensional object in response to a prompt for a pre-defined particular action.

6. The computer implemented method as recited in claim 1, creating macro correlations for a remote control so once a new three-dimensional object is defined, basic movements have the same effect.

7. The computer implemented method as recited in claim 1, saving multiple three-dimensional objects in the recognized object database and switching between them real-time, each object having customized controls.

8. The computer implemented method as in claim 2, wherein the actions performed with the three-dimensional object include moving the object, turning the object, waving the object, tapping the object, shaking the object, pointing with the objects or combinations thereof.

9. A computer implemented method to interactively capture and utilize a three-dimensional object to be a controlling device for a computer system, comprising:

(a) identifying the three-dimensional object, the identifying including,

(i) capturing depth data of the three-dimensional object;

(ii) processing captured depth data of the three-dimensional object to create geometric defining parameters for the three-dimensional object;

(iii) defining correlations between particular actions performed with the three-dimensional object and particular actions to be performed by the computer system; and

(iv) saving to a recognized object database the geometric defining parameters of the three-dimensional object and correlations between particular actions performed with the three-dimensional object and particular actions to be performed by the computer system; and

(b) presenting the three-dimensional object to a camera; and

(c) moving the presented three-dimensional object in front of the camera so as to trigger one or more of the particular actions to be performed by the computer system.

10. The computer implemented method as in claim 9, further comprising:

(d) activating an interactive session processed by the computer system, the interactive session includes processing through (b) and (c) at particular times during the interactive session.

11. The computer implemented method as in claim 10, wherein during the interactive session of (d), the method includes,

(i) capturing depth data using the camera and analyzing the depth data, the depth data being compared to the geometric defining parameters stored in the recognized object database;

and

(ii) triggering activation of actions during the interactive session in response to actions performed with the three-dimensional object, as defined by the correlations between particular actions performed with the three-dimensional object and particular actions to be performed by the computer system

12. The computer implemented method as recited in claim 9, wherein depth data for the three-dimensional object is captured from a plurality of angles by the camera that is a single camera, the plurality of angles allowing the computer system to process the depth data and create the geometric parameters for the three-dimensional object.

13. The computer implemented method as recited in claim 9, wherein the correlations are defined by assigning a particular action performed with the three-dimensional object to a pre-defined particular action to be performed by the computer system.

14. The computer implemented method as recited in claim 9, wherein defining the correlations by having the user interactively move the three-dimensional object in response to a prompt for a pre-defined particular action.

15. The computer implemented method as recited in claim 9, creating macro correlations for a remote control so once a new three-dimensional object is defined, basic movements have a same effect.

16. The computer implemented method as recited in claim 9, saving multiple three-dimensional objects in the recognized object database and switching between them real-time, each object having customized controls.

17. The computer implemented method as in claim 9, wherein the actions performed with the three-dimensional object include moving the object, turning the object, waving the object, tapping the object, shaking the object, pointing with the objects or combinations thereof.

18. A system for using a three-dimensional object as a controlling device when interfacing with a computer system, comprising:

a camera that is configured to capture data from a three-dimensional space;

the computer system being interfaced with the camera;

storage being linked to the computer system; and

a display coupled to the computer system, the display being configured to display a plurality of graphical display screens to enable set-up of a capture session to obtain geometric parameters of an object and assign actions to be performed with the object when moved in front of the camera during an interactive session, the geometric parameters and the assigned actions being saved to a database on the storage for access during the interactive session;

wherein the assigned actions being custom defined by a user for particular movements made by the user on the object when holding the object in front of the camera.

19. The system for using a three-dimensional object as a controlling device as recited in claim 18, wherein the storage is local to the computer system.

20. The system for using a three-dimensional object as a controlling device as recited in claim 18, wherein the database on the storage includes a plurality of previously defined objects and assigned actions.

21. The system for using a three-dimensional object as a controlling device as recited in claim 18, further comprising,

an network connection provided for the computer system, the network connection providing access to a remote storage, the remote storage having a database of previously defined objects and assigned actions.

22. The system for using a three-dimensional object as a controlling device as recited in claim 18, wherein the computer system executes one of a game program, an application, a search engine, a communication program, or combinations thereof