Action selection gesturing

• US 8,657,683 B2
• Filed: 05/31/2011
• Issued: 02/25/2014
• Est. Priority Date: 05/31/2011
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Langridge, Adam Jethro, Matthews, Christopher, Simmons, Guy, Molyneux, Peter Douglas
• Primary Examiner(s): Suhol, Dmitry
• Assistant Examiner(s): DUFFY, DAVID W
• Application Number:
  US13/149,749
• Publication Number:
  US 20120309535A1
• Time in Patent Office: 1,001 Days
• Field of Search: 463 30- 32, 463 36- 39
• US Class Current: 463/36
• CPC Class Codes: A63F 13/213 comprising photodetecting means, e.g. cameras, photodiodes or infrared cells A63F13/219, A63F13/655 take precedence
• CPC Class Codes: A63F 13/215 comprising means for detecting acoustic signals, e.g. using a microphone
• CPC Class Codes: A63F 13/42 by mapping the input signals into game commands, e.g. mapping the displacement of a stylus on a touch screen to the steering angle of a virtual vehicle
• CPC Class Codes: A63F 13/428 involving motion or position input signals, e.g. signals representing the rotation of an input controller or a player's arm motions sensed by accelerometers or gyroscopes
• CPC Class Codes: A63F 13/533 for prompting the player, e.g. by displaying a game menu
• CPC Class Codes: A63F 13/80 Special adaptations for executing a specific game genre or game mode
• CPC Class Codes: A63F 2300/1087 comprising photodetecting means, e.g. a camera
• CPC Class Codes: A63F 2300/6045 for mapping control signals received from the input arrangement into game commands
• CPC Class Codes: A63F 2300/6607 for animating game characters, e.g. skeleton kinematics
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0304 Detection arrangements using opto-electronic means constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042
• CPC Class Codes: G06V 40/28 Recognition of hand or arm movements, e.g. recognition of deaf sign language static hand signs G06V40/113

Abstract

Gestures of a computer user are observed with a depth camera. A first gesture of the computer user is identified as one of a plurality of different action selection gestures, each action selection gesture associated with a different action performable within an interactive interface controlled by gestures of the computer user. A second gesture is identified as a triggering gesture that causes performance of the action associated with the action selection gesture.

Claims

1. A data-holding subsystem holding instructions executable by a logic subsystem to:

render a spell-casting game for display on a display device;

receive a virtual skeleton including a plurality of joints, the virtual skeleton providing a machine readable representation of a game player observed with a depth camera;

identify a first gesture of the virtual skeleton as one of a plurality of different gestures, each of the plurality of different gestures characterized by different relative positions of the plurality of joints over time, and each such gesture associated with a different spell castable within the spell-casting game by a player character controlled by gestures of the game player;

identify a second gesture of the virtual skeleton that is performed after completion of the first gesture and is characterized by different relative positions of the plurality of joints over time than any of the plurality of different gestures; and

responsive to identifying the second gesture, cast the spell associated with the first gesture within the spell-casting game with a magnitude proportional to one or both of a speed at which hands of the game player move when performing the first gesture and a distance hands of the game player move when performing the first gesture, wherein a same second gesture causes casting of each of the different spells associated with the plurality of different gestures when that spell has been readied by the associated gesture.

2. The data-holding subsystem of claim 1, further holding instructions executable by a logic subsystem to:

output a spell magnitude visual displayable to the game player to visually indicate a magnitude with which the spell is readied.

3. The data-holding subsystem of claim 1, further holding instructions executable by a logic subsystem to:

translate a position of one or more joints of the virtual skeleton as a ready posture; and

acknowledge the ready posture before readying the spell associated with the first gesture for casting.

4. The data-holding subsystem of claim 3, further holding instructions executable by a logic subsystem to:

output a spell ready visual displayable to the game player to visually indicate activation of a spell-casting game mode.

5. The data-holding subsystem of claim 1, further holding instructions executable by a logic subsystem to:

output a spell selection visual displayable to the game player to visually indicate which of the different spells castable within the spell-casting game is readied for casting.

6. The data-holding subsystem of claim 1, further holding instructions executable by a logic subsystem to:

identify an audible command as one of a plurality of different spell selection commands, each spell selection command associated with a different spell castable by the player character within the spell-casting game.

7. The data-holding subsystem of claim 1, further holding instructions executable by a logic subsystem to:

identify an audible command while in a spell-casting game mode; and

augment an aspect of a spell in accordance with the audible command.

8. A method of executing a spell-casting game on a computing system, the method comprising:

receiving at the computing system information from a depth camera useable by the computing system to recognize gestures of a game player, each recognized gesture characterized by a changing pose of the game player over time;

identifying, with the computing system, a first gesture of the game player as one of a plurality of different gestures, each such gesture associated with a different spell castable within the spell-casting game executed on the computing system by a player character controlled by gestures of the game player;

identifying, with the computing system, a second gesture of the game player that is performed after completion of the first gesture and is different than any of the plurality of different gestures; and

responsive to identifying the second gesture, casting the spell associated with the first gesture within the spell-casting game executed on the computing system with a magnitude proportional to one or both of a speed at which hands of the game player move when performing the first gesture and a distance hands of the game player move when performing the first gesture, wherein a same second gesture causes casting of each of the different spells associated with the plurality of different gestures when that spell has been readied by the associated gesture.

9. The method of claim 8, further comprising:

outputting a spell magnitude visual displayable to the game player to visually indicate a magnitude with which the spell is readied.

10. The method of claim 8, further comprising:

translating a position of one or more joints of a virtual skeleton as a ready posture; and

acknowledging the ready posture before readying the spell associated with the first gesture for casting.

11. The method of claim 8, further comprising:

outputting a spell ready visual displayable to the game player to visually indicate activation of a spell casting game mode.

12. An entertainment system, comprising:

a peripheral input to receive depth images from a depth camera;

a display output to output an interactive interface to a display device;

a logic subsystem operatively connectable to the depth camera via the peripheral input and to the display device via the display output;

a data holding subsystem holding instructions executable by the logic subsystem to;

receive from the depth camera one or more depth images of a world space scene including a computer user;

model the computer user with a virtual skeleton including a plurality of joints;

translate a position of one or more joints of the virtual skeleton as a selected one of a plurality of different gestures, each of the plurality of different gestures characterized by different relative positions of the plurality of joints over time, and each such gesture associated with a different action performable within the interactive interface;

translate a position of one or more joints of the virtual skeleton as a different gesture; and

responsive to identifying the different gesture, perform within the interactive interface the action associated with the selected one of the plurality of different gestures with a magnitude proportional to one or both of a speed at which hands of the game player move when performing the selected one of the plurality of different gestures and a distance hands of the game player move when performing the selected one of the plurality of different gestures.

13. The entertainment system of claim 12, wherein a same gesture triggers each of the different actions associated with the plurality of different gestures when that action has been readied by the associated gesture.

14. The entertainment system of claim 12, further holding instructions executable by a logic subsystem to:

translate a position of one or more joints of the virtual skeleton as a ready posture; and

acknowledge the ready posture before readying an action associated with a gesture.