ACTION TRIGGER GESTURING

• US 20120309516A1
• Filed: 05/31/2011
• Published: 12/06/2012
• Est. Priority Date: 05/31/2011
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Langridge, Adam Jethro, Matthews, Christopher
• Granted Patent: US 8,740,702 B2
• Time in Patent Office: Days
• US Class Current: 463/31
• CPC Class Codes: A63F 13/213 comprising photodetecting means, e.g. cameras, photodiodes or infrared cells A63F13/219, A63F13/655 take precedence
• CPC Class Codes: A63F 13/42 by mapping the input signals into game commands, e.g. mapping the displacement of a stylus on a touch screen to the steering angle of a virtual vehicle
• CPC Class Codes: A63F 13/428 involving motion or position input signals, e.g. signals representing the rotation of an input controller or a player's arm motions sensed by accelerometers or gyroscopes
• CPC Class Codes: A63F 2300/1087 comprising photodetecting means, e.g. a camera
• CPC Class Codes: A63F 2300/1093 using visible light
• CPC Class Codes: A63F 2300/6045 for mapping control signals received from the input arrangement into game commands
• CPC Class Codes: A63F 2300/6607 for animating game characters, e.g. skeleton kinematics
• CPC Class Codes: A63F 2300/6692 using special effects, generally involving post-processing, e.g. blooming
• CPC Class Codes: A63F 2300/8029 Fighting without shooting
• CPC Class Codes: A63F 2300/8076 Shooting
• CPC Class Codes: G03B 15/16 for photographing the track of moving objects high-speed photography G03B39/00
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0304 Detection arrangements using opto-electronic means constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042
• CPC Class Codes: G06V 40/28 Recognition of hand or arm movements, e.g. recognition of deaf sign language static hand signs G06V40/113

Abstract

Gestures of a computer user are observed with a depth camera. A throwing gesture of the computer user is identified and an aiming vector is calculated from a path of a left and/or right hand during the throwing gesture. An interface action is directed along the aiming vector within an interactive interface.

Claims

1. A data-holding subsystem holding instructions executable by a logic subsystem to:

render a spell-casting game for display on a display device;

receive a virtual skeleton including a plurality of joints, the virtual skeleton providing a machine readable representation of a game player observed with a depth camera;

recognize a throwing gesture of the virtual skeleton;

track a path of a left hand joint of the virtual skeleton and/or a path of a right hand joint of the virtual skeleton during the throwing gesture; and

calculate an original aiming vector within the spell-casting game from the path of the left hand joint and/or the path of the right hand joint during the throwing gesture.

2. The data-holding subsystem of claim 1, further holding instructions to direct a spell along the original aiming vector within the spell-casting game.

3. The data-holding subsystem of claim 1, further holding instructions to direct a spell at a primary target within a threshold distance of the original aiming vector within the spell-casting game.

4. The data-holding subsystem of claim 1, further holding instructions to resolve a spell with a relatively larger magnitude if the original aiming vector is within a relatively shorter threshold distance of a primary target, to resolve the spell with a relatively smaller magnitude if the original aiming vector is within a relatively longer threshold distance of the primary target, and to resolve the spell with a zero magnitude if the original aiming vector is not within the relatively longer threshold distance of the primary target.

5. The data-holding subsystem of claim 1, further holding instructions to resolve a first spell if the throwing gesture is a one-handed throwing gesture and to resolve a second spell, different than the first spell, if the throwing gesture is a two-handed throwing gesture.

6. The data-holding subsystem of claim 1, further holding instructions to select a spell to resolve responsive to identifying a spell selection gesture associated with the selected spell prior to recognizing the throwing gesture, the spell selection gesture being one of a plurality of different spell selection gestures, each spell selection gesture associated with a different spell castable within the spell-casting game by a player character controlled by gestures of the game player.

7. The data-holding subsystem of claim 1, further holding instructions to

recognize an after-touch gesture of the virtual skeleton;

track a path of a left hand joint of the virtual skeleton and/or a path of a right hand joint of the virtual skeleton during the after-touch gesture; and

calculate an after-touch aiming vector within the spell-casting game from the path of the left hand joint and/or the path of the right hand joint during the after-touch gesture.

8. The data-holding subsystem of claim 7, further holding instructions to redirect a spell along the after-touch aiming vector from a starting position derived from the original aiming vector.

9. The data-holding subsystem of claim 7, further holding instructions to redirect a spell at a secondary target within a threshold distance of the after-touch aiming vector.

10. The data-holding subsystem of claim 7, further holding instructions to resolve a spell with a relatively larger magnitude if the after-touch aiming vector is within a relatively shorter threshold distance of a secondary target, to resolve the spell with a relatively smaller magnitude if the after-touch aiming vector is within a relatively longer threshold distance of the secondary target, and to resolve the spell with a zero magnitude if the after-touch aiming vector is not within the relatively longer threshold distance of the secondary target.

11. A method of executing a spell-casting game on a computing system, the method comprising:

observing gestures of a game player with a depth camera;

recognizing a throwing gesture of the game player;

tracking a path of a left hand of the game player and/or a path of a right hand of the game player during the throwing gesture;

calculating an original aiming vector within the spell-casting game from the path of the left hand and/or the path of the right hand during the throwing gesture;

recognizing an after-touch gesture of the game player;

tracking a path of the left hand and/or a path of the right hand during the after-touch gesture; and

calculating an after-touch aiming vector within the spell-casting game from the path of the left hand and/or the path of the right hand during the after-touch gesture.

12. The method of claim 11, further comprising directing a spell along the original aiming vector within the spell-casting game.

13. The method of claim 11, further comprising directing a spell at a primary target within a threshold distance of the original aiming vector within the spell-casting game.

14. The method of claim 11, further comprising resolving a spell with a relatively larger magnitude if the original aiming vector is within a relatively shorter threshold distance of a primary target, resolving the spell with a relatively smaller magnitude if the original aiming vector is within a relatively longer threshold distance of the primary target, and resolving the spell with a zero magnitude if the original aiming vector is not within the relatively longer threshold distance of the primary target.

15. The method of claim 11, further comprising resolving a first spell if the throwing gesture is a one-handed throwing gesture and resolving a second spell, different than the first spell, if the throwing gesture is a two-handed throwing gesture.

16. The method of claim 11, further comprising selecting a spell to resolve responsive to identifying a spell selection gesture associated with the selected spell prior to recognizing the throwing gesture, the spell selection gesture being one of a plurality of different spell selection gestures, each spell selection gesture associated with a different spell castable within the spell-casting game by a player character controlled by gestures of the game player.

17. The method of claim 11, further comprising redirecting a spell along the after-touch aiming vector from a starting position derived from the original aiming vector.

18. The method of claim 11, further comprising redirecting a spell at a secondary target within a threshold distance of the after-touch aiming vector.

19. The method of claim 11, further comprising resolving a spell with a relatively larger magnitude if the after-touch aiming vector is within a relatively shorter threshold distance of a secondary target, resolving the spell with a relatively smaller magnitude if the after-touch aiming vector is within a relatively longer threshold distance of the secondary target, and resolving the spell with a zero magnitude if the after-touch aiming vector is not within the relatively longer threshold distance of the secondary target.

20. An entertainment system, comprising:

a peripheral input to receive depth images from a depth camera;

a display output to output an interactive interface to a display device;

a logic subsystem operatively connectable to the depth camera via the peripheral input and to the display device via the display output;

a data-holding subsystem holding instructions executable by the logic subsystem to;

receive from the depth camera one or more depth images of a world space scene including a computer user;

model the computer user with a virtual skeleton including a plurality of joints;

translate a position of one or more joints of the virtual skeleton as a throwing gesture of the virtual skeleton;

track a path of a left hand joint of the virtual skeleton and/or a path of a right hand joint of the virtual skeleton during the throwing gesture;

calculate an original aiming vector within the interactive interface from the path of the left hand joint and/or the path of the right hand joint during the throwing gesture; and

direct an interface action along the original aiming vector within the interactive interface.