Binding users to a gesture based system and providing feedback to the users

• US 9,015,638 B2
• Filed: 05/01/2009
• Issued: 04/21/2015
• Est. Priority Date: 05/01/2009
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: Kipman, Alex, Perez, Kathryn Stone, Polzin, R. Stephen, Guo, William
• Primary Examiner(s): Durkin, James T
• Application Number:
  US12/434,553
• Publication Number:
  US 20100281436A1
• Time in Patent Office: 2,181 Days
• Field of Search: 715/783, 715/863, 715742-746, 715/772, 715/850, 463 36- 39, 463/25, 463/7, 463/32, 345/156, 382/154, 382/181
• US Class Current: 715/863
• CPC Class Codes: A63F 13/213 comprising photodetecting means, e.g. cameras, photodiodes or infrared cells A63F13/219, A63F13/655 take precedence
• CPC Class Codes: A63F 13/22 Setup operations, e.g. calibration, key configuration or button assignment
• CPC Class Codes: A63F 13/28 responding to control signals received from the game device for affecting ambient conditions, e.g. for vibrating players' seats, activating scent dispensers or affecting temperature or light controlling the output signals based on the game progress A63F13/50
• CPC Class Codes: A63F 13/42 by mapping the input signals into game commands, e.g. mapping the displacement of a stylus on a touch screen to the steering angle of a virtual vehicle
• CPC Class Codes: A63F 13/428 involving motion or position input signals, e.g. signals representing the rotation of an input controller or a player's arm motions sensed by accelerometers or gyroscopes
• CPC Class Codes: A63F 13/53 involving additional visual information provided to the game scene, e.g. by overlay to simulate a head-up display [HUD] or displaying a laser sight in a shooting game
• CPC Class Codes: A63F 13/812 Ball games, e.g. soccer or baseball
• CPC Class Codes: A63F 13/814 Musical performances, e.g. by evaluating the player's ability to follow a notation
• CPC Class Codes: A63F 13/833 Hand-to-hand fighting, e.g. martial arts competition A63F13/837 takes precedence
• CPC Class Codes: A63F 13/843 involving concurrently two or more players on the same game device, e.g. requiring the use of a plurality of controllers or of a specific view of game data for each player
• CPC Class Codes: A63F 2300/1093 using visible light
• CPC Class Codes: A63F 2300/6045 for mapping control signals received from the input arrangement into game commands
• CPC Class Codes: A63F 2300/6607 for animating game characters, e.g. skeleton kinematics
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0304 Detection arrangements using opto-electronic means constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042
• CPC Class Codes: G06F 3/04883 for inputting data by handwriting, e.g. gesture or text

Abstract

Techniques for managing a set of states associated with a capture device are disclosed herein. The capture device may detect and bind to users, and may provide feedback about whether the capture device is bound to, or detecting a user. Techniques are also disclosed wherein virtual ports may be associated with users bound to a capture device and feedback about the state of virtual ports may be provided.

Claims

1. A method for managing a set of states associated with users of a gesture-based system, comprising:

setting a state of the gesture-based system to an unbound and detected state in which a first user is not bound to a first virtual port of the gesture-based system based on determining that the first user is present in a first depth image;

after setting the state for the first user to an unbound and detected state, binding the first user to the first virtual port and setting the state for the first user to a bound and detected state;

processing a motion or pose of the first user captured in a second depth image to invoke a command to the gesture-based system after setting the state for the first user to the bound and detected state; and

reassigning a second user from a second virtual port to the first virtual port after the first user becomes unbound from the first virtual port, based at least on input received at least at one of the first virtual port or the second virtual port.

2. The method of claim 1, further comprising:

before determining that the first user is present in the first depth image, setting the state of the gesture-based system to an unbound and undetected state in which the first user is not bound to the first virtual port based on determining that the first user is not present in a third depth image.

3. The method of claim 1, further comprising:

after binding the first user to the first virtual port and setting the state for the first user to the bound and detected state, setting the state for the first user to a bound and undetected state, where the first user is still bound to the first virtual port, but is not in a capture area of a depth image capture device, based on determining that the first user is no longer present in a third depth image.

4. The method of claim 3, further comprising:

after setting the state for the first user to the bound and undetected state, again setting the state for the first user to the bound and detected state where the first user is still bound to the first virtual port, based on determining that the first user is again present.

5. The method of claim 3, further comprising:

changing the state for the first user from bound and undetected to unbound and undetected based on determining that the first user has not been present for at least a threshold amount of time.

6. The method of claim 1, further comprising:

unbinding the first user from the first virtual port and changing the state for the first user to unbound and detected based on determining that the first user has made an exit gesture while being present.

7. The method of claim 1, further comprising:

providing visual or aural feedback in response to changing the state for the first user.

8. The method of claim 1, wherein reassigning a second user from a second virtual port to the first virtual port after the first user becomes unbound from the first virtual port comprises:

unbinding the first user from the first virtual port, unbinding the second user from the second virtual port, binding the first user to the second virtual port, and binding the second user to the first virtual port, based on determining that the first user has made a handoff gesture.

9. The method of claim 1, wherein reassigning a second user from a second virtual port to the first virtual port after the first user becomes unbound from the first virtual port comprises:

unbinding the user from the first virtual port, unbinding the second user from the second virtual port, binding the first user to the second virtual port, and binding the second user to the first virtual port based on determining that the first user and the second user have together made a handoff gesture.

10. A system for managing a set of states associated with a first user of a gesture-based system, comprising:

a processor; and

a memory communicatively-coupled to the memory when the system is operational, the memory bearing processor-executable instructions that, when executed on the processor, cause the system at least to;

set a state of the gesture-based system to an unbound and detected state in which the first user is not bound to a virtual port of the gesture based system based on determining that the first user is present in a first depth image;

after setting the state for the first user to an unbound and detected state, bind the first user to the first virtual port and setting the state for the first user to a bound and detected state;

in response to determining that the first user is in a bound and detected state where the first user is bound to the first virtual port, process a motion or pose of the first user captured in a second depth image to invoke a command to the gesture-based system; and

reassigning a second user from a second virtual port to the first virtual port after the first user changes from bound to unbound from the virtual port, based at least on input received at least at one of the first virtual port or the second virtual port.

11. The system of claim 10, wherein the memory further bears computer-executable instructions that, when executed on the processor, cause the system at least to:

before determining that the first user is present in the first depth image, set the state of the gesture-based system to an unbound and undetected state in which the first user is not bound to the first virtual port based on determining that the first user is not present in a third depth image.

12. The system of claim 10, wherein the memory further bears computer-executable instructions that, when executed on the processor, cause the system at least to:

after binding the first user to the first virtual port and setting the state for the first user to the bound and detected state, set the state for the first user to a bound and undetected state, where the first user is still bound to the first virtual port, but is not in a capture area of a depth image capture device, based on determining that the first user is no longer present in a third depth image.

13. The system of claim 12, wherein the memory further bears computer-executable instructions that, when executed on the processor, cause the system at least to:

after setting the state for the first user to the bound and undetected state, again set the state for the first user to the bound and detected state where the first user is still bound to the first virtual port, based on determining that the first user is again present in a fourth depth image.

14. The system of claim 12, wherein the memory further bears computer-executable instructions that, when executed on the processor, cause the system at least to:

change the state for the first user from bound and undetected to unbound and undetected based on determining that the first user has not been present for at least a threshold amount of time.

15. A computer-readable storage device that is not a propagating signal for managing a set of states associated with a first user of a gesture-based system, the computer-readable storage device bearing computer-readable instructions that, when executed on a computer, cause the computer to perform operations comprising:

setting a state of the gesture-based system to an unbound and detected state in which the first user is not bound to a first virtual port of the gesture based system, based on determining that the first user is present in a first depth image;

after setting the state for the first user to an unbound and detected state, binding the first user to the first virtual port and setting the state for the first user to a bound and detected state;

in response to determining that the first user is in a bound and detected state where the first user is bound to the first virtual port, processing a motion or pose of the first user captured in a second depth image to invoke a command to the gesture-based system; and

changing a binding of a second user from a second virtual port to the first virtual port after the first user becomes unbound from the first virtual port, based at least on input from at least one of the first virtual port or the second virtual port.

16. The computer-readable storage device of claim 15, further bearing computer-readable instructions that, when executed on the computer, cause the computer to perform operations comprising:

unbinding the first user from the virtual port and changing the state for the first user to unbound and detected, based on determining that the first user has made an exit gesture while being present in a third depth image.

17. The computer-readable storage device of claim 15, further bearing computer-readable instructions that, when executed on the computer, cause the computer to perform operations comprising:

providing visual or aural feedback in response to changing the state for the first user.

18. The computer-readable storage device of claim 15, wherein changing a binding of a second user from a second virtual port to the first virtual port after the first user becomes unbound from the virtual port comprises:

unbinding the first user from the first virtual port, unbinding the second user from the second virtual port, binding the first user to the second virtual port, and binding the second user to the first virtual port, based on determining that the first user has made a handoff gesture.