Assigning gesture dictionaries

• US 8,631,355 B2
• Filed: 01/08/2010
• Issued: 01/14/2014
• Est. Priority Date: 01/08/2010
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Murillo, Oscar E., Wilson, Andy, Kipman, Alex A., Galore, Janet
• Primary Examiner(s): VU, THANH T
• Application Number:
  US12/684,518
• Publication Number:
  US 20110173204A1
• Time in Patent Office: 1,467 Days
• Field of Search: 715/863, 715/864
• US Class Current: 715/863
• CPC Class Codes: A63F 13/213 comprising photodetecting means, e.g. cameras, photodiodes or infrared cells A63F13/219, A63F13/655 take precedence
• CPC Class Codes: A63F 13/22 Setup operations, e.g. calibration, key configuration or button assignment
• CPC Class Codes: A63F 13/42 by mapping the input signals into game commands, e.g. mapping the displacement of a stylus on a touch screen to the steering angle of a virtual vehicle
• CPC Class Codes: A63F 13/428 involving motion or position input signals, e.g. signals representing the rotation of an input controller or a player's arm motions sensed by accelerometers or gyroscopes
• CPC Class Codes: A63F 13/67 adaptively or by learning from player actions, e.g. skill level adjustment or by storing successful combat sequences for re-use
• CPC Class Codes: A63F 13/803 Driving vehicles or craft, e.g. cars, airplanes, ships, robots or tanks
• CPC Class Codes: A63F 13/812 Ball games, e.g. soccer or baseball
• CPC Class Codes: A63F 2300/1018 Calibration; Key and button assignment
• CPC Class Codes: A63F 2300/1087 comprising photodetecting means, e.g. a camera
• CPC Class Codes: A63F 2300/1093 using visible light
• CPC Class Codes: A63F 2300/5553 user representation in the game field, e.g. avatar
• CPC Class Codes: A63F 2300/6027 using adaptive systems learning from user actions, e.g. for skill level adjustment
• CPC Class Codes: A63F 2300/6045 for mapping control signals received from the input arrangement into game commands
• CPC Class Codes: A63F 2300/6607 for animating game characters, e.g. skeleton kinematics
• CPC Class Codes: A63F 2300/8011 Ball
• CPC Class Codes: G06F 16/436 using biological or physiological data of a human being, e.g. blood pressure, facial expression, gestures
• CPC Class Codes: G06F 16/9537 Spatial or temporal dependent retrieval, e.g. spatiotemporal queries
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883

Abstract

Techniques for assigning a gesture dictionary in a gesture-based system to a user comprise capturing data representative of a user in a physical space. In a gesture-based system, gestures may control aspects of a computing environment or application, where the gestures may be derived from a user'"'"'s position or movement in a physical space. In an example embodiment, the system may monitor a user'"'"'s gestures and select a particular gesture dictionary in response to the manner in which the user performs the gestures. The gesture dictionary may be assigned in real time with respect to the capture of the data representative of a user'"'"'s gesture. The system may generate calibration tests for assigning a gesture dictionary. The system may track the user during a set of short gesture calibration tests and assign the gesture dictionary based on a compilation of the data captured that represents the user'"'"'s gestures.

Claims

1. A method for assigning a gesture dictionary, the method comprising:

receiving data representative of a user in a physical space;

processing the received data to identify a first motion or pose by the user that invokes an input command to a computer;

correlating the first motion or pose to a first gesture dictionary of a plurality of gesture dictionaries, each gesture dictionary of the plurality of gesture dictionaries comprising a set of input commands to the computer that may be invoked by motions or poses of the user reflected in the received data; and

assigning the first gesture dictionary to the user, the first gesture dictionary corresponding to the first motion or pose; and

processing additional captured data with the first gesture dictionary to identify whether a second motion or pose by the user in the additional captured data invokes an input command to the computer.

2. The method of claim 1, further comprising requesting explicit gesture feedback from the user via instructions to the user to perform a gesture.

3. The method of claim 2, wherein the instructions comprise a display of a command and a request to the user to perform the gesture that corresponds to the command.

4. The method of claim 1, further comprising passively tracking the user and compiling captured data for processing to identify additional motions or poses by the user.

5. The method of claim 4, further comprising manipulating circumstances of an output to the user to elicit particular input gesture data.

6. The method of claim 1, wherein correlating the first motion or pose to at least one of the plurality of gesture dictionaries comprises correlating the first motion or pose to a cluster of gesture dictionaries.

7. The method of claim 1, wherein the user is a first user and correlating the first motion or pose to the at least one of the plurality of gesture dictionaries comprises:

correlating the first motion or pose of the first user to an input gesture data of a second user, and assigning a gesture dictionary assigned to the second user to the first user.

8. The method of claim 1, wherein the at least one of the plurality of gesture dictionaries is associated an identifier, wherein the identifier identifies when to implement the at least one of the plurality of gesture dictionaries.

9. The method of claim 8, wherein the identifier is at least one of an operating system, an application, a user, a feature of a user, a location, a type of application, a hardware configuration, a software configuration, a culture, current user, geography, demography, linguistic, culture, or a style.

10. The method of claim 1, wherein the gesture dictionary is assigned to the user in real time upon capturing the data representative of the user.

11. The method of claim 1, the method further comprising:

compiling captured data representative of the user;

identifying at least one identifier associated with the user;

correlating the at least one identifier to the plurality of gesture dictionaries;

assigning a portion of each of the plurality of gesture dictionaries correlating to the identifier to the user, such that a combination of data from each of the plurality of gesture dictionaries is assigned to the user.

12. The method of claim 1, further comprising:

identifying a level of failure of the first motion or pose to correspond to data in the gesture dictionary, and

providing instructional feedback to the user.

13. A system for assigning a gesture dictionary, the system comprising:

a memory communicatively coupled to a processor when the system is operational, the memory bearing processor-executable instructions that, when executed on the processor, cause the system;

at least to;

receive data representative of a user in a physical space;

process the received data to identify a first motion or pose by the user that invokes an input command to a computer;

correlate the first motion or pose to a first gesture dictionary of a plurality of gesture dictionaries, each gesture dictionary of the plurality of gesture dictionaries comprising a set of input commands to the computer that may be invoked by motions or poses of the user reflected in the received data; and

assign the at least one of the plurality of gesture dictionaries to the user, the gesture dictionary corresponding to the first motion or pose; and

process additional captured data with the first gesture dictionary to identify whether a second motion or pose by the user in the additional captured data invokes an input command to the computer.

14. The system of claim 13, wherein the memory further bears processor-executable instructions that, when executed on the processor, cause the system:

at least to;

request explicit gesture feedback from the user via instructions to the user to perform a gesture.

15. The system of claim 14, wherein the memory further bears processor-executable instructions that, when executed on the processor, cause the system:

at least to;

display a command on a display device and a request to the user to perform the gesture that corresponds to the command.

16. The system of claim 13, wherein the memory further bears processor-executable instructions that, when executed on the processor, cause the system at least to:

passively tracking the user and compiling captured data for processing to identify additional motions or poses by the user.

17. A computer-readable storage device bearing computer-executable instructions that, when executed on a computer, cause the computer to perform operations comprising:

receiving data representative of a user in a physical space;

processing the received data to identify a first motion or pose by the user that invokes an input command to the computer;

correlating the first motion or pose to a first gesture dictionary of a plurality of gesture dictionaries, each gesture dictionary of the plurality of gesture dictionaries comprising a set of input commands to the computer that may be invoked by motions or poses of the user reflected in the received data; and

assigning the first gesture dictionary to the user, the first gesture dictionary corresponding to the first motion or pose; and

processing additional captured data with the first gesture dictionary to identify whether a second motion or pose by the user in the additional captured data invokes an input command to the computer.

18. The computer-readable storage device of claim 17, wherein correlating the first motion or pose to the first gesture dictionary comprises:

determining whether the user is left-handed or right-handed; and

wherein assigning the first gesture dictionary to the user comprises;

assigning the first gesture dictionary to the user based on whether the user is left-handed or right-handed.

19. The computer-readable storage device of claim 17, wherein capturing data representative of the user in the physical space comprises:

powering down a component of the computer or a component communicatively coupled to the computer; and

capturing data representative of the user performing a motion or pose to power up the component of the computer or the component communicatively coupled to the computer.

20. The computer-readable storage device of claim 17, wherein correlating the first motion or pose to the first gesture dictionary comprises:

determining a physical location or Internet Protocol (IP) address of the user; and

correlating the first motion or pose and the physical location or IP address of the user to the first gesture dictionary of a plurality of gesture dictionaries.

21. The computer-readable storage device of claim 17, wherein correlating the first motion or pose to the first gesture dictionary comprises:

determining a skill level of the user; and

correlating the first motion or pose and the skill level of the user to the first gesture dictionary of a plurality of gesture dictionaries.