GESTURE STYLE RECOGNITION AND REWARD

• US 20110151974A1
• Filed: 12/18/2009
• Published: 06/23/2011
• Est. Priority Date: 12/18/2009
• Status: Abandoned Application

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Deaguero, Joel B.
• Application Number:
  US12/642,589
• Publication Number:
  US 20110151974A1
• Time in Patent Office: Days
• US Class Current: 463/37
• CPC Class Codes: A63F 13/213 comprising photodetecting means, e.g. cameras, photodiodes or infrared cells A63F13/219, A63F13/655 take precedence
• CPC Class Codes: A63F 13/428 involving motion or position input signals, e.g. signals representing the rotation of an input controller or a player's arm motions sensed by accelerometers or gyroscopes
• CPC Class Codes: A63F 13/44 involving timing of operations, e.g. performing an action within a time slot
• CPC Class Codes: A63F 13/812 Ball games, e.g. soccer or baseball
• CPC Class Codes: A63F 13/833 Hand-to-hand fighting, e.g. martial arts competition A63F13/837 takes precedence
• CPC Class Codes: A63F 2300/1093 using visible light
• CPC Class Codes: A63F 2300/5553 user representation in the game field, e.g. avatar
• CPC Class Codes: A63F 2300/6045 for mapping control signals received from the input arrangement into game commands
• CPC Class Codes: A63F 2300/64 for computing dynamical parameters of game objects, e.g. motion determination or computation of frictional forces for a virtual car
• CPC Class Codes: A63F 2300/6607 for animating game characters, e.g. skeleton kinematics
• CPC Class Codes: A63F 2300/69 Involving elements of the real world in the game world, e.g. measurement in live races, real video
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883

Abstract

Systems, methods and computer readable media are disclosed for determining whether a given gesture was performed with a particular style. This style information may then be used to personalize a gaming or multimedia experience, rewarding users for their individual style.

Claims

1. In a system comprising a computing environment coupled to a capture device for capturing user motion, a method of personalizing a user experience in a software application, comprising the steps of:

a) detecting, on a pass/fail basis, whether the user performed a gesture;

b) detecting at least one qualitative aspect, in addition to detecting whether the user performed a gesture, relating to how the user performed the gesture; and

c) providing feedback to the user based on the at least one qualitative aspect detected in step b).

2. The method of claim 1, said step b) comprising the step of comparing parameters obtained in said step a) against a stored set of rules relating to the at least one qualitative aspect.

3. The method of claim 1, said step b) comprising the step of comparing parameters derived from said step a) against a stored set of rules relating to the at least one qualitative aspect.

4. The method of claim 1, said step b) comprising the step of analyzing at least one of:

b1) a maximum and minimum position, relative to the capture device along three axes;

b2) a change in position over time along three axes;

b3) a maximum and minimum velocity along three axes;

b4) a change in velocity over time along three axes;

b5) a maximum and minimum acceleration along three axes;

b6) a change in acceleration over time along three axes; and

b7) facial expression.

5. The method of claim 4, said step b) further comprising the step of analyzing at least one of b1) through b6) for a plurality of different body parts.

6. The method of claim 5, said step b) further comprising the step of comparing the analyzed parameters of at least one of b1) through b7) against a stored set of rules defining styles associated with a given gesture.

7. The method of claim 5, said step b) further comprising the step of analyzing at least one of b1) through b7) from a period of time equal to one of:

b8) a predetermined period of time prior to completion of the gesture;

b9) a length of time required to perform the detected gesture; and

b10) a length of time required to perform the detected gesture plus a predetermined period of time before and/or after the gesture.

8. The method of claim 1, said step b) comprising the step of detecting at least one of the following qualitative aspects associated with the gesture:

b11) a graceful movement associated with the gesture;

b12) an amount of effort exerted in performing the gesture;

b13) body control in performing the gesture;

b14) precision of movement in performing the gesture;

b15) efficiency of movement in performing the gesture;

b16) how steady portions of the user'"'"'s body remained in performing the gesture;

b17) how relaxed portions of the user'"'"'s body remained in performing the gesture; and

b18) how dramatic the user'"'"'s movements were in performing the gesture.

9. The method of claim 1, said step c) comprising the step of altering a portion of the software application presented to the user to personalize what is presented to the user based on the qualitative aspect detected in said step b).

10. In a gaming system comprising a computing environment coupled to a capture device for capturing user motion, a computer-readable storage medium bearing computer-readable instructions that, when executed on a processor, cause the processor to perform a method comprising the steps of:

a) receiving data relating to user motion;

b) determining, on a pass/fail basis, whether the user motion data received in said step a) corresponds to a predefined gesture;

c) determining at least one qualitative aspect, in addition to whether the user motion data received in said step a) corresponds to a predefined gesture, relating to a style with which the user performed the motion in said step a); and

d) providing feedback to the user based on the at least one qualitative aspect detected in step c).

11. The method of claim 10, wherein said step c) may determine at least one qualitative aspect where said step b) determines a user did not perform a predefined gesture.

12. The method of claim 10, wherein said step c) comprises the step of comparing metadata relating to the qualitative aspect of the user motion against a set of rules defining when user motion qualifies as a predefined style.

13. The method of claim 12, wherein said step c) comprises storing a different set of rules for different predefined gestures.

14. The method of claim 12, wherein said step c) comprises storing a single rule that applies across different predefined gestures and/or different user motions.

15. The method of claim 12, wherein said step c) comprises storing rules with predefined parameter values relating to at least one of a position or change in position, a velocity or a change in velocity or an acceleration or change of acceleration of one or more portions of the user'"'"'s body that are interpreted as a particular style with which the user'"'"'s motion was performed.

16. The method of claim 12, wherein said step c) comprises storing rules defining when a user'"'"'s motion is interpreted as one of the following styles:

c1l) a graceful movement associated with the user'"'"'s motion;

c2) an amount of effort exerted in performing the user'"'"'s motion;

c3) body control in performing the user'"'"'s motion;

c4) precision of movement in performing the user'"'"'s motion;

c5) efficiency of movement in performing the user'"'"'s motion;

c6) how steady portions of the user'"'"'s body remained in performing the user'"'"'s motion;

c7) how relaxed portions of the user'"'"'s body remained in performing the user'"'"'s motion; and

c8) how dramatic the user'"'"'s movements were in performing the user'"'"'s motion.

17. A gaming system, comprising:

an image capture device for capturing data relating to motion of a user;

a computing environment for receiving image data from the capture device and for hosting a gaming application, the computing environment including,

a first order gesture recognition engine for receiving data relating to the motion of a user, and determining on a pass/fail basis whether the motion of the user qualifies as a predefined gesture,

a second order gesture recognition engine for receiving at least one of the data and information derived from the data, and determining, in addition to a threshold determination of whether the motion of the user qualifies as a predefined gesture, whether the motion of the user includes a stylistic attribute which qualifies as a predefined style associated with the user motion, and

a set of stored rules used by the second order gesture recognition engine, the set of stored rules including definitions of when a predefined set of user motions is to be interpreted as a predefined style; and

an audiovisual device, coupled to the computing environment, for presenting a graphical representation of the user and the user'"'"'s motion based on information received from the computing environment, the graphical representation being enhanced by showing a user'"'"'s motion or surroundings with graphics representing a style determined to exist by the second order gesture recognition engine.

18. The gaming system of claim 17, wherein the computing environment causes the audiovisual device to show a user gesture determined to exist by the first order gesture recognition engine, and the gesture is shown with graphics representing a style determined to exist by the second order gesture recognition engine.

19. The gaming system of claim 17, wherein the computing environment causes the audiovisual device to show graphics representing a style associated with a user motion where the first order gesture recognition engine determines the user motion does not qualify as a gesture.

20. The gaming system of claim 17, wherein the second order gesture recognition engine analyzes parameters associated with the user'"'"'s motion including at least one of a position or change in position, a velocity or a change in velocity and an acceleration or change of acceleration of one or more portions of the user'"'"'s body and compares those parameters against parameters in the set of stored rules.