MOVEMENT BASED RECOGNITION AND EVALUATION

• US 20120143358A1
• Filed: 11/03/2011
• Published: 06/07/2012
• Est. Priority Date: 10/27/2009
• Status: Active Grant

• Current Assignee: Harmonix Music Systems Incorporated (Epic Games, Inc.)
• Original Assignee: Harmonix Music Systems Incorporated (Epic Games, Inc.)
• Inventors: ADAMS, Isaac, BOCH, Matthew C., FLURY, Marc A.
• Granted Patent: US 9,981,193 B2
• Time in Patent Office: Days
• US Class Current: 700/92
• CPC Class Codes: A63F 13/211 using inertial sensors, e.g. accelerometers or gyroscopes
• CPC Class Codes: A63F 13/213 comprising photodetecting means, e.g. cameras, photodiodes or infrared cells A63F13/219, A63F13/655 take precedence
• CPC Class Codes: A63F 13/428 involving motion or position input signals, e.g. signals representing the rotation of an input controller or a player's arm motions sensed by accelerometers or gyroscopes
• CPC Class Codes: A63F 13/44 involving timing of operations, e.g. performing an action within a time slot
• CPC Class Codes: A63F 13/46 Computing the game score
• CPC Class Codes: A63F 13/5375 for graphically or textually suggesting an action, e.g. by displaying an arrow indicating a turn in a driving game
• CPC Class Codes: A63F 13/54 involving acoustic signals, e.g. for simulating revolutions per minute [RPM] dependent engine sounds in a driving game or reverberation against a virtual wall
• CPC Class Codes: A63F 13/67 adaptively or by learning from player actions, e.g. skill level adjustment or by storing successful combat sequences for re-use
• CPC Class Codes: A63F 13/814 Musical performances, e.g. by evaluating the player's ability to follow a notation
• CPC Class Codes: A63F 2300/1087 comprising photodetecting means, e.g. a camera
• CPC Class Codes: A63F 2300/305 for providing a graphical or textual hint to the player
• CPC Class Codes: A63F 2300/6045 for mapping control signals received from the input arrangement into game commands
• CPC Class Codes: A63F 2300/61 Score computation
• CPC Class Codes: A63F 2300/638 according to the timing of operation or a time limit
• CPC Class Codes: A63F 2300/8047 Music games
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/04815 Interaction with a metaphor-based environment or interaction object displayed as three-dimensional, e.g. changing the user viewpoint with respect to the environment or object
• CPC Class Codes: G06F 3/0482 Interaction with lists of selectable items, e.g. menus
• CPC Class Codes: G06V 40/23 Recognition of whole body movements, e.g. for sport training

Abstract

This disclosure relates generally to analysis of movement that is electronically captured and recognition of which movement was performed for purposes of evaluation. This type of motion capture, analysis and evaluation can have applicability across a number of fields, such as healthcare, entertainment, communication, and manufacturing. The recognition can occur once movement information collected from sensors is inputted into the system. The system can then analyze the movement information to determine what the movement is. In some games, such as sports games, movement recognition can provide feedback to a user about what they are doing. In other games, such as dance games, movement recognition can allow for users to dance a move from a set of moves providing for a more entertaining game.

Claims

1. A method executed on a game console comprising:

instructing a display to display a current set of dance moves that are available for performance during a song;

receiving at a dance game executing on the game console an input performance that includes information relating to a dance move performance;

evaluating in the dance game executing on the game console the input performance against a plurality of target performances that each relate to one of the current set of dance moves; and

determining a degree of matching between the input performance and each of the plurality of target performances for use in the dance game.

2. The method of claim 1, further comprising evaluating movement and position information of the input performance against a key frame corresponding to a target performance of the plurality of target performances to determine the degree of matching.

3. The method of claim 2, wherein each key frame includes a skeleton representation of a movement of the target performance and provides differing weights to different aspects of the position and movement information of the target performance.

4. The method of claim 1, further comprising providing a linear progression of dance moves for performance during the song.

5. The method of claim 1, further comprising instructing the display to display each of the current set of dance moves as an icon that is animated illustrating a target performance.

6. The method of claim 1, further comprising:

assigning a score based on the degree of matching between the input performance and each of the plurality of target performances; and

recognizing a match between the input performance and one of the plurality of target performances when the score exceeds a predetermined dance move performance threshold.

7. The method of claim 1, further comprising:

instructing the display to display the current set of dance moves as icons that are animated; and

offsetting the beginning of the animation relative to the beats of the song.

8. The method of claim 1, further comprising:

receiving sensory information that includes a plurality of input performances for each sample, where each input performance includes information relating to a dance move; and

evaluating the plurality of input performances against a plurality of target performances that each relate to one of the current set of dance moves to determine matches between the plurality of input performances and each of the plurality of target performances.

9. The method of claim 1, further comprising selecting one of the current set of dance moves dynamically using a dance move selection algorithm that considers one or more criteria for each dance move.

10. The method of claim 1, further comprising:

selecting a target performance based on the degree of matching between the input performance and each of the plurality of target performances, wherein a higher degree of matching indicates a closer match; and

adding the selected target performance to a dance routine that is being created.

18. The logic of claim 10, further comprising selecting one of the current set of dance moves dynamically using a dance move selection algorithm that considers one or more criteria for each dance move.

11. Logic encoded on one or more non-transient computer readable mediums that when executed performs the steps comprising:

instruct a display to display a current set of dance moves that are available for performance during a song;

receive an input performance that includes information relating to a dance move performance;

evaluate in a dance game the input performance against a plurality of target performances that each relate to one of the current set of dance moves; and

determine a degree of matching between the input performance and each of the plurality of target performances for use in the dance game.

12. The logic of claim 11, further comprising evaluating movement and position information of the input performance against a key frame corresponding to a target performance of the plurality of target performances to determine the degree of matching between the input performance and the target performance.

13. The logic of claim 12, wherein each key frame includes a skeleton representation of a movement of the target performance and provides differing weights to different aspects of the position and movement information of the target performance.

14. The logic of claim 11, further comprising providing a linear progression of dance moves for performance during the song.

15. The logic of claim 11, further comprising instructing the display to display each of the current set of dance moves as an icon that is animated illustrating a target performance.

16. The logic of claim 11, further comprising:

assigning a score based on the degree of matching between the input performance and each of the plurality of target performances; and

recognizing a match between the input performance and one of the plurality of target performances when the score exceeds a predetermined dance move performance threshold.

17. The logic of claim 11, further comprising:

receiving sensory information that includes a plurality of input performances for each sample, where each input performance includes information relating to a dance move; and

evaluating the plurality of input performances against a plurality of target performances that each relate to one of the current set of dance moves to determine matches between the plurality of input performances and each of the plurality of target performances.

19. A game platform system configured to receive movement sensor information including an input performance relating to a dance move performance and including a processor configured to read instructions from memory to instruct a display to display a current set of dance moves, evaluate the input performance against a plurality of target performances that each relate to one of the current set of dance moves, and determine a degree of matching between the input performance and each of the plurality of target performances.

20. The game platform of claim 19, wherein the processor is configured to evaluate movement and position information of the input performance against a key frame corresponding to a target performance of the plurality of target performances to determine a match, wherein each key frame includes a skeleton representation of a movement of the target performance and provides differing weights to different aspects of the position and movement information of the target performance.