Dance game and tutorial

• US 8,702,485 B2
• Filed: 11/05/2010
• Issued: 04/22/2014
• Est. Priority Date: 06/11/2010
• Status: Active Grant

• Current Assignee: Harmonix Music Systems Incorporated (Epic Games, Inc.)
• Original Assignee: Harmonix Music Systems Incorporated (Epic Games, Inc.)
• Inventors: Flury, Marc A., Crooker, Kasson D., Boch, Matthew C., Egozy, Eran B., Tate, Dean N., Goldstein, Seth, Adams, Isaac, Carriuolo, Adam
• Primary Examiner(s): Elisca, Pierre E
• Application Number:
  US12/940,794
• Publication Number:
  US 20110306396A1
• Time in Patent Office: 1,264 Days
• Field of Search: 463/7, 463/8, 463/34
• US Class Current: 463/7
• CPC Class Codes: A63F 13/213 comprising photodetecting means, e.g. cameras, photodiodes or infrared cells A63F13/219, A63F13/655 take precedence
• CPC Class Codes: A63F 13/428 involving motion or position input signals, e.g. signals representing the rotation of an input controller or a player's arm motions sensed by accelerometers or gyroscopes
• CPC Class Codes: A63F 13/44 involving timing of operations, e.g. performing an action within a time slot
• CPC Class Codes: A63F 13/46 Computing the game score
• CPC Class Codes: A63F 13/5372 for tagging characters, objects or locations in the game scene, e.g. displaying a circle under the character controlled by the player
• CPC Class Codes: A63F 13/54 involving acoustic signals, e.g. for simulating revolutions per minute [RPM] dependent engine sounds in a driving game or reverberation against a virtual wall
• CPC Class Codes: A63F 13/67 adaptively or by learning from player actions, e.g. skill level adjustment or by storing successful combat sequences for re-use
• CPC Class Codes: A63F 13/814 Musical performances, e.g. by evaluating the player's ability to follow a notation
• CPC Class Codes: A63F 2300/1093 using visible light
• CPC Class Codes: A63F 2300/6081 generating an output signal, e.g. under timing constraints, for spatialization
• CPC Class Codes: A63F 2300/61 Score computation
• CPC Class Codes: A63F 2300/638 according to the timing of operation or a time limit
• CPC Class Codes: A63F 2300/6607 for animating game characters, e.g. skeleton kinematics
• CPC Class Codes: A63F 2300/69 Involving elements of the real world in the game world, e.g. measurement in live races, real video
• CPC Class Codes: A63F 2300/8047 Music games
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883

Abstract

Presented herein are methods, apparatuses, programs, and systems for scoring a player performance comprising one or more poses in a dance-based video game. a performance 3D skeleton is received that indicates a pose of the player. A score is calculated by comparing a position, a timing, or both, associated with one or more joints of the performance 3D skeleton to a position, a timing, or both, associated with one or more joints of a target pose and altering one or more characteristics of the dance-based video game based on the score.

Claims

1. A method, executed on at least one processor, for scoring a player performance comprising one or more poses in a dance-based video game based on input received via a sensor, the method comprising:

receiving, by the at least one processor, a performance 3D skeleton indicating a pose of the player;

calculating, using the at least one processor, a score by comparing a position, a timing, or both, associated with one or more joints of the performance 3D skeleton to a position, a timing, or both, associated with one or more joints of a target pose; and

altering, using the at least one processor, one or more characteristics of the dance-based video game based on the score.

2. The method of claim 1 further comprising generating, using the at least one processor, the target pose based on motion capture data.

3. The method of claim 1 wherein the position associated with the one or more joints of the 3D skeleton and the position associated with the one or more joints of the target pose are based on a normalization of the spatial position of the one or more joints of the 3D skeleton and the one or more joints of the target pose, respectively.

4. The method of claim 3 wherein the normalizing the spatial position of a joint comprises computing a unit vector reflecting an offset of the joint relative to an origin joint.

5. The method of claim 4 wherein the normalizing the spatial position of the joint comprises:

defining a coordinate system originated at the origin joint; and

translating the unit vector into the coordinate system.

6. The method of claim 5 wherein the origin joint is one of left shoulder, right shoulder, left hip, or right hip.

7. The method of claim 3 wherein the normalizing the spatial position of a joint comprises:

computing, using the at least one processor, a first vector reflecting the offset of the joint relative to an origin joint;

computing, using the at least one processor, a second vector reflecting the offset of an intermediate joint relative to the origin joint;

computing, using the at least one processor, a third vector reflecting the offset of the joint relative to the intermediate joint; and

dividing, using the at least one processor, the first vector by the sum of the magnitudes of the second and third unit vectors.

8. A method, executed on at least one processor, for evaluating a player performance based on input received via a sensor, the method comprising:

receiving, by the at least one processor, a performance 3D skeleton indicating a portion of the player performance;

providing, by the at least one processor, a target 3D skeleton indicating a portion of a target performance;

defining, using the at least one processor, a per joint error function;

calculating, using the at least one processor, an error using the per joint error function based on the performance 3D skeleton and the target 3D skeleton; and

producing, using the at least one processor, an audio or visual indication of the error.

9. A method, executed on at least one processor, for scoring a player performance comprising one or more poses in a dance-based video game based on input received via a sensor, the method comprising:

providing, by the at least one processor, a target 3D skeleton indicating a target pose;

receiving, by the at least one processor, a plurality of 3D performance skeletons indicating player poses;

generating, using the at least one processor, an overall score for a particular 3D performance skeleton by comparing a position associated with one or more joints of the 3D performance skeleton to a corresponding position associated with one or more joints of the target 3D skeleton;

repeating, using the at least one processor, the score generating step for each of the plurality of 3D performance skeletons that fall within a predetermined temporal range to generate a plurality of overall scores; and

producing, on a display in communication with the at least one processor, an audio or visual indication based on one or more of the plurality of overall scores.

10. The method of claim 9 further comprising:

providing, by the at least one processor, a second target 3D skeleton indicating a second target pose;

receiving, by the at least one processor, a second plurality of 3D performance skeletons indicating player poses;

generating, using the at least one processor, an overall score for a second particular 3D performance skeleton by comparing a position associated with one or more joints of the second 3D performance skeleton to a corresponding one or more joints of the second target 3D skeleton;

repeating, using the at least one processor, the score generating step for each of the second plurality of 3D performance skeletons that fall within a second predetermined temporal range to generate a second plurality of overall scores; and

producing, on the display in communication with the at least one processor, a second audio or visual indication based on one or more of the first and second plurality of overall scores.

11. The method of claim 9 wherein the overall score comprises a positional score and a timing-based score.

12. The method of claim 11 wherein the target pose is associated with a target performance timing, and the timing-based score varies inversely as the difference between the target performance timing and a timing of performance of the player pose.

13. The method of claim 11 wherein the timing-based score is a first constant if the difference is less than a first predetermined threshold, varies inversely as the difference if the difference is between the first predetermined threshold and a second predetermined threshold, and is a second constant if the difference is greater than the second predetermined threshold.

14. The method of claim 11 wherein the positional score comprises one or more per joint scores.

15. The method of claim 14 wherein the contribution of each per-joint score to the overall score is weighted.

16. The method of claim 11 wherein the positional score comprises one or more body part scores, each body part score comprising one or more per-joint scores.

17. The method of claim 16 wherein the contribution of each body part score to the overall score is weighted.

18. The method of claim 9 wherein the overall score comprises a displacement score.

19. The method of claim 9 wherein the overall score comprises a velocity score.

20. The method of claim 9 wherein the overall score comprises an acceleration score.

21. The method of claim 9 wherein the comparing step comprises calculating, using the at least one processor, a Euclidean distance between the position associated with the one or more joints of the 3D performance skeleton and the corresponding position associated with the one or more joints of the target 3D skeleton to generate a per-joint score.

22. The method of claim 21 wherein the per-joint score vanes inversely as the Euclidean distance.

23. The method of claim 21 wherein the per-joint score is a first constant if the Euclidean distance is less than a first predetermined threshold, varies inversely as the Euclidean distance if the Euclidean distance is between the first predetermined threshold and a second predetermined threshold, and is a second constant if the Euclidean distance is greater than the second predetermined threshold.

24. The method of claim 21 wherein calculating, using the at least one processor, the Euclidean distance comprises weighting the contributions of the x, y, and z axes to the Euclidean distance.

25. The method of claim 24 wherein the orientations of the x, y, and z axes are relative to each body zone.

26. The method of claim 25 wherein the x, y, and z axes of an arm body zone originate at a shoulder connected to the arm and are based on a first vector from the shoulder to an opposite shoulder, a second vector in the direction of gravity, and a third vector that is a cross product of the first and second vectors.

27. The method of claim 25 wherein the x, y, and z axes of a leg body zone originate at a hip connected to the leg and are based on a first vector from the hip to an opposite hip, a second vector in the direction of gravity, and a third vector that is a cross product of the first and second vectors.

28. The method of claim 24 wherein the orientations of the x, y, and z axes are relative to the target 3D skeleton.

29. The method of claim 28 wherein the x, y, and z axes are rotated based on an angle between a vector normal to a plane of the target 3D skeleton and a vector normal to a view plane.

30. The method of claim 9 wherein the audio or visual indication comprises a visual indication of a body part score.

31. The method of claim 9 wherein the audio or visual indication comprises an audible indication of a body part score.

32. The method of claim 9 wherein the audio or visual indication comprises one more graphics, the number of graphics based on the one or more overall scores.

33. The method of claim 9 wherein the audio or visual indication comprises a simulated crowd and one or more characteristics of the simulated crowd varies based on the one or more of the plurality of overall scores.

34. The method of claim 33 wherein the one or more characteristics of the simulated crowd comprises the number of people in the crowd.

35. The method of claim 9 wherein the audio or visual indication comprises a simulated environment and one or more characteristics of the simulated environment varies based on the one or more of the plurality of overall scores.

36. The method of claim 9 wherein the target pose is selected based on its importance to the target performance.

37. A method, executed on at least one processor, for scoring a player performance comprising one or more poses in a dance-based video game based on input received via a sensor, the method comprising:

receiving, by the at least one processor, a performance 3D skeleton indicating a pose of the player;

calculating, using the at least one processor, a score by comparing a measurement of one or more reference points of the performance 3D skeleton to a measurement of one or more reference points of a target pose; and

altering, using the at least one processor, one or more characteristics of the dance-based video game based on the score.

38. The method of claim 37 wherein the measurement is a displacement.

39. The method of claim 37 wherein the measurement is a velocity.

40. The method of claim 37 wherein the measurement is an acceleration.

41. The method of claim 37 wherein the one or more reference points are one or more joints.

42. The method of claim 37 wherein the one or more reference points are one or more bones.

43. The method of claim 37 wherein the one or more reference points are derived from one or more joints.

44. The method of claim 37 wherein the one or more reference points are derived from one or more bones.

45. A computer program product, tangibly embodied in a non-transitory computer readable storage medium, for scoring a player performance comprising one or more poses in a dance-based video game based on input received via a sensor, the computer program product including instructions being operable to cause a data processing apparatus to:

receive a performance 3D skeleton indicating a pose of the player;

calculate a score by comparing a position, a timing, or both, associated with one or more joints of the performance 3D skeleton to a position, a timing, or both, associated with one or more joints of a target pose; and

alter one or more characteristics of the dance-based video game based on the score.

46. A computer program product, tangibly embodied in a non-transitory computer readable storage medium, for evaluating a player performance based on input received via a sensor, the computer program product including instructions being operable to cause a data processing apparatus to:

receive a performance 3D skeleton indicating a portion of the player performance;

provide a target 3D skeleton indicating a portion of a target performance;

define a per joint error function;

calculate an error using the per joint error function based on the performance 3D skeleton and the target 3D skeleton; and

produce, with the game platform, an audio or visual indication of the error.

47. A computer program product, tangibly embodied in a non-transitory computer readable storage medium, for scoring a player performance comprising one or more poses in a dance-based video game based on input received via a sensor, the computer program product including instructions being operable to cause a data processing apparatus to:

provide a target 3D skeleton indicating a target pose;

receive a plurality of 3D performance skeletons indicating player poses;

generate an overall score for a particular 3D performance skeleton by comparing a position associated with one or more joints of the 3D performance skeleton to a corresponding position associated with one or more joints of the target 3D skeleton;

repeat the score generating step for each of the plurality of 3D performance skeletons that fall within a predetermined temporal range to generate a plurality of overall scores; and

produce, on a display in communication with the game platform, an audio or visual indication based on one or more of the plurality of overall scores.

48. A computer program product, tangibly embodied in a non-transitory computer readable storage medium, for scoring a player performance comprising one or more poses in a dance-based video game based on input received via a sensor, the computer program product including instructions being operable to cause a data processing apparatus to:

receive a performance 3D skeleton indicating a pose of the player;

calculate a score by comparing a measurement of one or more reference points of the performance 3D skeleton to a measurement of one or more reference points of a target pose; and

alter one or more characteristics of the dance-based video game based on the score.

49. A system for scoring a player performance comprising one or more poses in a dance-based video game, comprising:

a sensor to receive an input; and

a game platform configured to;

receive a performance 3D skeleton indicating a pose of the player;

calculate a score by comparing a position, a timing, or both, associated with one or more joints of the performance 3D skeleton to a position, a timing, or both, associated with one or more joints of a target pose; and

alter one or more characteristics of the dance-based video game based on the score.

50. A system for evaluating a player performance, the system comprising:

a sensor to receive an input; and

a game platform configured to;

receive a performance 3D skeleton indicating a portion of the player performance;

provide a target 3D skeleton indicating a portion of a target performance;

define a per joint error function;

calculate an error using the per joint error function based on the performance 3D skeleton and the target 3D skeleton; and

produce an audio or visual indication of the error.

51. A system for scoring a player performance comprising one or more poses, the system comprising:

a display;

a sensor to receive an input; and

a game platform configured to;

provide a target 3D skeleton indicating a target pose;

receive a plurality of 3D performance skeletons indicating player poses;

generate an overall score for a particular 3D performance skeleton by comparing a position associated with one or more joints of the 3D performance skeleton to a corresponding position associated with one or more joints of the target 3D skeleton;

repeat the score generating step for each of the plurality of 3D performance skeletons that fall within a predetermined temporal range to generate a plurality of overall scores; and

produce, on the display, an audio or visual indication based on one or more of the plurality of overall scores.

52. A system for scoring a player performance comprising one or more poses in a dance-based video game, the system comprising:

a sensor to receive an input; and

a-game platform configured to;

receive a performance 3D skeleton indicating a pose of the player;

calculate a score by comparing a measurement of one or more reference points of the performance 3D skeleton to a measurement of one or more reference points of a target pose; and

alter one or more characteristics of the dance-based video game based on the score.