Methods and apparatus of displaying and evaluating motion data in a motion game apparatus

• US 6,554,706 B2
• Filed: 06/15/2001
• Issued: 04/29/2003
• Est. Priority Date: 05/31/2000
• Status: Expired due to Fees

• Current Assignee: Euijae Ahn, Gerard Jounghyun Kim, Ungyeon Yang
• Original Assignee: Euijae Ahn, Gerard Jounghyun Kim, Ungyeon Yang
• Inventors: Yang, Ungyeon, Ahn, Euijae, Kim, Gerard Jounghyun
• Primary Examiner(s): Esquivel, Denise L.
• Assistant Examiner(s): NORMAN, MARC E
• Application Number:
  US09/881,066
• Publication Number:
  US 20020019258A1
• Time in Patent Office: 683 Days
• Field of Search: 463/36, 463/37, 463/1, 463/4, 340/825, 341/20, 345/156
• US Class Current: 463/36
• CPC Class Codes: A63F 13/10 Control of the course of the game, e.g. start, progess, end
• CPC Class Codes: A63F 13/213 comprising photodetecting means, e.g. cameras, photodiodes or infrared cells A63F13/219, A63F13/655 take precedence
• CPC Class Codes: A63F 13/45 Controlling the progress of the video game
• CPC Class Codes: A63F 13/5375 for graphically or textually suggesting an action, e.g. by displaying an arrow indicating a turn in a driving game
• CPC Class Codes: A63F 13/79 involving player-related data, e.g. identities, accounts, preferences or play histories
• CPC Class Codes: A63F 13/814 Musical performances, e.g. by evaluating the player's ability to follow a notation
• CPC Class Codes: A63F 2300/1012 involving biosensors worn by the player, e.g. for measuring heart beat, limb activity
• CPC Class Codes: A63F 2300/1087 comprising photodetecting means, e.g. a camera
• CPC Class Codes: A63F 2300/8005 Athletics

Abstract

Methods of displaying and evaluating of a motion data in a motion game apparatus are disclosed. According to the present invention, the motion of a game player following the motion of a dancer can be tracked and evaluated in real time. To induce the correct motion out of the player, in addition to simply showing the animated dancing (or any other motion) character, a display method called the “sliding ghost” metaphor is proposed. Sliding ghosts refers to consecutive freeze frames of the motion data representing key postures of the motion. Also, for enhanced viewing of the motion data, the player can set the view point and key postures may be augmented, with short texts/audio display to further inform the player of the next/current dance (motion) segment to perform. The motion of the player is captured by tracking five specific positions of the player'"'"'s body. The five body positions are tracked by having the player wear reflective markers on the respective positions, by using four analog/digital cameras with infrared light diodes/filter, a digital signal processing (DSP) board and a personal computer. The computational cost is made lower by taking advantage of the history based prediction and reference motion data. As the motion capture of the player occurs, an instant evaluation is performed by a simple comparison at each discrete time instant and later totaled and averaged for an overall score.

Claims

1. A method of displaying motion guidance cues, comprising:

receiving inputs related to a user;

displaying a main animated character whose motion the user is to emulate; and

displaying one or more ghosts of the main animated character, wherein each ghost represents a body position of the main animated character at a future time frame.

2. The method of claim 1, wherein each of the one or more ghosts is displayed at a physical distance away from the animated main character such that the physical distance is substantially in proportion to a temporal distance of the future time frame of the ghost from a current time frame.

3. The method of claim 1, further comprising:

capturing motion of the user;

comparing the motion of the user to the motion of the main animated character; and

providing feedback to the user based on the comparison.

4. The method of claim 3, wherein:

the step of capturing includes tracking one or more specific body points of the user; and

the step of comparing includes comparing the one or more specific body points of the user with corresponding one or more specific body points of the main animated character.

5. The method of claim 4, wherein the step of capturing further includes:

determining a two-dimensional position of each of the one or more specific body points of the user from two or more perspectives; and

calculating a three-dimensional position of each of the one or more specific body points of the user based on the two-dimensional position information of the user gathered from the two or more perspectives.

6. The method of claim 5, wherein the specific body points of the user include articulation points of the user.

7. The method of claim 6, wherein the two-dimensional position of each articulation point of the user is determined by sensing a two-dimensional position of a corresponding marker attached to each articulation point of the user.

8. The method of claim 7, further including predicting marker positions to compensate for at least one of:

a missing marker problem wherein a marker is occluded from detection from a particular perspective; and

a switched marker problem wherein positions of the two or more markers fail to be distinguished from sensing.

9. The method of claim 3, wherein the step of comparing includes at least one of:

scaling and retargeting motion data of the main animated character to the user;

compensating for an offset problem wherein the user has substantially followed the retargeted motion of the main animated character but wherein a physical position of the user is at an offset from a physical position specified by the main animated character; and

compensating for a delay problem wherein the user has substantially followed the retargeted motion of the main animated character but wherein the motion of the user is delayed from a timing specified by the main animated character.

10. The method of claim 9, wherein scaling and retargeting the motion data is based on a height of the user.

11. A system to display motion guidance cues, comprising:

an input unit configured to receive inputs related to a user; and

a display unit configured to;

display a main animated character whose motion the user is to emulate;

display one or more ghosts of the main animated character, wherein each ghost represents a body position of the main animated character at a future time frame; and

provide feedback information to the user.

12. The system of claim 11, wherein each of the one or more ghosts is displayed at a physical distance away from the animated main character such that the physical distance is substantially in proportion to a temporal distance of the future time frame of the ghost from a current time frame.

13. The system of claim 11, further comprising:

a motion data acquisition unit configure to capture the motion of the user; and

a computing unit configured to compare the motion of the user to the motion of the main animated character.

14. The system of claim 13, wherein:

the motion data acquisition unit is further configured to track positions of one or more specific body points of the user; and

the computing unit is further configured to compare the one or more specific body points of the user with corresponding one or more specific body points of the main animated character.

15. The system of claim 14, wherein:

the motion data acquisition unit is further configured to determine a two-dimensional position of each of the one or more specific body points of the user from two or more perspectives; and

the computing unit is further configured to calculate a three-dimensional position of each of the one or more specific body points of the user based on the two-dimensional position information of the user gathered from the two or more perspectives.

16. The system of claim 15, wherein the specific body points of the user include articulation points of the user and wherein the motion data acquisition unit is further configured to determine the two-dimensional position of each articulation point by sensing a two-dimensional position of a corresponding marker attached to each articulation point of the user.

17. The system of claim 16, wherein the motion data acquisition unit is further configured to predict the marker position to compensate for at least one of:

a missing marker problem wherein a marker is occluded from detection from a particular perspective; and

a switched marker problem wherein positions of two or more markers fail to be distinguished.

18. The system of claim 13, wherein the computing unit is further configured to perform at least one of:

scaling and retargeting motion data of the main animated character to the user;

compensating for an offset problem wherein the user has substantially followed the retargeted motion of the main animated character, but wherein a physical position of the user is at an offset from a physical position specified by the main animated character; and

compensating for a delay problem wherein the user has substantially followed the retargeted motion of the main animated character, but wherein the motion of the user is delayed from a timing specified by the main animated character.

19. The system of claim 18, wherein scaling and retargeting the motion data is based on a height of the user.

20. A method to display motion guidance cues, comprising:

receiving inputs related to a user;

displaying a main animated character whose motion the user is to emulate;

displaying one or more ghosts of the main animated character, wherein each ghost represents a body position of the main animated character at a future time frame;

scaling and retargeting motion data of the main animated character to the user;

capturing the motion of the user;

comparing the motion of the user to the retargeted motion of the main animated character;

compensating for an offset problem wherein the user has substantially followed the retargeted motion of the main animated character, but wherein a physical position of the user is at an offset from a physical position specified by the main animated character;

compensating for a delay problem wherein the user has substantially followed the retargeted motion of the main animated character, but wherein the motion of the user is delayed from a timing specified by the main animated character; and

providing feedback to the user.