Methods and apparatus of displaying and evaluating motion data in a motion game apparatus

• US 20020019258A1
• Filed: 06/15/2001
• Published: 02/14/2002
• Est. Priority Date: 05/31/2000
• Status: Active Grant

• Current Assignee: Euijae Ahn, Gerard Jounghyun Kim, Ungyeon Yang
• Original Assignee: Euijae Ahn, Gerard Jounghyun Kim, Ungyeon Yang
• Inventors: Yang, Ungyeon, Ahn, Euijae, Kim, Gerard Jounghyun
• Granted Patent: US 6,554,706 B2
• Time in Patent Office: Days
• US Class Current: 463/36
• CPC Class Codes: A63F 13/10 Control of the course of the game, e.g. start, progess, end
• CPC Class Codes: A63F 13/213 comprising photodetecting means, e.g. cameras, photodiodes or infrared cells A63F13/219, A63F13/655 take precedence
• CPC Class Codes: A63F 13/45 Controlling the progress of the video game
• CPC Class Codes: A63F 13/5375 for graphically or textually suggesting an action, e.g. by displaying an arrow indicating a turn in a driving game
• CPC Class Codes: A63F 13/79 involving player-related data, e.g. identities, accounts, preferences or play histories
• CPC Class Codes: A63F 13/814 Musical performances, e.g. by evaluating the player's ability to follow a notation
• CPC Class Codes: A63F 2300/1012 involving biosensors worn by the player, e.g. for measuring heart beat, limb activity
• CPC Class Codes: A63F 2300/1087 comprising photodetecting means, e.g. a camera
• CPC Class Codes: A63F 2300/8005 Athletics

Abstract

Methods of displaying and evaluating of a motion data in a motion game apparatus are disclosed. According to the present invention, the motion of a game player following the motion of a dancer can be tracked and evaluated in real time. To induce the correct motion out of the player, in addition to simply showing the animated dancing (or any other motion) character, a display method called the “sliding ghost” metaphor is proposed. Sliding ghosts refers to consecutive freeze frames of the motion data representing key postures of the motion. Also, for enhanced viewing of the motion data, the player can set the view point and key postures may be augmented, with short texts/audio display to further inform the player of the next/current dance (motion) segment to perform. The motion of the player is captured by tracking five specific positions of the player'"'"'s body. The five body positions are tracked by having the player wear reflective markers on the respective positions, by using four analog/digital cameras with infrared light diodes/filter, a digital signal processing (DSP) board and a personal computer. The computational cost is made lower by taking advantage of the history based prediction and reference motion data. As the motion capture of the player occurs, an instant evaluation is performed by a simple comparison at each discrete time instant and later totaled and averaged for an overall score.

Claims

1. A method of displaying motion guidance cues, namely the annotated “

sliding”

ghost metaphor. In addition to showing a computer character dance according to the motion (or dance) data, for any given time instance (say at t=t1), few frozen faiture motion frames (drawn as a transparent ghostly figure) of t1+k1*Δ

t, t1+k2*Δ

t, t1+k3*Δ

t, t+k4*Δ

t, . . . t1+kn*Δ

t are shown beside the main character as well. The ghosts are drawn at a fixed distance from the main character proportional to how advanced in time of frame it represents. Therefore, as time progresses, each frozen ghostly frame is seen as approaching toward, and finally merging into, the main frame.

The number of frozen frames displayed (the value of n), values of k_i'"'"'s and the amount of basic time increment At is user settable (usually n=3 and Δ

t 0.5 second). Usually, k_i'"'"'s are set (by the developer) so that the frozen frames represent important “

key postures”

during a particular motion. These frozen motion frames of slight future are used as cues to remind players the next dance/stance/motion to perform/take.

Key postures may be augmented with short texts/audio display (located in the lower part of the screen underneath the main character) to further inform the player of the next/current dance (motion) segment to perform (e.g. “

hands up!”

, “

disco time”

, etc.). The key postures and annotated texts/sounds are designated using a separate management tool preferably by the motion expert (e.g. dancer) and saved in a separate file. The motion of the player is captured by tracking five specific positions on the player'"'"'s body (its mechanism to be explained later). These are the two wrists, two ankles and the belly. To give feedback and inform the player of how well one is following the motion on the screen, whenever the player'"'"'s tracked positions fall within a threshold value of where it should be, the corresponding positions on the animated figure are highlighted with special effects (e.g. exploding lights) and is given commendation (e.g. “

perfect”

, “

excellent”

, “

good”

, etc.).

2. A method of approximately tracking user'"'"'s motion or collecting motion data in real time by both employing history based prediction and taking advantage of the reference motion data. At the beginning of the dance or motion, the player takes a known posture in order to initialize the marker'"'"'s position. Thereafter, markers are tracked by using the marker'"'"'s profile history, namely their travel direction, velocity and acceleration. To reduce image processing time, small windows for searching markers within are opened around predicted marker positions. When users make abrupt motions and thereby violate the prediction principle, the markers will be “

missed”

(markers are not present in the search window). After a few frames of empty search windows, the prediction windows are reset at new positions predicted by the original motion data. When tracked markers deviate too much from their course as predicted by the original motion, search windows are reset assuming that a “

switched”

marker problem has occurred and the player is making an attempt to follow the original motion. Such a prediction based approach reduces the computational cost and thus the overall hardware cost.

3. A method of evaluating the motion of a game player by making a frame by frame comparison of the 3D position data of the important body positions. When a marker is missed, it can be interpreted as having too much deviation from the predicted path, thus resulting in a low score. In order to overcome the “

offset”

problem, where the player has managed to follow the motion qualitatively, but off the center position (thus resulting in a low score), a simple compensation can be made by monitoring the player'"'"'s offset from the enter position. The offset is computed by estimating the center position of the player as being at the midpoint of one'"'"'s two ankles (tracked by the cameras). Tracking the belly marker for this purpose is an unreliable method because it often gets confused with the passing wrist markers. The “

delay”

problem, in which the player is slightly late in following the motion, can be overcome by applying “

time shifts”

, by fitting the player'"'"'s motion data to that of the onginal motion data at nearby critical points.