Method and apparatus for displaying musical data in a three dimensional environment

• US 6,429,863 B1
• Filed: 02/22/2000
• Issued: 08/06/2002
• Est. Priority Date: 02/22/2000
• Status: Expired due to Term

• Current Assignee: Harmonix Music Systems Incorporated (Epic Games, Inc.)
• Original Assignee: Harmonix Music Systems Incorporated (Epic Games, Inc.)
• Inventors: Lesser, Ryan, Randall, Joshua, Rigopulos, Alexander, LoPiccolo, Greg, Schmidt, Dan, Egozy, Eran, Bell, Jonathon
• Primary Examiner(s): Jankus, Almis R.
• Application Number:
  US09/510,590
• Time in Patent Office: 896 Days
• Field of Search: 345/419, 345/619, 345/440, 345/440.1, 345/440.2, 345/764, 846/09, 844/77.R, 844/78, 434/433
• US Class Current: 345/419
• CPC Class Codes: A63F 13/44 involving timing of operations, e.g. performing an action within a time slot
• CPC Class Codes: A63F 13/5258 by dynamically adapting the position of the virtual camera to keep a game object or game character in its viewing frustum, e.g. for tracking a character or a ball
• CPC Class Codes: A63F 13/5375 for graphically or textually suggesting an action, e.g. by displaying an arrow indicating a turn in a driving game
• CPC Class Codes: A63F 13/57 Simulating properties, behaviour or motion of objects in the game world, e.g. computing tyre load in a car race game A63F13/56 takes precedence
• CPC Class Codes: A63F 13/814 Musical performances, e.g. by evaluating the player's ability to follow a notation
• CPC Class Codes: A63F 13/95 Storage media specially adapted for storing game information, e.g. video game cartridges
• CPC Class Codes: A63F 2300/206 Game information storage, e.g. cartridges, CD ROM's, DVD's, smart cards
• CPC Class Codes: A63F 2300/303 for displaying additional data, e.g. simulating a Head Up Display
• CPC Class Codes: A63F 2300/305 for providing a graphical or textual hint to the player
• CPC Class Codes: A63F 2300/638 according to the timing of operation or a time limit
• CPC Class Codes: A63F 2300/64 for computing dynamical parameters of game objects, e.g. motion determination or computation of frictional forces for a virtual car
• CPC Class Codes: A63F 2300/6684 by dynamically adapting its position to keep a game object in its viewing frustrum, e.g. for tracking a character or a ball
• CPC Class Codes: A63F 2300/8047 Music games
• CPC Class Codes: G06T 11/206 Drawing of charts or graphs
• CPC Class Codes: G06T 15/00 3D [Three Dimensional] image rendering
• CPC Class Codes: G10H 1/0008 Associated control or indicating means
• CPC Class Codes: G10H 2220/015 Musical staff, tablature or score displays, e.g. for score reading during a performance. graphical musical score editing G10H2220/121; musical score displays resulting from a transcription G10H2210/086

Abstract

A method for displaying musical data in three dimensions includes displaying musical data along a spatial path that does not lie within the image plane of a display. Graphical symbols corresponding to the musical data may be displayed on graphical surfaces aligned with the spatial path. The graphical surfaces form, in some embodiments, a tunnel through which the time axis associated with the musical data runs. The musical data may be manipulated to provide the viewer with the appearance of motion along the time axis associated with the musical data. In some embodiments, the viewer is represented by a virtual camera in the three-dimensional environment and the virtual camera is “moved” to provide the appearance of motion. A corresponding apparatus is also discussed.

Claims

1. A method for displaying musical data having a musical time axis on a display having an image plane, the method comprising the steps of:

(a) representing the musical time axis as a spatial path that does not lie within an image plane of a display; and

(b) rendering the spatial path into the image plane.

2. The method of claim 1 further comprising the step of providing a display having an image plane.

3. The method of claim 1 wherein step (a) comprises a spatial path that is continuous.

4. The method of claim 1 wherein step (a) comprises a spatial path that is discontinuous.

5. The method of claim 1 wherein step (a) comprises a spatial path that is linear.

6. The method of claim 1 wherein step (a) comprises a spatial path that is non-linear.

7. The method of claim 1 further comprising the step of displaying graphical symbols corresponding to musical data along the spatial path corresponding to the musical time axis.

8. The method of claim 7 further comprising the step of manipulating the spatial positions of the graphical symbols associated with musical data to achieve the appearance of motion along the spatial path.

9. The method of claim 1 further comprising the step of displaying graphical symbols corresponding to musical data along graphical surfaces aligned with the spatial path.

10. The method of claim 9 wherein the graphical surfaces aligned with the spatial path form a tunnel.

11. The method of claim 1 further comprising the step of changing the position of a virtual camera to achieve the appearance of motion substantially along the spatial path.

12. The method of claim 1 further comprising the step of retrieving musical data for display from a memory element.

13. The method of claim 1 further comprising the step of visually representing the position of the viewer along the musical time axis.

14. The method of claim 13 further comprising the step of aurally outputting musical data associated with the visual position of the viewer.

15. An apparatus for displaying musical data comprising:

a memory element storing musical data; and

a graphics engine, in communication with said memory element, rendering the musical data such that a spatial path corresponding to a musical time axis associated with the musical data does not lie in an image plane of a display.

16. The apparatus of claim 15 further comprising a display having an image plane.

17. The apparatus of claim 15 wherein the spatial path is non-linear.

18. The apparatus of claim 15 wherein the spatial path is discontinuous.

19. The apparatus of claim 15 wherein said graphics engine changes the position of a virtual camera to provide the appearance of motion along the spatial path.

20. The apparatus of claim 15 further comprising an audio output device.

21. A method for displaying musical data on a display having an image plane, the method comprising the steps of:

(a) providing a display having an image plane;

(b) representing a time axis associated with musical data as a path through a three-dimensional space;

(c) plotting graphical symbols associated with the musical data along the spatial path; and

(d) rendering the time axis and plotted symbols onto the image plane of the provided display.

22. The method of claim 21 further comprising the step of retrieving musical data for display from a memory element.