Performance image information creating and reproducing apparatus and method

• US 6,245,982 B1
• Filed: 09/21/1999
• Issued: 06/12/2001
• Est. Priority Date: 09/29/1998
• Status: Active Grant

• Current Assignee: Yamaha Corporation
• Original Assignee: Yamaha Corporation
• Inventors: Suzuki, Hideo, Isozaki, Yoshimasa, Sekine, Satoshi, Miyaki, Tsuyoshi
• Primary Examiner(s): Nappi, Robert E.
• Assistant Examiner(s): FLETCHER, MARLON T
• Application Number:
  US09/400,573
• Time in Patent Office: 630 Days
• Field of Search: 846/00--06, 846/09--12, 846/26, 846/34--36, 846/49--52, 846/66--68, 844/70.R, 844/77.R, 844/78, 846/62, 84D/IG.6, 345/473, 345/474, 348/155, 348/399
• US Class Current: 84/477.00R
• CPC Class Codes: G09B 15/002 Electrically operated systems
• CPC Class Codes: G10H 1/0016 Means for indicating which keys, frets or strings are to be actuated, e.g. using lights or leds
• CPC Class Codes: G10H 2220/046 Drumpad indicator, e.g. drumbeat strike indicator light on a drumpad or rim
• CPC Class Codes: G10H 2230/321 Spint cymbal, i.e. mimicking thin center-held gong-like instruments made of copper-based alloys, e.g. ride cymbal, china cymbal, sizzle cymbal, swish cymbal, zill, i.e. finger cymbals
• CPC Class Codes: G10H 2240/241 Telephone transmission, i.e. using twisted pair telephone lines or any type of telephone network
• CPC Class Codes: G10H 2240/305 Internet or TCP/IP protocol use for any electrophonic musical instrument data or musical parameter transmission purposes
• CPC Class Codes: G10H 2240/311 MIDI transmission G10H2240/056 takes precedence

Abstract

Image information representing a manner of playing a piece of music can be diplayed while playing the piece of music. Performance image information is created, which enables displaying an image showing a manner of playing a piece of music while playing the piece of music. A motion component database is created, which comprises a plurality of motion components each created for each musical instrument or each part and stores a trajectory of a fraction of motion of a player and a musical instrument in one of at least one performance method being typical of the each musical instrument or the each part. Motion descriptive information is created, which designates one of the motion components corresponding to performance information on the piece of music to be played and a designated one of the at least one performance method, on a time schedule determined by a predetermined musical time unit of the piece of music to be played. Sequence information is created, which comprises the motion descriptive information, and the performance information. The designated one of the motion components can be edited, and the edited motion component is stored in the motion component database.

Claims

1. A performance image information creating apparatus for creating performance image information for displaying an image showing a manner of playing a piece of music while playing the piece of music, comprising:

a motion component database that stores a plurality of motion components each of which corresponds to a performance method and performance information, each of said motion components representing a trajectory of a fraction of performance motion of both a musical instrument and a player of said musical instrument;

an information generating-device that generates motion component information designating at least one of said motion components in accordance with a performance method and performance information on a time schedule determined by a predetermined musical time unit of music to be played, said performance method representing how to perform said music and said performance information being information for performance of said music; and

a sequence information creating device that creates sequence information comprising said motion component information and said performance information.

2. A performance image information creating apparatus according to claim 1, wherein said predetermined musical time unit is defined by beats of music to be played.

3. A performance image information creating apparatus according to claim 1, wherein said predetermined musical time unit is defined by bars or measures of music to be played.

4. A performance image information creating apparatus according to claim 1, wherein said information generating device includes an editing device that edits the designated one of the motion components, and stores a result of editing in the motion component database as an edited motion component.

5. A performance image information creating apparatus according to claim 1, wherein the motion components each comprise rewarded information on the trajectory of the fraction of motion of at least one of each part of a body of the player and each part of the musical instrument.

6. A performance image information creating apparatus according to claim 1, wherein the motion components each comprise a motion waveform representing the trajectory of the fraction of motion, a sounding point marker indicative of timing of tone generation, and a muting point marker indicative of timing of tone elimination.

7. A performance image information creating apparatus according to claim 1, further including a time resolution setting device that individually sets time resolutions of waveforms of motion of the motion components.

8. A performance image information creating apparatus according to claim 1, further including a scene information creating device that creates scene information comprising scene components including at least background information and light source information in said image, said scene components being arranged on said time schedule, and wherein said sequence information comprises said performance information, said motion component information, and said scene information.

9. A performance image information creating apparatus according to claim 8, wherein said scene components include information for instructing information externally entered to be displayed.

10. A method of creating performance image information for displaying an image showing a manner of playing a piece of music while playing the piece of music, comprising the steps of:

creating a motion component database that stores a plurality of motion components each of which corresponds to a performance method and performance information, each of said motion components representing a trajectory of a fraction of performance motion of both a musical instrument and a player of said musical instrument;

creating motion component information designating at least one of said motion components in accordance with a performance method and performance in formation on a time schedule determined by a predetermined musical time unit of music to be played, said performance information being information for performance of said music; and

creating sequence information comprising said motion component information and said performance information.

11. A storage medium storing a program that is executable by a computer to practice a method of creating performance image information for displaying an image showing a manner of playing a piece of music while playing the piece of music, the program comprising:

a module for creating a motion component database that stores a plurality of motion components each of which corresponds to a performance method and performance information, each of said motion components representing a trajectory of a fraction of performance motion of both a musical instrument and a player of said musical instrument;

a module for creating motion component information designating at least one of said motion components in accordance with a performance method and performance in formation on a time schedule determined by a predetermined musical time unit of music to be played, said performance information being information for performance of said music; and

a module for creating sequence information comprising said motion component information and said performance information.

12. A performance image information creating apparatus for creating performance image information for displaying an image in synchronism with playing of a piece of music while playing the piece of music, comprising:

an information creating device that creates information comprising instruction information arranged on a time schedule determined by a predetermined musical time unit of the piece of music to be played, for instructing image information externally entered to be displayed; and

a sequence information creating device that creates sequence information comprising said information created by said information creating device, and performance information on the piece of music to be played.

13. A performance image information creating apparatus according to claim 12, wherein said predetermined musical time unit is defined by beats of the piece of music to be played.

14. A performance image information creating apparatus according to claim 12, wherein said predetermined musical time unit is defined by bars or measures of the piece of music to be played.

15. A performance image information creating apparatus according to claim 12, wherein said instruction information includes information for controlling visual effects to be applied to said image information externally entered.

16. A performance image information creating apparatus comprising:

an information creating device that creates information comprising instruction information arranged on a time schedule determined by a predetermined musical time unit of the piece of music to be played, for instructing image information externally entered to be displayed;

a sequence information creating device that creates sequence information comprising said information created by said information creating device, and performance information on the piece of music to be played; and

a motion component information creating device that creates motion component information comprising designation information arranged on said time schedule, for designating one of a plurality of motion components each storing a trajectory of a fraction of motion of a player and a musical instrument, said one of the plurality of motion components corresponding to said performance information and a performance method being typical of said musical instrument, and wherein said sequence information comprises said information comprising said instruction information, and said motion component information.

17. A method of creating performance image information for displaying an image in synchronism with playing of a piece of music while playing the piece of music, comprising the steps of:

creating information comprising instruction information arranged on a time schedule determined by a predetermined musical time unit of the piece of music to be played, for instructing image information externally entered to be displayed; and

creating sequence information comprising said information created by said step of creating information, and performance information on the piece of music to be played.

18. A storage medium storing a program that is executable by a computer to practice a method of creating performance image information for displaying an image in synchronism with playing of a piece of music while playing the piece of music, the program comprising:

a module for creating information comprising instruction information arranged on a time schedule determined by a predetermined musical time unit of the piece of music to be played, for instructing image information externally entered to be displayed; and

a module for creating sequence information comprising said information created by said module for creating information, and performance information on the piece of music to be played.

19. A performance image information reproducing apparatus for generating musical tones corresponding to performance information and displaying an image showing a manner of playing a piece of music represented by the performance information, based on sequence information comprising the performance information, and motion component information used for displaying the image showing the manner playing the piece of music, comprising:

a musical tone generating device that generates the musical tones based on the performance information;

an image creating device that creates the image showing the manner of playing the piece of music con-responding to the performance information, based on the motion component information; and

a motion component database that stores a plurality of motion components each of which corresponds to a performance method and performance information, each of said motion components representing a trajectory of a fraction of performance motion of both a musical instrument and a player of said musical instrument;

wherein said image creating device reads out the motion components from said motion component database, based on the motion component information, and creates the image using the motion components read from the motion component database.

20. A performance image information reproducing apparatus according to claim 19, wherein said image creating device creates the image to be displayed, by correcting skeleton data indicative of sizes of the player and the musical instrument contained in each of the motion components, based on information contained in the sequence information, for designating a size of the image to be displayed.

21. A performance image information reproducing apparatus according to claim 19, further including:

an input device that enters image information from an external device;

an image synthesizing device that synthesizes an image based on the image created by the image creating device and an image based on the image information entered by the input device; and

an effects-applying device that applies visual effects to at least one of the image created by the image correcting device, the image based on the image information entered by the input device, and the image synthesized by said image synthesizing device, in timing determined by a predetermined time unit of the piece of music being played, based on the sequence information.

22. A method of reproducing performance image information for generating musical tones corresponding to performance information and displaying an image showing a manner of playing a piece of music represented by the performance information, based on sequence information comprising the performance information, and motion component information used for displaying the image showing the manner playing the piece of music, comprising the steps of:

generating the musical tones based on the performance information;

creating the image showing the manner of playing the piece of music corresponding to the performance information, based on the motion component information; and

creating a motion component database that stores a plurality-of motion components each of which corresponds to a performance method and performance information, each of said motion components representing a trajectory of a fraction of performance motion of both a musical instrument and a player of said musical instrument;

wherein said step of creating the image reads out the motion components from said motion component database, based on the motion component information, and creates the image using the motion components read from the motion component database.

23. A storage medium storing a program that is executable by a computer to practice a method of reproducing performance image information for generating musical tones corresponding to performance information and displaying an image showing a manner of playing a piece of music represented by the performance information, based on sequence information comprising the performance information, and motion component information used for displaying the image showing the manner playing the piece of music, the program comprising:

a module for generating the musical tones based on the performance information;

a module for creating the image showing the manner of playing the piece of music corresponding to the performance information, based on the motion component information; and

a module for creating a motion component database that stores a plurality-of motion components each of which corresponds to a performance method and performance information, each of said motion components representing a trajectory of a fraction of performance motion of both a musical instrument and a player of said musical instrument;

wherein said module for creating the image reads out the motion components from said motion component database, based on the motion component information, and creates the image using the motion components read from the motion component database.

24. A performance image information creating apparatus for creating performance image information for displaying an image showing a manner of playing a piece of music while playing the piece of music, comprising:

a motion component database that stores a plurality of motion components each of which corresponds to a performance method, each of said motion components representing a trajectory of a fraction of performance motion of both a musical instrument and a player of said musical instrument; and

a sequence information creating device that creates sequence information comprising designation information arranged on a time schedule determined by a musical time unit of performance information, for designating one of the motion components stored in said motion component database, said motion component being designated in accordance with a performance method determined by said performance information.