Music apparatus with motion picture responsive to body action

• US 20040000225A1
• Filed: 06/13/2003
• Published: 01/01/2004
• Est. Priority Date: 06/28/2002
• Status: Active Grant

• Current Assignee: Yamaha Corporation
• Original Assignee: Yamaha Corporation
• Inventors: Kobayashi, Eiko, Nishitani, Yoshiki
• Granted Patent: US 7,012,182 B2
• Time in Patent Office: Days
• US Class Current: 84/610
• CPC Class Codes: G10H 1/00 Details of electrophonic musical instruments
• CPC Class Codes: G10H 1/0083 using wireless transmission, e.g. radio, light, infrared
• CPC Class Codes: G10H 2210/091 for performance evaluation, i.e. judging, grading or scoring the musical qualities or faithfulness of a performance, e.g. with respect to pitch, tempo or other timings of a reference performance
• CPC Class Codes: G10H 2220/321 Garment sensors, i.e. musical control means with trigger surfaces or joint angle sensors, worn as a garment by the player, e.g. bracelet, intelligent clothing vital parameter sensing G10H2220/371; wearable interfaces in general G06F3/00

Abstract

A performance processing apparatus is operable by an operation device and equipped with a sound device and a display device. In the apparatus, a storing section stores song data representative of a music sound constituting a music song, and stores image data representative of an image. An acquiring section acquires detection information from the operation device, which is used by a user and which has a detector for detecting a chronological action or state of the user and outputting the detection information representative of the detected chronological action or state. A sound control section generates the music sound through the sound device according to the song data, and controls a progression degree of the generation of the music sound according to the acquired detection information. A display control section displays the image by the display device according to the image data in correspondence with the progression degree of the generating of the music sound. The display control section further controls a rendering mode of the image in accordance with the detection information.

Claims

1. A performance processing apparatus operable by an operation device and equipped with a sound device and a display device, comprising:

a storing section that stores song data representative of a music sound constituting a music song, and stores image data representative of an image;

an acquiring section that acquires detection information from the operation device, which is used by a user and which has a detector for detecting a chronological action or state of the user and outputting the detection information representative of the detected chronological action or state;

a sound control section that generates the music sound through the sound device according to the song data, and that controls a progression degree of the generation of the music sound according to the acquired detection information; and

a display control section that displays the image by the display device according to the image data in correspondence with the progression degree of the generating of the music sound.

2. The performance processing apparatus according to claim 1, wherein the sound control section controls the progression degree of the music sound in terms of a tempo or a beat of the music song.

3. The performance processing apparatus according to claim 1, wherein the storing section stores the image data representative of a plurality of images associated to one music song, and the display control section sequentially displays the plurality of the images in synchronization with the progression degree of the music sound.

4. The performance processing apparatus according to claim 1, wherein the storing section stores timing specification data which specifies a timing of displaying the image in association with the music song, so that the display control section displays the image in correspondence to the progression degree of the music sound based on the timing specification data.

5. The performance processing apparatus according to claim 4, wherein the storing section stores the song data, the image data and the timing specification data in the form of a composite file.

6. The performance processing apparatus according to claim 1, wherein the storing section stores the song data representative of music sounds constituting a plurality of parts of the music song, the acquiring section acquires a plurality of detection information from a plurality of operation devices, each of which is used by each user in correspondence with at least one part of the music song, the sound control section controls the progression degrees of the music sounds of the respective parts according to the plurality of the detection information acquired from the plurality of the operation devices corresponding to the respective parts, and the display control section displays a plurality of images according to the image data in correspondence with the plurality of the parts, and controls the displaying of the images in synchronization to the progression degree of the corresponding parts.

7. The performance processing apparatus according to claim 6, wherein the storing section stores the image data representative of the plurality of the images in correspondence with the plurality of the parts of the music song, and the display control section divides a screen frame of the display device into a plurality of sub frames and displays the respective images in the respective sub frames in synchronization to the progression degree of the respective parts.

8. The performance processing apparatus according to claim 6, wherein the storing section stores the image data representative of the plurality of the images in correspondence with the plurality of the parts of the music song, and the display control section displays the respective images on respective ones of separate display devices in synchronization to the progression degree of the respective parts.

9. The performance processing apparatus according to claim 1, wherein the display control section controls a rendering mode of the image displayed on the display device in accordance with a progression state of the generating of the music sound.

10. The performance processing apparatus according to claim 9, wherein the display control section controls the rendering mode of the displayed image in terms of a brightness, a color tone, an effect, a resolution or a size of the image.

11. The performance processing apparatus according the claim 1, wherein the sound control section controls the music sound generated by the sound device according to the detection information in terms of a volume, a tone, an effect and a pitch of the music sound.

12. A performance processing apparatus operable by an operation device and equipped with a sound device and a display device, comprising:

a storing section that stores song data representative of a music sound constituting a music song, and stores image data representative of an image;

an acquiring section that acquires detection information from the operation device, which is used by a user and which has a detector for detecting a chronological action or state of the user and outputting the detection information representative of the detected chronological action or state;

a sound control section that generates the music sound through the sound device according to the song data and controls the generating of the music sound according to the acquired detection information; and

a display control section that displays the image by the display device according to the image data and controls a rendering mode of the image displayed on the display device according to the acquired detection information.

13. The performance processing apparatus according to claim 12, wherein the storing section stores the song data representative of the music sounds constituting a plurality of parts of the music song, the acquiring section acquires a plurality of detection information from a plurality of operation devices, each of which is used by each user in correspondence with at least one part of the music song, the sound control section controls the progression degrees of the music sounds of the respective parts according to the plurality of the detection information acquired from the plurality of the operation devices corresponding to the respective parts, and the display control section displays a plurality of images according to the image data in correspondence with the plurality of the parts, and controls the displaying of the images in synchronization to the progression degree of the corresponding parts.

14. The performance processing apparatus according to claim 12, wherein the display control section controls the rendering mode of the displayed image in terms of a brightness, a color tone, an effect, a resolution or a size of the image.

15. The performance processing apparatus according the claim 12, wherein the sound control section controls the music sound generated by the sound device according to the detection information in terms of a volume, a tone, an effect and a pitch of the music sound.

16. A computer program for use in a performance processing apparatus being operable by an operation device and being equipped with a sound device, a display device, and a storage device which stores song data representative of a music sound constituting a music song and image data representative of an image, the computer program being executable by the performance processing apparatus for performing a process comprising the steps of:

acquiring detection information from the operation device, which is used by a user and which has a detector for detecting a chronological action or state of the user and outputting the detection information representative of the detected chronological action or state;

generating the music sound through the sound device according to the song data;

controlling a progression degree of the generation of the music sound according to the acquired detection information; and

displaying the image by the display device according to the image data in correspondence with the progression degree of the generating of the music sound.

17. A computer program for use in a performance processing apparatus being operable by an operation device and being equipped with a sound device, a display device, and a storage device which stores song data representative of a music sound constituting a music song and image data representative of an image, the computer program being executable by the performance processing apparatus for performing a process comprising the steps of:

acquiring detection information from the operation device, which is used by a user and which has a detector for detecting a chronological action or state of the user and outputting the detection information representative of the detected chronological action or state;

generating the music sound through the sound device according to the song data;

controlling the generating of the music sound according to the acquired detection information;

displaying the image by the display device according to the image data; and

controlling a rendering mode of the image displayed on the display device according to the acquired detection information.

18. A file creating apparatus for creating a data file used in a performance processing apparatus which generates a music sound constituting a music song through a sound device at a progression degree corresponding to a chronological action or state of a user and which displays an image on a display device at a timing corresponding to progression of the music sound, the file creating apparatus comprising:

a storing section that stores song data representative of the music sound to be generated through the sound device;

another storing section that stores image data representative of the image to be displayed on the display device along with the generating of the music sound; and

a creating section that provides timing specification data which specifies a timing to display the image in correspondence to the progression of the music sound, and that integrates the song data, the image data and the timing specification data into the data file for use in the performance processing apparatus.

19. A file creating apparatus for creating a data file used in a performance processing apparatus which controls a music sound constituting a music song generated by a sound device and which displays an image on a display device along with a progression of the music song in response to a chronological action or state of a user, the file creating apparatus comprising:

a storing section that stores song data representative of the music sound to be generated through the sound device;

another storing section that stores image data representative of the image to be displayed on the display device along with the progression of the music song; and

a creating section that provides rendering specification data which specifies a rendering mode of the image in accordance with the chronological action or state of the user, and that integrates the song data, the image data and the rendering specification data into the data file for use in the performance processing apparatus.

20. A method carried out in a performance processing apparatus being operable by an operation device and being equipped with a sound device, a display device, and a storage device which stores song data representative of a music sound constituting a music song and image data representative of an image, the method comprising the steps of:

acquiring detection information from the operation device, which is used by a user and which has a detector for detecting a chronological action or state of the user and outputting the detection information representative of the detected chronological action or state;

generating the music sound through the sound device according to the song data;

controlling a progression degree of the generation of the music sound according to the acquired detection information; and

displaying the image by the display device according to the image data in correspondence with the progression degree of the generating of the music sound.

21. A method carried out in a performance processing apparatus being operable by an operation device and being equipped with a sound device, a display device, and a storage device which stores song data representative of a music sound constituting a music song and image data representative of an image, the method comprising the steps of:

acquiring detection information from the operation device, which is used by a user and which has a detector for detecting a chronological action or state of the user and outputting the detection information representative of the detected chronological action or state;

generating the music sound through the sound device according to the song data;

controlling the generating of the music sound according to the acquired detection information;

displaying the image by the display device according to the image data; and

controlling a rendering mode of the image displayed on the display device according to the acquired detection information.