Musical interaction assisting apparatus

• US 7,750,223 B2
• Filed: 06/27/2006
• Issued: 07/06/2010
• Est. Priority Date: 06/27/2005
• Status: Active Grant

• Current Assignee: Yamaha Corporation
• Original Assignee: Yamaha Corporation
• Inventors: Nishida, Kenichi, Sakurada, Shinya, Fukada, Atsushi, Nakamura, Yoshinari, Ohshima, Osamu
• Primary Examiner(s): Donels; Jeffrey
• Assistant Examiner(s): Millikin; Andrew R
• Application Number:
  US11/475,547
• Publication Number:
  US 20070039450A1
• Time in Patent Office: 1,470 Days
• Field of Search: 446/297, 446/300, 446/175, 434/320, 434/307.A, 434/307.R, 844/70.R, 901/1
• US Class Current: 84/470.00R
• CPC Class Codes: G10H 2210/031 Musical analysis, i.e. isolation, extraction or identification of musical elements or musical parameters from a raw acoustic signal or from an encoded audio signal
• CPC Class Codes: G10H 2220/005 Non-interactive screen display of musical or status data
• CPC Class Codes: G10H 2220/155 User input interfaces for electrophonic musical instruments
• CPC Class Codes: G10H 2220/351 Environmental parameters, e.g. temperature, ambient light, atmospheric pressure, humidity, used as input for musical purposes
• CPC Class Codes: G10H 2220/455 Camera input, e.g. analyzing pictures from a video camera and using the analysis results as control data
• CPC Class Codes: G10H 5/005 Voice controlled instruments

Abstract

A musical interaction assisting apparatus is to enhance friendliness between an electronic musical instrument and the player. The player'"'"'s actions are detected acoustically, visually or physically, and the interaction assisting apparatus interprets the detected player'"'"'s actions and generates interactive responses thereto. The interactive responses are outputted acoustically, visually or physically for the player, and electronically to control the electronic musical instrument. The interaction assisting apparatus also has a learning function to provide proper responses to the player.

Claims

1. A musical interaction assisting apparatus configured to be operatively connected to an electronic musical apparatus, the musical interaction assisting apparatus comprising:

an input device for inputting action information from user with the user'"'"'s eye contact with the input device and the users sound expression or visual expression;

an interpreting device for recognizing the user'"'"'s eye contact with the input device and the user'"'"'s sound or visual expression input via said input device;

a response generating device for generating an interactive response signal based on the recognized sound or visual expression of the user making eye contact with the input device;

an interactive response output device for outputting, based on the interactive response signal, a musical performance control signal to the electronic musical apparatus to instruct start of musical performance on the electronic musical apparatus or control progression of musical performance on the electronic musical apparatus; and

a physical response output device for outputting, based on the interactive response signal, physical response to the user, including at least one of temperature change, touching, or patting felt by the user.

2. A musical interaction assisting apparatus as claimed in claim 1, wherein said musical interaction assisting apparatus is incorporated in said electronic musical apparatus.

3. A musical interaction assisting apparatus as claimed in claim 1, wherein:

said input device includes a camera for visually detecting the user'"'"'s eye contact and visual expression; and

said interpreting device interprets the detected visual expression as any of an eye movement, a behavior, a facial expression, or a gesture of the user.

4. A musical interaction assisting apparatus as claimed in claim 1, wherein said input device inputs a predetermined sound expression from the user making eye contact with the input device, and said interactive response output device outputs the electronic response signal for instructing the electronic musical apparatus to start a musical performance.

5. A musical interaction assisting apparatus as claimed in claim 1, wherein the controlling progression of musical performance includes shifting a performance part to another part or prolongs an ending of the musical performance.

6. A computer readable medium storing a computer program for an electronic musical apparatus having an input device for inputting action information from user with the user'"'"'s eye contact with the input device and the users sound expression or visual expression, the computer program containing instructions for:

recognizing the user'"'"'s eye contact with said input device and the user'"'"'s sound or visual expression input via said input device;

generating an interactive response signal based on the recognized sound or visual expression of the user making eye contact with the input device;

outputting a musical performance control signal, based on the interactive response signal, to the electronic musical apparatus to instruct start of musical performance on the electronic musical apparatus or control progression of musical performance on the electronic musical apparatus; and

controlling a physical response output device to output, based on the interactive response signal, physical response to the user, including at least one of temperature change, touching, or patting felt by the user.