Apparatus and method for detecting performer's motion to interactively control performance of music or the like

• US 7,781,666 B2
• Filed: 04/07/2006
• Issued: 08/24/2010
• Est. Priority Date: 01/11/2000
• Status: Active Grant

• Current Assignee: Yamaha Corporation
• Original Assignee: Yamaha Corporation
• Inventors: Sato, Masaki, Nishitani, Yoshiki, Usa, Satoshi
• Primary Examiner(s): Fletcher; Marlon T
• Application Number:
  US11/400,710
• Publication Number:
  US 20060185502A1
• Time in Patent Office: 1,600 Days
• Field of Search: None
• US Class Current: 84/723
• CPC Class Codes: A63B 2071/0625 Emitting sound, noise or music
• CPC Class Codes: A63B 2071/0647 Visualisation of executed movements
• CPC Class Codes: A63B 2220/30 Speed
• CPC Class Codes: A63B 2220/34 Angular speed
• CPC Class Codes: A63B 2220/40 Acceleration
• CPC Class Codes: A63B 2220/803 Motion sensors
• CPC Class Codes: A63B 2220/805 Optical or opto-electronic sensors
• CPC Class Codes: A63B 2225/50 Wireless data transmission, e.g. by radio transmitters or telemetry
• CPC Class Codes: A63B 2230/00 Measuring physiological parameters of the user
• CPC Class Codes: A63B 2230/065 within a certain range
• CPC Class Codes: A63B 2230/62 posture
• CPC Class Codes: A63B 69/0028 for running, jogging or speed-walking movable endless bands A63B22/02
• CPC Class Codes: A63B 71/0686 Timers, rhythm indicators or pacing apparatus using electric or electronic means
• CPC Class Codes: G10H 1/00 Details of electrophonic musical instruments
• CPC Class Codes: G10H 2220/135 Musical aspects of games or videogames; Musical instrument-shaped game input interfaces game background music G10H2210/026; musical game scoring or performance evaluation G10H2210/091
• CPC Class Codes: G10H 2220/206 Conductor baton movement detection used to adjust rhythm, tempo or expressivity of, e.g. the playback of musical pieces
• CPC Class Codes: G10H 2220/371 Vital parameter control, i.e. musical instrument control based on body signals, e.g. brainwaves, pulsation, temperature, perspiration; biometric information signals from body positions or movements G10H2220/321
• CPC Class Codes: G10H 2220/395 Acceleration sensing or accelerometer use, e.g. 3D movement computation by integration of accelerometer data, angle sensing with respect to the vertical, i.e. gravity sensing. conductor baton movement sensing G10H2220/206, angle sensing without reference to gravity G10H2220/391; player body joint angle sensing G10H2220/321
• CPC Class Codes: G10H 2240/211 Wireless transmission, e.g. of music parameters or control data by radio, infrared or ultrasound beam G10H2220/405

Abstract

Performance interface system includes a motion detector provided for movement with a performer, and a control system for receiving detection data transmitted from the motion detector and controlling a performance of a tone in response to the received detection data. State of a performer'"'"'s motion is detected via a sensor of the motion detector, and detection data representative of the detected motion state is transmitted to the control system. The control system receives the detection data from the motion detector, analyzes the performer'"'"'s motion on the basis of the detection data, and then controls a tone performance in accordance with the analyzed data. With this arrangement, the performer can readily take part in the tone performance in the control system. For example, as the performer moves his or her hand, leg or trunk while listening to a manual or automatic performance of a music piece being carried out by a performance apparatus of the control system, the motion detector detects the performer'"'"'s motion and transmits corresponding detection data to the control system, which in turn variably controls a predetermined one of tonal factors in the music piece performance. This arrangement can readily provide interactive performance control and thereby allows an inexperienced or unskilled performer to take part in a performance with enjoyment.

Claims

1. A performance system comprising:

a motion detector held by or attached to a performer, said motion detector detecting an operating state of the motion detector responsive to a motion made by the performer and outputting operation signals of a plurality of components indicative of the operating state;

a living body state detector held by or attached to the performer, said living body state detector detecting a body state of the performer from a body region of the performer and outputting a body signal indicative of the body state;

an identification section that compares the operation signals of the plurality of components and, based on results of the comparison between the operation signals, identifies, from among a plurality of types of motions, a type of motion corresponding to the motion being made by the performer;

an analysis section that analyzes the body state of the performer on the basis of the body signal; and

a generation section that, in accordance with the identified type of motion and the analyzed body state, generates control information to be used for control of a tone performance.

2. The performance system as claimed in claim 1 wherein the body state detected via said living body state detector is at least one of a pulse, heart rate, number of breaths, skin resistance, blood pressure, body temperature, brain wave and eyeball movement.

3. The performance system as claimed in claim 1 which further comprises:

a guide data generation section that generates guide data for providing a guide or assistance as to a motion to be made by the performer; and

a guide mechanism that receives the guide data and performs the guide or assistance as to the motion to be made by the performer in accordance with the received guide data.

4. The performance system as claimed in claim 1 wherein the control of the tone performance controls a tone volume of the tone to be performed.

5. The performance system as claimed in claim 1 wherein the control of the tone performance controls a tempo of the tone to be performed.

6. The performance system as claimed in claim 1 wherein the control of the tone controls performance timing of the tone to be performed.

7. The performance system as claimed in claim 1 wherein the control of the tone performance controls a tone color of the tone to be performed.

8. The performance system as claimed in claim 1 wherein the control of the tone performance controls an effect of the tone to be performed.

9. The performance system as claimed in claim 1 wherein the control of the tone performance controls a tone pitch of the tone to be performed.

10. The performance system as claimed in claim 1 wherein the tone performance is carried out on the basis of automatic performance data.

11. The performance system as claimed in claim 1 wherein said performer is a human being.

12. The performance system as claimed in claim 1 wherein said performer is an animal.

13. A performance system as claimed in claim 1 wherein said motion detector includes a three-axis sensor, and the operation signals contain signals of components of three-axis directions.

14. A performance system as claimed in claim 13 wherein said identification section compares said signals of components of three-axis directions and identifies the type of the motion by determining that said signals of components of three-axis directions meet a predetermined relationship.

15. A performance system as claimed in claim 1 which further comprises a transmitter, and both said living body state detector and said motion detector are provided in said transmitter.

16. A performance system as claimed in claim 1 which further comprises a plurality of transmitters, and said living body state detector and said motion detector are provided in different ones of the transmitters.

17. A method for controlling, by use of a motion detector held by or attached to a performer and a living body state detector held by or attached to the performer, a tone performance in a control system carrying out the tone performance, said motion detector detecting an operating state of the motion detector responsive to a motion made by the performer, said living body state detector detecting a body state of the performer from a body region of the performer, said method comprising the steps of:

receiving operation signals of a plurality of components indicative of the operating state detected by the motion detector;

receiving a body signal indicative of the body state detected by the living body state detector;

comparing the received operation signals of the plurality of components and based on results of the comparison between the received operation signals, identifying, from among a plurality of types of motions, a type of motion corresponding to the motion being made by the performer;

analyzing the body state of the performer on the basis of the received body signal; and

generating control information to be used for control of the tone performance in accordance with the identified type of motion and the analyzed body state.

18. A computer-readable storage medium containing a group of instructions to cause a computer to implement a method for controlling, by use of a motion detector held by or attached to a performer and a living body state detector held by or attached to the performer, a tone performance in a control system carrying out the tone performance, said motion detector detecting an operating state of the motion detector responsive to a motion made by the performer, said living body state detector detecting a body state of the performer from a body region of the performer, said method comprising the steps of:

receiving operation signals of a plurality of components indicative of the operating state detected by the motion detector;

receiving a body signal indicative of the body state detected by the living body state detector;

comparing the received operation signals of the plurality of components and based on results of the comparison between the received operation signals, identifying, from among a plurality of types of motions, a type of motion corresponding to the motion being made by the performer;

analyzing the body state of the performer on the basis of the received body signal; and

generating control information to be used for control of the tone performance in accordance with the identified type of motion and the analyzed body state.