Apparatus and method for detecting performer's motion to interactively control performance of music or the like

• US 20060185502A1
• Filed: 04/07/2006
• Published: 08/24/2006
• Est. Priority Date: 01/11/2000
• Status: Active Grant

• Current Assignee: Yamaha Corporation
• Original Assignee: Yamaha Corporation
• Inventors: Nishitani, Yoshiki, Sato, Masaki, Usa, Satoshi, Kobayashi, Eiko
• Granted Patent: US 7,781,666 B2
• Time in Patent Office: Days
• US Class Current: 84/615
• CPC Class Codes: A63B 2071/0625 Emitting sound, noise or music
• CPC Class Codes: A63B 2071/0647 Visualisation of executed movements
• CPC Class Codes: A63B 2220/30 Speed
• CPC Class Codes: A63B 2220/34 Angular speed
• CPC Class Codes: A63B 2220/40 Acceleration
• CPC Class Codes: A63B 2220/803 Motion sensors
• CPC Class Codes: A63B 2220/805 Optical or opto-electronic sensors
• CPC Class Codes: A63B 2225/50 Wireless data transmission, e.g. by radio transmitters or telemetry
• CPC Class Codes: A63B 2230/00 Measuring physiological parameters of the user
• CPC Class Codes: A63B 2230/065 within a certain range
• CPC Class Codes: A63B 2230/62 posture
• CPC Class Codes: A63B 69/0028 for running, jogging or speed-walking movable endless bands A63B22/02
• CPC Class Codes: A63B 71/0686 Timers, rhythm indicators or pacing apparatus using electric or electronic means
• CPC Class Codes: G10H 1/00 Details of electrophonic musical instruments
• CPC Class Codes: G10H 2220/135 Musical aspects of games or videogames; Musical instrument-shaped game input interfaces game background music G10H2210/026; musical game scoring or performance evaluation G10H2210/091
• CPC Class Codes: G10H 2220/206 Conductor baton movement detection used to adjust rhythm, tempo or expressivity of, e.g. the playback of musical pieces
• CPC Class Codes: G10H 2220/371 Vital parameter control, i.e. musical instrument control based on body signals, e.g. brainwaves, pulsation, temperature, perspiration; biometric information signals from body positions or movements G10H2220/321
• CPC Class Codes: G10H 2220/395 Acceleration sensing or accelerometer use, e.g. 3D movement computation by integration of accelerometer data, angle sensing with respect to the vertical, i.e. gravity sensing. conductor baton movement sensing G10H2220/206, angle sensing without reference to gravity G10H2220/391; player body joint angle sensing G10H2220/321
• CPC Class Codes: G10H 2240/211 Wireless transmission, e.g. of music parameters or control data by radio, infrared or ultrasound beam G10H2220/405

Abstract

Performance interface system includes a motion detector provided for movement with a performer, and a control system for receiving detection data transmitted from the motion detector and controlling a performance of a tone in response to the received detection data. State of a performer'"'"'s motion is detected via a sensor of the motion detector, and detection data representative of the detected motion state is transmitted to the control system. The control system receives the detection data from the motion detector, analyzes the performer'"'"'s motion on the basis of the detection data, and then controls a tone performance in accordance with the analyzed data. With this arrangement, the performer can readily take part in the tone performance in the control system. For example, as the performer moves his or her hand, leg or trunk while listening to a manual or automatic performance of a music piece being carried out by a performance apparatus of the control system, the motion detector detects the performer'"'"'s motion and transmits corresponding detection data to the control system, which in turn variably controls a predetermined one of tonal factors in the music piece performance. This arrangement can readily provide interactive performance control and thereby allows an inexperienced or unskilled performer to take part in a performance with enjoyment.

Claims

1. A living body state detector comprising:

a sensor adapted to detect a body state of a living thing; and

a transmitter coupled with said sensor and adapted to transmit, to a control system carrying out a tone performance, the body state, detected via said sensor, as body state data to be used for control of the tone performance.

2. A living body state detector as claimed in claim 1 wherein the body state detected via said sensor is at least one of a pulse, heart rate, number of breaths, skin resistance, blood pressure, body temperature, brain wave and eyeball movement.

3. A living body state detector as claimed in claim 1 wherein said living body state detector is held by a hand of the living thing.

4. A living body state detector as claimed in claim 1 wherein said living body state detector is attached to a body of the living thing.

5. A living body state detector as claimed in claim 1 which further comprises:

a motion sensor adapted to detect a state of a motion of the living thing; and

a transmitter coupled with said motion sensor and adapted to transmit detection data representing said state of a motion detected via said motion sensor.

6. A living body state detector as claimed in claim 5 wherein said living body state detector is held by a hand of the living thing.

7. A living body state detector as claimed in claim 5 wherein said living body state detector is attached to a body of the living thing.

8. A living body state detector as claimed in claim 1 which further comprises a receiver adapted to receive guide data for providing a guide or assistance as to a motion to be made by the living thing.

9. A living body state detector as claimed in claim 1 wherein the control of the tone performance controls a tone volume of the tone to be performed.

10. A living body state detector as claimed in claim 1 wherein the control of the tone performance controls a tempo of the tone to be performed.

11. A living body state detector as claimed in claim 1 wherein the control of the tone controls performance timing of the tone to be performed.

12. A living body state detector as claimed in claim 1 wherein the control of the tone performance controls a tone color of the tone to be performed.

13. A living body state detector as claimed in claim 1 wherein the control of the tone performance controls an effect of the tone to be performed.

14. A living body state detector as claimed in claim 1 wherein the control of the tone performance controls a tone pitch of the tone to be performed.

15. A living body state detector as claimed in claim 1 wherein the tone performance is carried out on the basis of automatic performance data.

16. A living body state detector as claimed in claim 1 wherein said living thing is a human being.

17. A living body state detector as claimed in claim 1 wherein said living thing is an animal.

18. A control system comprising:

a receiver adapted to receive body state data transmitted from a living body state detector, the body state data representing a body state of a living thing detected via a sensor that is included in said living body state detector;

a performance apparatus adapted to carry out a performance of a tone on the basis of performance data; and

a controller coupled with said receiver and said performance apparatus and adapted to control said performance of a tone by said performance apparatus in accordance with the body state data received via said receiver.

19. A control system as claimed in claim 18 wherein the body state represented by the body state data is at least one of a pulse, heart rate, number of breaths, skin resistance, blood pressure, body temperature, brain wave and eyeball movement.

20. A control system as claimed in claim 18 wherein said living body state detector is held by a hand of the living thing.

21. A control system as claimed in claim 18 wherein said living body state detector is attached to a body of the living thing.

22. A control system as claimed in claim 18 wherein said receiver is further adapted to receive detection data, the detection data being transmitted from a motion detector provided for movement with the living thing and representing a state of a motion of the living thing, and

wherein said controller is adapted to control said performance of a tone by said performance apparatus, on the basis of the body state data and the detection data.

23. A control system as claimed in claim 22 wherein said living body state detector and said motion detector are held by a hand of the living thing.

24. A control system as claimed in claim 22 wherein said living body state detector and said motion detector are attached to a body of the living thing.

25. A control system as claimed in claim 22 which further comprises a transmitter adapted to transmit, to said motion detector, guide data for providing a guide or assistance as to a motion to be made by the living thing.

26. A control system as claimed in claim 18 wherein control of said performance of a tone by said controller controls a tone volume of the tone to be performed.

27. A control system as claimed in claim 18 wherein control of said performance of a tone by said controller controls a tempo of the tone to be performed.

28. A control system as claimed in claim 18 wherein control of the performance of a tone by said controller controls performance timing of the tone to be performed.

29. A control system as claimed in claim 18 wherein control of the performance of a tone by said controller controls a tone color of the tone to be performed.

30. A control system as claimed in claim 18 wherein control of the performance of a tone by said controller controls an effect of the tone to be performed.

31. A control system as claimed in claim 18 wherein control of the performance of a tone by said controller controls a tone pitch of the tone to be performed.

32. A control system as claimed in claim 18 wherein said performance of a tone is carried out on the basis of automatic performance data.

34. A control system as claimed in claim 18 wherein said living thing is an animal.

33. A control system as claimed as claimed 18 wherein said living thing is a human being.

35. A control system comprising:

a receiver adapted to receive body state data of a plurality of living things transmitted from a plurality of living body state detectors associated with the plurality of living things, each of the body state data representing a body state of one of the living things detected via a sensor that is included in said living body state detector associated with the one living thing;

a performance apparatus adapted to carry out a performance of a tone on the basis of performance data; and

a controller coupled with said receiver and said performance apparatus and adapted to control said performance of a tone by said performance apparatus in accordance with the body state data of the plurality of living things received via said receiver.

36. A control system as claimed in claim 35 wherein the body state represented by the body state data is at least one of a pulse, heart rate, number of breaths, skin resistance, blood pressure, body temperature, brain wave and eyeball movement.

37. A control system as claimed in claim 35 wherein each of said living body state detectors is held by a hand of one of the living things.

38. A control system as claimed in claim 35 wherein each of said living body state detectors is attached to a body of one of the living things.

39. A control system as claimed in claim 35 wherein said receiver is further adapted to receive detection data from a plurality of motion detectors associated with the plurality of living things and provided for movement with corresponding ones of the living things, each of said motion detectors transmitting the detection data representing a state of a motion of the corresponding living thing, and

wherein said controller is adapted to control said performance of a tone by said performance apparatus, on the basis of the body state data and the detection data of the plurality of living things.

40. A control system as claimed in claim 39 wherein each of said living body state detectors and said motion detectors is held by a hand of the corresponding living thing.

41. A control system as claimed in claim 39 wherein each of said living body state detectors and said motion detectors is attached to a body of the corresponding living thing.

42. A control system as claimed in claim 39 which further comprises a transmitter adapted to transmit, to each of said motion detectors, guide data for providing a guide or assistance as to a motion to be made by the living thing.

43. A control system as claimed in claim 35 wherein control of said performance of a tone by said controller controls a tone volume of the tone to be performed.

44. A control system as claimed in claim 35 wherein control of said performance of a tone by said controller controls a tempo of the tone to be performed.

45. A control system as claimed in claim 35 wherein control of the performance of a tone by said controller controls performance timing of the tone to be performed.

46. A control system as claimed in claim 35 wherein control of the performance of a tone by said controller controls a tone color of the tone to be performed.

47. A control system as claimed in claim 35 wherein control of the performance of a tone by said controller controls an effect of the tone to be performed.

48. A control system as claimed in claim 35 wherein control of the performance of a tone by said controller controls a tone pitch of the tone to be performed.

49. A control system as claimed in claim 35 wherein the performance of a tone is carried out on the basis of automatic performance data.

51. A control system as claimed in claim 35 wherein each of said living things is an animal.

50. A control system as claimed as claimed 35 wherein each of said living things is a human being.

52. A control apparatus for controlling readout of time-serial data, said control apparatus comprising:

a storage device adapted to store therein time-serial data of a plurality of data groups;

a data supplier adapted to supply tempo control data for each of the data groups; and

a readout controller coupled with said storage device and said data supplier and adapted to read out the time-serial data of the plurality of data groups from said storage device at a predetermined readout tempo, said readout controller being adapted to control the readout tempo for each of the data groups in accordance with the tempo control data supplied by said data supplier for the data group.

53. A control apparatus as claimed in claim 52 wherein the tempo control data for each of the data groups is stored in said storage device along with the time-serial data for the data group, and wherein said data supplier reads out, from said storage device, the tempo control data for each of the data groups and thereby supplies the tempo control data to said readout controller.

54. A control apparatus as claimed in claim 52 wherein said data supplier generates the tempo control data for each of the data groups on the basis of control data transmitted from a plurality of controllers.

55. A control apparatus as claimed in claim 54 wherein each of said control data represents a state of a motion made by a performer operating a corresponding one of said controllers.

56. A control apparatus as claimed in claim 54 wherein each of said control data represents a body state of a performer operating a corresponding one of said controllers.

57. A control apparatus as claimed in claim 52 wherein the tempo control data for each of the data groups, supplied to said readout controller by said data supplier, is further adapted to be written into said storage device.

58. A control apparatus as claimed in claim 52 wherein said data supplier generates first tempo control data on the basis of control data transmitted from one or more controllers and generates second tempo control data by reading out tempo control data stored in said storage device, and

wherein said readout controller controls the readout tempo of one or some of the time-serial data of the plurality of data groups on the basis of said first tempo control data and controls the readout tempo of other one or some of the time-serial data of the plurality of data groups on the basis of said second tempo control data.

59. A control apparatus as claimed in claim 52 wherein said data supplier is further adapted to generate modification data on the basis of control data transmitted from a controller and modify the tempo control data for each of the data groups on the basis of the modification data, and

wherein said readout controller controls the readout tempo for each of the data groups on the basis of the tempo control data for each of the data groups modified on the basis of the modification data.

60. A control apparatus as claimed in claim 52 wherein said storage device is further adapted to store therein display data corresponding to the plurality of data groups, and

wherein said readout controller is further adapted to read out the display data from said storage device on the basis of the tempo control data for each of the data groups supplied by said data supplier and cause a display device to display a visual image based on the display data read out from said storage device.

61. A control apparatus as claimed in claim 52 wherein said time-serial data are performance data.

62. A control apparatus as claimed in claim 52 wherein said time-serial data are image data.

63. A light-emitting toy comprising:

a sensor provided for movement with a motion of a performer to detect a state of the motion of the performer;

a light-emitting device; and

a controller coupled with said sensor and said light-emitting device and adapted to control a style of light emission of said light-emitting device on the basis of the state of the motion detected via said sensor.

64. A light-emitting toy as claimed in claim 63 wherein a plurality of the sensors are provided in corresponding relation to a plurality of axes so that the state of the motion for each of the axes may be detected via a different one of said sensors, and

wherein said controller controls the style of light emission of said light-emitting device on the basis of the state of the motion for each of the axes detected via said sensor.

65. A light-emitting toy as claimed in claim 63 which further comprises a body state detector for detecting a body state of the performer.

66. A light-emitting toy as claimed in claim 65 wherein said controller is adapted to control the style of light emission of said light-emitting device in accordance with the body state detected via said body state detector.

67. A light-emitting toy as claimed in claim 65 which further comprises a storage device, and wherein said controller is further adapted to store, into said storage device, the body state detected via said body state detector.

68. A light-emitting toy as claimed in claim 67 wherein said controller is further adapted to store, into said storage device, the state of the motion of the performer detected via said sensor.

69. A light-emitting toy as claimed in claim 63 which further comprises a receiver coupled with said controller and adapted to receive data transmitted from outside said light-emitting toy, and wherein said controller is further adapted to control the style of light emission of said light-emitting device on the basis of the data received via said receiver.

70. A method for controlling, by use of a living body state detector, a tone performance in a control system carrying out the tone performance, said method comprising the steps of:

detecting a body state of a living thing by use of said living body state detector; and

transmitting, to the control system carrying out the tone performance, the body state, detected via said step of detecting, as body state data to be used for control of the tone performance.

71. A method for controlling a tone performance by use of a living body state detector for detecting a body state of a living thing, said method comprising the steps of:

receiving body state data transmitted from said living body state detector, the body state data representing a body state of a living thing detected via said living body state detector;

carrying out a performance of a tone on the basis of performance data; and

controlling said performance of a tone by said step of carrying out, in accordance with the body state data received via said step of receiving.

72. A method of controlling a tone performance by use of a living body state detector for detecting a body state of a living thing, said method comprising the steps of:

receiving body state data of a plurality of living things transmitted from a plurality of the living body state detectors associated with the plurality of living things, each of the body state data representing a body state of one of the living things detected via a sensor that is included in said living body state detector associated with the one living thing;

carrying out a performance of a tone on the basis of performance data; and

controlling said performance of a tone by said step of carrying out, in accordance with the body state data of the plurality of living things received via said step of receiving.

73. A method for controlling readout of time-serial data of a plurality of data groups stored in a storage device, said method comprising the steps of:

supplying tempo control data for each of the data groups; and

reading out the time-serial data of the plurality of data groups from said storage device at a predetermined readout tempo, said step of reading out controlling the readout tempo for each of the data groups in accordance with the tempo control data supplied via said step of supplying for the data group.

74. A method for controlling light emission of a light-emitting device, said method comprising the steps of:

detecting a state of a motion of a performer by use of a sensor; and

controlling a style of light emission of said light-emitting device on the basis of the state of the motion detected via said step of detecting.

75. A machine-readable storage medium containing a group of instructions to cause said machine to implement a method for controlling, by use of a living body state detector, a tone performance in a control system carrying out the tone performance, said method comprising the steps of:

detecting a body state of a living thing by use of said living body state detector; and

transmitting, to the control system carrying out the tone performance, the body state, detected via said step of detecting, as body state data to be used for control of the tone performance.

76. A machine-readable storage medium containing a group of instructions to cause said machine to implement a method for controlling a tone performance by use of a living body state detector for detecting a body state of a living thing, said method comprising the steps of:

receiving body state data transmitted from said living body state detector, the body state data representing a body state of a living thing detected via said living body state detector;

carrying out a performance of a tone on the basis of performance data; and

controlling said performance of a tone by said step of carrying out, in accordance with the body state data received via said step of receiving.

77. A machine-readable storage medium containing a group of instructions to cause said machine to implement a method of controlling a tone performance by use of a living body state detector for detecting a body state of a living thing, said method comprising the steps of:

receiving body state data of a plurality of living things transmitted from a plurality of the living body state detectors associated with the plurality of living things, each of the body state data representing a body state of one of the living things detected via a sensor that is included in said living body state detector associated with the one living thing;

carrying out a performance of a tone on the basis of performance data; and

controlling said performance of a tone by said step of carrying out, in accordance with the body state data of the plurality of living things received via said step of receiving.

78. A machine-readable storage medium containing a group of instructions to cause said machine to implement a method for controlling readout of time-serial data of a plurality of data groups stored in a storage device, said method comprising the steps of:

supplying tempo control data for each of the data groups; and

reading out the time-serial data of the plurality of data groups from said storage device at a predetermined readout tempo, said step of reading out controlling the readout tempo for each of the data groups in accordance with the tempo control data supplied via said step of supplying for the data group.

79. A machine-readable storage medium containing a group of instructions to cause said machine to implement a method for controlling light emission of a light-emitting device, said method comprising the steps of:

detecting a state of a motion of a performer by use of a sensor; and

controlling a style of light emission of said light-emitting device on the basis of the state of the motion detected via said step of detecting.

80. A signal to be transmitted comprising:

time-serial data of a plurality of data groups; and

tempo control data for controlling a reproduction tempo of the time-serial data for each of the data groups.

81. A signal to be transmitted as claimed in claim 80 wherein the time-serial data are performance data.

82. A signal to be transmitted as claimed in claim 80 wherein the time-serial data are image data.