Motion discrimination method and device using a hidden markov model

• US 5,808,219 A
• Filed: 11/01/1996
• Issued: 09/15/1998
• Est. Priority Date: 11/02/1995
• Status: Expired due to Term

• Current Assignee: Yamaha Corporation
• Original Assignee: Yamaha Corporation
• Inventors: Usa, Satoshi
• Primary Examiner(s): Shoop, Jr., William M.
• Assistant Examiner(s): DONELS, JEFFREY
• Application Number:
  US08/742,346
• Time in Patent Office: 683 Days
• Field of Search: 84/477 B, 84/600, 84/723, 382/103, 382/155, 382/228, 395/93, 395/97
• US Class Current: 84/600
• CPC Class Codes: G10H 1/00 Details of electrophonic musical instruments
• CPC Class Codes: G10H 2220/206 Conductor baton movement detection used to adjust rhythm, tempo or expressivity of, e.g. the playback of musical pieces
• CPC Class Codes: G10H 2250/015 Markov chains, e.g. hidden Markov models [HMM], for musical processing, e.g. musical analysis or musical composition
• CPC Class Codes: G10H 2250/151 Fuzzy logic
• CPC Class Codes: G10H 2250/311 Neural networks for electrophonic musical instruments or musical processing, e.g. for musical recognition or control, automatic composition or improvisation musical analysis G10H2210/031; neural networks G06N3/02

Abstract

A motion discrimination method or a motion discrimination device is provided to discriminate a kind of a motion, i.e., one of conducting operations which are made by a human operator by swinging a baton to conduct music of a certain time (e.g., quadruple time). Herein, sensors are provided to detect the motion, made by the human operator, to produce detection values. The detection values are converted to operation labels, which are assembled together in a certain time unit (e.g., 10 ms) to form label series. In addition, there are provided a plurality of Hidden Markov Models, each of which is constructed to learn label series corresponding to a specific motion in advance. Calculations are performed to produce probabilities that multiple Hidden Markov Models respectively output the label series corresponding to the detected motion. Then, a kind of the motion is discriminated on the basis of result of the calculations. Further, a beat label representing the discriminated kind of the motion is inserted into the label series. Herein, the discrimination is made only when a highest one of the probabilities exceeds a certain threshold value so that designation of a beat is detected. Incidentally, the discriminated kind of the motion is used as a detected beat, designated by the human operator, by which a tempo of automatic performance is controlled.

Claims

1. A motion discrimination method comprising the steps of:

detecting a motion by a sensor to produce detection values;

converting the detection values to labels by a certain time unit so as to create label series corresponding to the detected motion;

performing calculations to produce a probability that at least one of Hidden Markov Models outputs the label series corresponding to the detected motion, wherein each of the Hidden Markov Models is constructed to learn specific label series regarding a specific motion; and

discriminating a kind of the detected motion, detected by the sensor, on the basis of result of the calculations.

2. A motion discrimination method according to claim 1 further comprising the steps of:

producing a specific label based on the discriminated kind of the motion; and

inserting the specific label into the label series.

3. A motion discrimination method comprising the steps of:

detecting a motion made by a human operator to produce detection values;

creating labels based on the detection values, so that the labels are assembled together by a unit time to form label series corresponding to the detected motion;

providing a plurality of Hidden Markov Models each of which is constructed to learn specific label series regarding a specific motion;

performing calculations to produce a probability that at least one of the plurality of Hidden Markov Models outputs the label series corresponding to the detected motion; and

discriminating a kind of the detected motion based on result of the calculations.

4. A motion discrimination method according to claim 3 wherein the motion corresponds to one of a series of conducting operations which are made by a human operator to swing a baton to conduct music of a certain time, so that the label series consists of operation labels.

5. A motion discrimination method according to claim 3 wherein the motion corresponds to one of a series of conducting operations which are made by a human operator to swing a baton to conduct music of a certain time, so that the label series is constructed by operation labels accompanied with a beat label representing the discriminated kind of the motion.

6. A motion discrimination method according to claim 3 wherein the calculations are performed to produce probabilities that multiple Hidden Markov Models respectively output the label series corresponding to the detected motion, so that the kind of the detected motion is discriminated as a motion corresponding to a Hidden Markov Model having a highest one of the probabilities within the multiple Hidden Markov Models only when the highest one of the probabilities exceeds a certain threshold value.

7. A motion discrimination device comprising:

sensor means for detecting a motion to produce detection values;

labeling means for converting the detection values to labels by a certain time unit;

label-series creating means for creating label series consisting of the labels which are outputted from the labeling means by the certain time unit;

Hidden-Markov-Model storage means for storing a plurality of Hidden Markov Models each of which is constructed to learn specific label series corresponding to a specific motion;

calculation means for performing calculations to obtain a probability that at least one of Hidden Markov Models outputs the label series; and

discrimination means for discriminating a kind of the detected motion, detected by the sensor means, on the basis of result of the calculations.

8. A motion discrimination device according to claim 7 wherein the label-series creating means is constructed such that a specific label, representing the discriminated kind of the motion by the discrimination means, is inserted into the label series.

9. A motion discrimination device comprising:

sensor means for detecting a motion made by a human operator to produce detection values;

labeling means for creating labels based on the detection values;

label-series creating means for creating label series corresponding to the detected motion, wherein the label series contains the labels which are supplied thereto from the labeling means by a time unit which is determined in advance;

a plurality of Hidden Markov Models, each of which is constructed to learn specific label series corresponding to a specific motion;

probability calculating means for performing calculations to produce a probability that at least one of the plurality of Hidden Markov Models outputs the label series corresponding to the detected motion; and

discrimination means for discriminating a kind of the detected motion based on result of the calculations.

10. A motion discrimination device according to claim 9 wherein the motion corresponds to one of a series of conducting operations which are made by the human operator to swing a baton to conduct music of a certain time, so that the label series consists of operation labels.

11. A motion discrimination device according to claim 9 wherein the motion corresponds to one of a series of conducting operations which are made by the human operator to swing a baton to conduct music of a certain time, so that the label series is constructed by operation labels accompanied with a beat label representing the discriminated kind of the motion.

12. A motion discrimination device according to claim 9 wherein the calculations are performed to produce probabilities that multiple Hidden Markov Models output the label series corresponding to the detected motion, so that the kind of the detected motion is discriminated as a motion corresponding to a Hidden Markov Model having a highest one of the probabilities within the multiple Hidden Markov Models only when the highest one of the probabilities exceeds a certain threshold value.

13. A motion discrimination device according to claim 9 wherein the motion corresponds to one of a series of conducting operations which are made by the human operator to swing a baton to conduct music of a certain time, so that the label-series creating means is constructed by first storage means to store operation labels and second storage means to store a beat label representing the discriminated kind of the motion.

14. A motion discrimination device according to claim 9 wherein each of the plurality of Hidden Markov Models is realized by a plurality of state transitions, each of which occurs from one state to another with a probability.

15. A motion discrimination device according to claim 9 wherein each of the plurality of Hidden Markov Models is realized by a plurality of state transitions, each of which occurs from one state to another with a probability, as well as at least one self state transition in which a system remains at a same state with a probability.

16. A motion discrimination device according to claim 9 wherein each of the plurality of Hidden Markov Models is constructed to learn one of beats of the certain time.

17. A storage device storing programs and data which cause an electronic apparatus to execute a motion discrimination method comprising the steps of:

detecting a motion made by a human operator to produce detection values;

creating labels based on the detection values, so that the labels are assembled together by a unit time to form label series corresponding to the detected motion;

providing a plurality of Hidden Markov Models each of which is constructed to learn specific label series regarding a specific motion;

performing calculations to produce a probability that at least one of the plurality of Hidden Markov Models outputs the label series corresponding to the detected motion; and

discriminating a kind of detected motion based on result of the calculations.

18. A storage device according to claim 17 wherein the motion corresponds to one of a series of conducting operations which are made by a human operator to swing a baton to conduct music of a certain time, so that the label series consists of operation labels.

19. A storage device according to claim 17 wherein the motion corresponds to one of a series of conducting operations which are made by a human operator to swing a baton to conduct music of a certain time, so that the label series is constructed by operation labels accompanied with a beat label representing the discriminated kind of the motion.

20. A storage device according to claim 17 wherein the calculations are performed to produce probabilities that multiple Hidden Markov Models respectively output the label series corresponding to the detected motion, so that the kind of the detected motion is discriminated as a motion corresponding to a Hidden Markov Model having a highest one of the probabilities within the multiple Hidden Markov Models only when the highest one of the probabilities exceeds a certain threshold value.

21. A machine-readable medium storing program instructions for controlling a machine to perform a method including a plurality of steps,

creating a label series comprising labels which are created by detecting a specific motion made by a human operator;

and

performing a plurality of calculations corresponding to each of a plurality of Hidden Markov Models to determine the most appropriate Hidden Markov Model to represent the label series. wherein each of the Hidden Markov Models is represented by a series of state transitions which occur among a series of states with associated probabilities.

22. A storage medium according to claim 21 wherein the labels are created by detecting a specific motion which corresponds to beats of a certain time of music.