Pattern representation model training apparatus

• US 5,289,562 A
• Filed: 03/21/1991
• Issued: 02/22/1994
• Est. Priority Date: 09/13/1990
• Status: Expired due to Term

• Current Assignee: Mitsubishi Denki Kabushiki Kaisha (Mitsubishi Electric Corporation)
• Original Assignee: Mitsubishi Denki Kabushiki Kaisha (Mitsubishi Electric Corporation)
• Inventors: Nakajima, Kunio, Mizuta, Shinobu
• Primary Examiner(s): Fleming, Michael R.
• Assistant Examiner(s): Doerrler, Michelle
• Application Number:
  US07/674,069
• Time in Patent Office: 1,069 Days
• Field of Search: 381/41-48, 395/2, 358/212
• US Class Current: 704/200
• CPC Class Codes: G06F 18/295 Markov models or related models, e.g. semi-Markov models; Markov random fields; Networks embedding Markov models
• CPC Class Codes: G10L 15/144 Training of HMMs

Abstract

Disclosed is an Hidden Markov Model (HMM) training apparatus in which a capacity for discriminating between models is taken into consideration so as to allow a high level of recognition accuracy to be obtained. A probability of a vector sequence appearing from HMMs is computed with respect to an input vector and continuous mixture density HMMs. Through this computation, the nearest different-category HMM, with which the maximum probability is obtained and which belongs to a category different from that of a training vector sequence of a known category, is selected. The respective central vectors of continuous densities constituting the output probability densities of the same-category HMM belonging to the same category as that of the training vector sequence and the nearest different-category HMM are moved on the basis of the vector sequence.

Claims

1. An apparatus for training a pattern representation model for classifying and discriminating an input acoustic feature vector sequence, comprised of at least one vector, into one of a plurality of categories, the model including a hidden Markov model for each category having output probability densities defined by a mixture of continuous densities and having a central vector representing the means of the continuous densities, the apparatus comprising:

means for comparing a training acoustic feature vector sequence of a known category to the hidden Markov models to compute a probability for each of the categories;

means for selecting the hidden Markov model, of a category other than the known category, which provided the maximum probability in response to the training acoustic feature vector sequence; and

vector control means for moving, on the basis of the training acoustic feature vector sequence, the central vectors of the selected hidden Markov model and of the hidden Markov model of the known category.

2. The apparatus of claim 1, wherein the vector control means comprises:

means for correlating the vectors constituting the training acoustic feature vector sequence with the plurality of output probability densities of the plurality of hidden Markov models;

means for selecting, from among the continuous densities of the correlated output probability densities, the continuous density which maximizes a partial probability with respect to the vectors; and

means for moving, on the basis of the vectors constituting the training acoustic feature vector sequence, the central vector of the selected continuous density in the direction of the vectors when the selected continuous density belongs to the known category and in the opposite direction when the selected continuous density belongs to a category other than the known category.

3. The apparatus of claim 2, wherein the means for correlating includes means for applying a Viterbi algorithm.

4. The apparatus of claim 1, wherein said means for comparing includes means for applying a Viterbi algorithm.

5. The apparatus of claim 1, further comprising means for repeatedly applying the training acoustic feature vector sequence so as to train parameters of the hidden Markov models sequentially.

6. The apparatus of claim 1, further comprising speech analysis means for analyzing speech to transform the speech into a time sequence of acoustic feature vectors, said speech analysis means being used to provide a training acoustic feature vector sequence to train the hidden Markov models of the pattern recognition model to be used as a speech recognition apparatus.

7. An apparatus for training a pattern representation model for classifying a vector sequence of physical data into one of a plurality of categories, the model including a hidden Markov model for each category having output probability densities defined by a mixture of continuous densities and having a central vector representing the means of the continuous densities, the apparatus comprising:

means for comparing a training vector sequence of a known category to continuous mixture density hidden Markov models to compute a probability for each of the categories;

means operative in response to the means for comparing for selecting the hidden Markov model, of a category other than the known category, which provided the maximum probability in response to the training vector sequence; and

means responsive to a selection by the means for selecting for moving, on the basis of the training vector sequence, the central vectors of the selected hidden Markov model and of the hidden Markov model of the known category.

8. The apparatus of claim 7, wherein the vector sequence of physical data is an acoustic feature vector sequence obtained through analysis of an audio signal.

9. The apparatus of claim 7, wherein the vector sequence of physical data is a vector sequence of image data.

10. A method for training a continuous density hidden Markov model representation of patterns of physical data, comprising the steps of:

preparing a plurality of hidden Markov models, each hidden Markov model corresponding to a category and having a central vector indicative of the means of the continuous mixture density;

computing a probability corresponding to an input pattern of a known category for each prepared hidden Markov model;

selecting the category of the prepared hidden Markov model which has the maximum probability corresponding to the input pattern;

moving, when the selected category is not the known category, the central vectors of the hidden Markov models of the known category and of the selected category prepared hidden Markov model.

11. The method of claim 10, wherein the vector sequence of physical data is an acoustic feature vector sequence obtained through analysis of an audio signal.

12. The method of claim 10, wherein the vector sequence of physical data is a vector sequence of image data.

13. An apparatus for training a pattern representation model comprised of a plurality of hidden Markov models with continuous mixture densities, each hidden Markov model representing a separate category of pattern of physical data and having a central vector indicative of the means of the continuous mixture density, the apparatus comprising:

first means for comparing a training pattern of a known category to the plurality of hidden Markov models to obtain a probability for each hidden Markov model;

means for selecting each hidden Markov model, of a category other than the known category, which provided in response to the training pattern a probability within a predetermined distance of the probability provided by the hidden Markov model of the known category;

second means, operative in response to a selection by the means for selecting, for comparing the maximum probability provided by the selected hidden Markov model to the probability provided by the hidden Markov model of the known category; and

means, operative in response to the comparison by the second means for comparing, for moving the central vectors of the hidden Markov model of the known category and of the selected hidden Markov model.

14. An method for training a pattern representation model for classifying and discriminating an acoustic feature vector sequence into one of a plurality of categories, the model including a hidden Markov model for each category having output probability densities defined by a mixture of continuous densities and having a central vector representing the means of the continuous densities, the method comprising the steps of:

comparing a training acoustic feature vector sequence of a known category to the continuous mixture density hidden Markov models of compute a probability for each of the categories;

selecting the hidden Markov model, of a category other than the known category, which provided the maximum probability in response to the training acoustic feature vector sequence; and

moving, on the basis of the training acoustic feature vector sequence, the central vectors of the selected hidden Markov model and of the hidden Markov model of the known category.

15. The method of claim 14, wherein the step of moving comprises the steps of:

correlating the vectors constituting said training acoustic feature vector sequence with the plurality of output probability densities of the plurality of hidden Markov models;

selecting, from among the continuous densities of the correlated output probability densities, the continuous density which maximizes a partial probability with respect to the vectors; and

moving, on the basis of the vectors constituting the training input acoustic feature vector sequence, the central vector of the selected continuous density in the direction of the vectors when the selected continuous density belongs to the known category and in the opposite direction when the selected continuous density belongs to a category other than the known category.

16. The method of claim 15, wherein the step of correlating includes the step of applying a Viterbi algorithm.

17. The method of claim 14, wherein the step of comparing includes the step of applying a Viterbi algorithm.

18. The method of claim 14, further comprising the step of repeatedly applying the training acoustic feature vector sequence so as to train parameters of the hidden Markov models sequentially.

19. The method of claim 14, further comprising the step of analyzing speech to transform the speech into a time sequence of acoustic feature vectors, the step of analyzing providing a training acoustic feature vector sequence to train the hidden Markov models of the pattern recognition model to be used as a speech recognition apparatus.

20. A method for training a pattern representation model comprised of a plurality of hidden Markov models with continuous mixture densities, each hidden Markov model representing a separate category of pattern of physical data and having a central vector indicative of the means of the continuous mixture density, the method comprising:

(a) comparing a training pattern of a known category to the plurality of hidden Markov models to obtain a probability for each hidden Markov model;

(b) selecting each hidden Markov model, of a category other than the known category, which provided in response to the training pattern a probability within a predetermined distance of the probability provided by the hidden Markov model of the known category;

(c) comparing the maximum probability provided by the selected hidden Markov model to the probability provided by the hidden Markov model of the known category; and

(d) moving the central vectors of the hidden Markov model of the known category and of the selected hidden Markov model on the basis of the step (c) of comparing.