Pattern recognition using a predictive neural network

• US 5,696,877 A
• Filed: 11/30/1993
• Issued: 12/09/1997
• Est. Priority Date: 05/10/1990
• Status: Expired due to Fees

• Current Assignee: NEC Corporation
• Original Assignee: NEC Corporation
• Inventors: Iso, Ken-Ichi
• Primary Examiner(s): Knepper, David D.
• Application Number:
  US08/159,306
• Time in Patent Office: 1,470 Days
• Field of Search: 381/41-43, 395/2, 395/2.11, 395/2.41, 395/2.47, 395/2.5
• US Class Current: 704/241
• CPC Class Codes: G06F 18/00 Pattern recognition
• CPC Class Codes: G06N 3/049 Temporal neural networks, e.g. delay elements, oscillating neurons or pulsed inputs
• CPC Class Codes: G06V 30/194 References adjustable by an adaptive method, e.g. learning
• CPC Class Codes: G10L 15/12 using dynamic programming techniques, e.g. dynamic time warping [DTW]
• CPC Class Codes: G10L 15/16 using artificial neural networks

Abstract

Input feature vectors (a(t)) is considered a pattern selected from a plurality of reference patterns which represent categories of recognition objects. Each reference pattern is defined by a sequence of state models, successively supplied with the time sequence of the input feature vectors and with a sequence of preceding state vectors (h(t, s, n)). The sequence of the state models produces a time sequence of predicted feature vectors (A(t+1, s, n) and a sequence of new state vectors (h(t+1, s, n)). The recognized pattern is selected from one of the reference patterns that minimizes a prediction error between the time sequence of the input feature vectors and the time sequence of the predicted feature vectors. The prediction error is calculated by using a dynamic programming algorithm. Training of the reference pattern is carried out by a gradient descent method such as back-propagation technique.

Claims

1. A method of recognizing a speech sound having a variable time interval as a recognized pattern selected from a plurality of reference patterns which represent categories of recognition objects, respectively, said method comprising the steps of:

converting the speech sound into a digital speech signal;

analyzing the digital speech signal by using a spectral analysis method into a time sequence of input feature vectors;

storing parameters of the reference patterns, each of which is defined by a sequence of predictors, each of the predictors comprising an input layer, a hidden layer, and an output layer, the input layer comprising a plurality of primary input units and a plurality of secondary input units, the hidden layer comprising a plurality of hidden units, the output layer comprising a plurality of output units, the primary input units being connected with the hidden units by a first coefficient matrix, the secondary input units being connected with the hidden units by a second coefficient matrix, the hidden units being connected with the output units by a third coefficient matrix, said parameters corresponding to the first through the third coefficient matrices;

successively deriving a time sequence of predicted feature vectors and a sequence of new state vectors by supplying the time sequence of the input feature vectors and a sequence of preceding state vectors to the primary input units and the secondary input units, respectively, so that feedback occurs through successively supplying the preceding state vectors from said hidden units to the secondary input units to successively produce the new state vectors;

calculating a prediction error between the time sequence of the input feature vectors and the time sequence of the predicted feature vectors; and

selecting, as the recognized pattern, one of the reference patterns that minimizes the prediction error.

2. A speech recognition device for recognizing a speech sound having a variable time interval as a recognized pattern selected from a plurality of reference patterns which represent categories of recognition objects, respectively, said speech recognition device comprising:

a converter for converting the speech sound into a digital speech signal;

an analyzer connected to said converter for processing the digital speech signal, using a spectral analysis method, into a time sequence of input feature vectors;

a storing unit for storing parameters of the reference patterns, each of which is defined by a sequence of predictors, each of the predictors comprising an input layer, a hidden layer, and an output layer, the input layer comprising a plurality of primary input units and a plurality of secondary input units, the hidden layer comprising a plurality of hidden units, the output layer comprising a plurality of output units, the primary input units being connected with the hidden units by a first coefficient matrix, the secondary input units being connected with the hidden units by a secondary coefficient matrix, the hidden units being connected with the output units by a third coefficient matrix, said parameters corresponding to the first through the third coefficient matrices;

a calculator connected to said analyzer and to said storing unit for successively deriving a time sequence of predicted feature vectors and a sequence of new state vectors by supplying the time sequence of the input feature vectors and a sequence of preceding state vectors to the primary input units and the secondary input units, respectively, so that feedback occurs through successively supplying the preceding state vectors from said hidden units to the secondary input units to successively produce the new state vectors, said calculator calculating a prediction error between the time sequence of the input feature vectors and the time sequence of the predicted feature vectors; and

a selector connected to said calculator for selecting, as the recognized pattern, one of the reference patterns that minimizes the prediction error.

3. A speech recognition device as claimed in claim 2, wherein said calculator comprises a plurality of calculation devices which are equal in number to the reference patterns, each of said calculation devices comprising first through N-th calculation units which are operable in parallel, where N represents a predetermined positive integer which is not less than two, each of said first through N-th calculation units comprising:

a predictor for predicting the time sequence of the predicted feature vectors and the sequence of the new state vectors on the basis of the time sequence of the input feature vectors and the sequence of the preceding state vectors,

a distance calculator for calculating a distance between the input feature vector and the predicted feature vector, and

an accumulator for accumulating the distance and a predicting accumulated distance supplied from an (n-1)-th calculation unit to produce an accumulated distance, where n represents a variable between 2 and N, both inclusive,

whereby said accumulator of said N-th calculating unit produces the accumulated distance as the prediction error.

4. A method of recognizing a speech sound, comprising the steps of:

analyzing a digital speech signal using a spectral analysis to produce a time sequence of input feature vectors;

storing parameters of the reference patterns which represents a plurality of speech objects to be recognized, each of said parameters previously defined by a training operation performed on one of a plurality of predictors said predictors having an input layer, a hidden layer, and an output layer;

successively supplying a time delay feedback from an output of said predictors to an input of said predictors by feeding back a sequence of new state vectors derived by said predictors to said input after a time delay so that said input is supplied with a sequence of preceding state vectors output from said predictors, wherein said time delay feedback supplies the preceding state vectors from said hidden layer to said input layer;

successively deriving a time sequence of predicted feature vectors and said sequence of new state vectors from said time sequence of input feature vectors and said sequence of preceding state vectors by using said predictors;

calculating a prediction error between said time sequence of the input feature vectors and said time sequence of the predicted feature vectors; and

selecting, as the recognized pattern, one of the reference patterns that minimizes the prediction error.

5. A speech recognition device comprising:

an analyzer for processing a digital speech signal, using a spectral analysis method, into a time sequence of input feature vectors;

a storing unit for storing parameters of reference patterns, each of said parameters being defined by a training operation performed on said speech recognition device;

a calculator connected to said analyzer and to said storing unit, said calculator comprising;

a plurality of predictors each having a first input, a second input, a first output, and a second output, said first output outputting a time sequence of predicted feature vectors, each of said plurality of predictors having an input layer, a hidden layer, and an output layer connected in series, said output layer outputting said time sequence of predicted feature vectors;

a time delay feedback loop connecting said hidden layer to said input layer so that a sequence of new state vectors output from said hidden layer to the input layer are fed back as a sequence of preceding state vectors while said input layer simultaneously receives said time sequence of the input feature vectors;

a plurality of distance calculators each connected to said first output of one of said predictors for determining a prediction error between the time sequence of the input feature vectors and the time sequence of predicted feature vectors; and

a selector connected to said calculator for selecting, as a recognized pattern, one of said reference patterns that minimizes said prediction error.