Multi-layer neural network to which dynamic programming techniques are applicable

• US 4,975,961 A
• Filed: 10/27/1988
• Issued: 12/04/1990
• Est. Priority Date: 10/28/1987
• Status: Expired due to Term

• Current Assignee: NEC Corporation
• Original Assignee: NEC Corporation
• Inventors: Sakoe, Hiroaki
• Primary Examiner(s): Shaw, Dale M.
• Assistant Examiner(s): Merecki, John A.
• Application Number:
  US07/263,208
• Time in Patent Office: 768 Days
• Field of Search: 381/41-45, 364/513, 364/513.5, 364/200 MS File, 364/900 MS File, 382/14-15, 307/201
• US Class Current: 704/232
• CPC Class Codes: G06N 3/04 Architecture, e.g. interconnection topology
• CPC Class Codes: G06N 3/10 Interfaces, programming languages or software development kits, e.g. for simulating neural networks
• CPC Class Codes: G10L 15/12 using dynamic programming techniques, e.g. dynamic time warping [DTW]
• CPC Class Codes: G10L 15/16 using artificial neural networks

Abstract

In a neural network, input neuron units of an input layer are grouped into first through J-th input layer frames, where J represents a predetermined natural number. Intermediate neuron units of an intermediate layer are grouped into first through J-th intermediate layer frames. An output layer comprises an output neuron unit. Each intermediate neuron unit of a j-th intermediate layer frame is connected to the input neuron units of j'"'"'-th input layer frames, where j is variable between 1 and j and j'"'"' represents at least two consecutive integers, one of which is equal to j and at least one other of which is less than j. Each output neuron unit is connected to the intermediate neuron units of the intermediate layer. For recognition of an input pattern represented by a time sequence of feature vectors, each consisting of K vector components, where K represents a predetermined positive integer, each input layer frame consists of K input neuron units. Each intermediate layer frame consists of M intermediate neuron units, where M represents a positive integer which is less than K. The vector components of each feature vector are supplied to the respective input neuron units of one of the input layer frames that is preferably selected from three consecutively numbered input layer frames. The neural network is readily trained to make a predetermined one of the output neuron units produce an output signal indicative of the input pattern and can be implemented by a microprocessor.

Claims

1. A neural network for recognizing an input pattern represented by a pattern time sequence of feature vectors which are positioned at respective pattern time instants, said neural network comprising:

an input layer of input neuron units grouped into first through J-th input layer frames, where J represents a predetermined natural number;

an intermediate layer of intermediate neuron units grouped into first through J-th intermediate layer frames;

an output layer comprising an output neuron unit assigned to a predetermined pattern;

input-intermediate connections connecting each intermediate neuron unit of a j-th intermediate layer frame to the input neuron units of at least two consecutive input layer frames beginning at the j-th input layer frame and proceeding in a descending order in the direction of (j-1), where j is variable between 1 and J, both inclusive, said input-intermediate connections connecting each intermediate neuron unit of said j-th intermediate layer frame to no input neuron unit when (j-1) is not a positive integer;

intermediate-output connections connecting said output neuron unit to the intermediate neuron units of said first through said J-th intermediate layer frames;

input means for supplying said feature vectors to the input neuron units of said first through said J-th input layer frames with correspondence established between said pattern time instants and said first through said J-th input layer frames; and

adjusting means connected to said input-intermediate and said intermediate-output connections for adjusting said input-intermediate and said intermediate-output connections to make said output neuron unit produce an output signal, said neural network recognizing said input pattern as said predetermined pattern when said adjusting means maximizes said output signal.

2. A neural network for recognizing an input pattern represented by a pattern time sequence of feature vectors which are positioned at respective pattern time instants, said neural network comprising:

an input layer of input neuron units grouped into first through J-th input layer frames, where J represents a predetermined natural number;

an intermediate layer of intermediate neuron units grouped into first through J-th intermediate layer frames;

an output layer comprising output neuron units assigned to respective predetermined patterns;

input-intermediate connections connecting each intermediate neuron unit of a j-th intermediate layer frame to the input neuron units of at least two consecutive input layer frames beginning at the j-th input layer frame and proceeding in a descending order in the direction of (j-1), where j is variable between 1 and J, both inclusive, said input-intermediate connections connecting each intermediate neuron unit of said j-th intermediate layer frame to no input neuron unit when (j-1) is not a positive integer;

intermediate-output connections connecting each output neuron unit to the intermediate neuron units of said first through said J-th intermediate layer frames;

input means for supplying said feature vectors to the input neuron units of said first through said J-th input layer frames with correspondence established between said pattern time instants and said first through said J-th input layer frames; and

adjusting means connected to said input-intermediate and said intermediate-output connections for adjusting said input-intermediate and said intermediate-output connections to make said output neuron units produce respective output signal components with one of said output neuron units made to maximize its output signal component, said neural network recognizing said input pattern as one of said predetermined patterns that is assigned to said one of the output neuron units.

3. A neural network for recognizing an input pattern represented by a pattern time sequence of feature vectors which are positioned at respective pattern time instants, each of said feature vectors consisting of first through K-th vector components, where K represents a first predetermined positive integer, said neural network comprising:

an input layer of input neuron units grouped into first through J-th input layer frames, where J represents a predetermined natural number;

an intermediate layer of intermediate neuron units grouped into first through J-th intermediate layer frames;

an output layer comprising output neuron units assigned to respective predetermined patterns;

input-intermediate connections connecting each intermediate neuron unit of a j-th intermediate layer frame to the input neuron units of at least two consecutive input layer frames beginning at the j-th input layer frame and proceeding in a descending order in the direction of (j-1), where j is variable between 1 and J, both inclusive, said input-intermediate connections connecting each intermediate neuron unit of said j-th intermediate layer frame to no input neuron unit when (j-1) is not a positive integer;

intermediate-output connections connecting each output neuron unit to the intermediate neuron units of said first through said J-th intermediate layer frames;

input means for supplying said feature vectors to the input neuron units of said first through said J-th input layer frames with correspondence established between said pattern time instants and said first through said J-th input layer frames; and

adjusting means connected to said input-intermediate and said intermediate-output connections for adjusting said input-intermediate and said intermediate-output connections to make said output neuron units produce respective output signal components with one of said output neuron units made to maximize its output signal component, said neural network recognizing said input pattern as one of said predetermined patterns that is assigned to said one of the output neuron units;

wherein each of said first through said J-th input layer frames consists of first through K-th input neuron units, each of said first through said J-th intermediate layer frames consisting of first through M-th intermediate neuron units, where M represents a second predetermined positive integer which is less than said first predetermined positive integer;

said input means supplying the first through the k-th vector components of each feature vector to the first through the K-th input neuron units of one of said first through said J-th input layer frames, respectively, with said correspondence established by selecting one of the first through the J-th input layer frames from three input layer frames consisting of the j-th, the (j-1)-th, and the (j-2)-th input layer frames, said one of the first through the J-th input layer frames being said first input layer frame when one of (j-1) and (j-2) is not a positive integer.

4. A neural network for recognizing an input pattern represented by a pattern time sequence of feature vectors which are positioned at respective pattern time instants, each feature vector being first through I-th feature vectors, where I represents a positive integer dependent on said input pattern, each of said first through said I-th feature vectors consisting of first through K-th vector components, where K represents a first predetermined positive integer, said neural network comprising:

an input layer of input neuron units grouped into first through J-th input layer frames, where J represents a predetermined natural number;

an intermediate layer of intermediate neuron units grouped into first through J-th intermediate layer frames;

an output layer comprising output neuron units assigned to respective predetermined patterns;

input-intermediate connections connecting each intermediate neuron unit of a j-th intermediate layer frame to the input neuron units of at least two consecutive input layer frames beginning at the j-th input layer frame and proceeding in a descending order in the direction of (j-1), where j is variable between 1 and J, both inclusive, said input-intermediate connections connecting each intermediate neuron unit of said j-th intermediate layer frame to no input neuron unit when (j-1) is not a positive integer;

intermediate-output connections connecting each output neuron unit to the intermediate neuron units of said first through said J-th intermediate layer frames;

input means for supplying said feature vectors to the input neuron units of said first through said J-th input layer frames with correspondence established between said pattern time instants and said first through said J-th input layer frames; and

adjusting means connected to said input-intermediate and said intermediate-output connections for adjusting said input-intermediate and said intermediate-output connections to make said output neuron units produce respective output signal components with one of said output neuron units made to maximize its output signal component, said neural network recognizing said input pattern as one of said predetermined patterns that is assigned to said one of the output neuron units;

wherein each of said first through said J-th input layer frames consists of first through K-th input neuron units, each of said first through said J-th intermediate layer frames consisting of first through M-th intermediate neuron units, where M represents a second predetermined positive integer which is less than said first predetermined positive integer;

said input means supplying the first through the K-th input neuron units of each input layer frame with the first through the K-th vector components of one of said first through said I-th feature vectors, respectively, with said correspondence established by selecting said one of the first through the I-th feature vectors from three feature vectors consisting of an i-th, and (i-1)-th, and an (i-2)-th feature vector, where i is variable between 1 and I, both inclusive, said one of the first through the I-th feature vectors being said first feature vector when one of (i-1) and (i-2) is not a positive integer.