Method and apparatus for signal classification using a multilayer network

• US 6,208,963 B1
• Filed: 06/24/1998
• Issued: 03/27/2001
• Est. Priority Date: 06/24/1998
• Status: Expired due to Term

• Current Assignee: Fonix Corp.
• Original Assignee: Fonix Corp.
• Inventors: Hansen, Carl Hal, Wilson, D. Randall, Parr, Randall J., Moncur, R. Brian, Martinez, Tony R., Shepherd, D. Lynn
• Primary Examiner(s): {haeck over (S)}mits, Ta̅livaldis I.
• Application Number:
  US09/103,569
• Time in Patent Office: 1,007 Days
• Field of Search: 704/232, 704/270
• US Class Current: 704/232
• CPC Class Codes: G06F 18/241 relating to the classification model, e.g. parametric or non-parametric approaches
• CPC Class Codes: G06N 3/04 Architecture, e.g. interconnection topology
• CPC Class Codes: G10L 15/16 using artificial neural networks

Abstract

A method and apparatus for signal classification using a multilayer temporal relaxation network involves receiving an input signal feature vector, classifying a first signal feature, and classifying a second signal feature using contextual information. The multilayer temporal relaxation network applies a relaxation process that updates an activation value of a node in a first layer and updates an activation value of a node in a second layer. The multilayer network then generates a signal classification according to an activation value of a node in the multilayer network.

Claims

1. A method for classifying a signal using a multilayer temporal relaxation network comprising the steps of:

receiving an input signal feature vector in a first layer;

applying a multilayer relaxation process that updates an activation value of a node in the multilayer network for a current time step;

providing interaction between multilayer relaxation networks, wherein each multilayer relaxation network represents a step in time;

determining whether there is a candidate classification for the current time step in a second layer;

outputting the candidate classification for the current time step; and

incrementing the current time step.

2. The method of claim 1, wherein the step of applying a multilayer relaxation process includes the substep of computing an activation value according to an activation function f(t), where f(t)=k₀(t+1)−

1, when t<

−

1;

−

k₁L((−

t/k₂)−

2)+k₃, when −

1≦

t<

0;

0 when t=0;

k₁L((t/k₂)−

2)−

k₃, when 0<

t≦

1; and

k₀(t−

1)+1, when t>

1;

wherein operator L(t)=1/(1+e<sup>−

2t</sup>) and k₀, k₁, k₂, and k₃ are scaling constants.

3. The method of claim 1, wherein the step of applying a multilayer relaxation process includes the substep of computing an activation value according to an activation function representing scaled and translated logistic functions combined to produce a flat response near the middle of the activation function domain and a linear response in a clamping region.

4. The method of claim 1, wherein the step of applying a multilayer relaxation process includes the substep of:

updating said activation value through a controlled sequence of partial updates.

5. A method for classifying a signal representing spoken sounds using a multilayer temporal relaxation network comprising the steps of:

receiving in a first layer, having a plurality of interconnected phoneme nodes with associated activation values, an input signal feature vector representing a phoneme included in said spoken sounds;

applying a multilayer relaxation process for a current time step, wherein the multilayer relaxation process updates selected ones of the activation values for the phoneme nodes and updates selected ones of activation values associated with a plurality of word nodes in a second layer;

providing interaction between multilayer relaxation networks, wherein each multilayer relaxation network represents a step in time; and

outputting a word in the current time step.

6. The method of claim 5, wherein the step of applying a multilayer relaxation process further includes the substep of:

updating in a third layer, having a plurality of interconnected grammar nodes with associated activation values, selected ones of the grammar node activation values.

7. The method of claim 6, wherein the step of updating a grammar node activation value includes the substeps of:

summing grammar node inputs from competing nodes in said third layer;

summing grammar node inputs from sequential word nodes in said second layer;

summing grammar node inputs from grammar nodes in a preceding time step;

summing grammar node inputs from grammar nodes in a following time step;

summing grammar node inputs from word nodes; and

computing an activation value for a grammar node.

8. The method of claim 7, further comprising the step of:

modifying an active grammar list according to the computed grammar node activation value.

9. The method of claim 5, wherein the step of applying a multilayer relaxation process includes the substeps of:

summing phoneme node inputs from competing nodes within said first layer;

updating selected ones of the phoneme activation values based on an input from a phoneme node in a preceding time step;

updating selected ones of the phoneme activation values based on an input from a phoneme node in a following time step;

summing phoneme node inputs from nodes in said second layer corresponding to active words; and

computing an activation value of a phoneme node.

10. The method of claim 5, wherein the step of applying a multilayer relaxation process that updates the phoneme node activation value and updates the word node activation value includes the substeps of:

summing word node inputs from competing word nodes within said second layer;

summing word node inputs from sequential word nodes within said second layer;

summing word node inputs from word nodes in a preceding time step;

summing word node inputs from word nodes in a following time step;

summing word node inputs from phoneme match nodes; and

computing an activation value for a word node.

11. The method of claim 10, further including the step of summing word node inputs from a grammar node in a third layer.

12. The method of claim 5, wherein the step of outputting the candidate word further includes the substep of

outputting the candidate word from an active word list.

13. The method of claim 5, wherein the step of applying a multilayer relaxation process includes the substep of:

updating said selected ones of the activation values through a controlled sequence of partial updates.

14. A method for classifying an image of handwritten text using a multilayer temporal relaxation network comprising the steps of:

receiving in a first layer an input image feature vector;

applying a multilayer relaxation process that updates an activation value of a node in the multilayer network for a current spatial scan interval;

determining whether there is a candidate classification of the image of handwritten text for the current spatial scan interval in a second layer;

providing interaction between multilayer relaxation networks, wherein each multilayer relaxation network represents a step in time;

outputting the candidate classification for the current spatial scan interval; and

incrementing the current spatial scan interval.

15. A method for forecasting weather using a multilayer temporal relaxation network comprising the steps of:

receiving in a first layer an input reading corresponding to a weather variable;

updating an activation value of a node in the multilayer network for a current time or spatial interval using multilayer relaxation;

providing interaction between multilayer relaxation networks, wherein each multilayer relaxation network represents a spatial scan interval;

determining whether there is a candidate weather forecast for the current time or spatial interval in a second layer;

outputting the candidate weather forecast for the current time or spatial interval; and

incrementing the current time or spatial interval.

16. A method for identifying an image using a multilayer temporal relaxation network comprising the steps of:

receiving in a first layer an input pixel from said image;

updating an activation value of a node in the multilayer network for a current spatial scan interval using multilayer relaxation;

providing interaction between multilayer relaxation networks, wherein each multilayer relaxation network represents a step in time;

determining whether there is a candidate identification for the current spatial scan interval in a second layer;

outputting the candidate identification for the current spatial scan interval; and

incrementing the current spatial scan interval.

17. A method for controlling a process using a multilayer temporal relaxation network comprising the steps of:

receiving in a first layer an input reading corresponding to a process control parameter;

updating an activation value of a node in the multilayer network for a current time interval using multilayer relaxation;

determining whether there is a candidate control level for the current time interval in a second layer;

providing interaction between multilayer relaxation networks, wherein each multilayer relaxation network represents a step in time;

outputting the candidate control level for the current time interval; and

incrementing the current time interval.

18. A method for classifying a signal using a multilayer relaxation network comprising the steps of:

receiving an input signal feature vector in a first layer;

applying a relaxation process that updates an activation value of a node in the multilayer network for a current time step, wherein the step of applying a relaxation process includes the substep of computing an activation value according to an activation function f(t), where f(t)=k₀(t+1)−

1, when t<

−

1;

−

k₁L((−

t/k₂)−

2)+k₃, when −

1≦

t<

0;

0 when t=0;

k₁L((t/k₂)−

2)−

k₃, when 0<

t≦

1; and

k₀(t−

1)+1, when t>

1;

wherein operator L(t)=1/(1+e<sup>−

2t</sup>) and k₀, k₁, k₂, and k₃ are scaling constants;

determining whether there is a candidate classification for the current time step in a second layer;

outputting the candidate classification for the current time step; and

incrementing the current time step.

19. A method for classifying a signal using a multilayer relaxation network comprising the steps of:

receiving an input signal feature vector in a first layer;

applying a relaxation process that updates an activation value of a node in the multilayer network for a current time step, wherein the step of applying a relaxation process includes the substep of computing an activation value according to an activation function representing scaled and translated logistic functions combined to produce a flat response near the middle of the activation function domain and a linear response in a clamping region;

determining whether there is a candidate classification for the current time step in a second layer;

outputting the candidate classification for the current time step; and

incrementing the current time step.

20. A multilayer temporal relaxation network apparatus for signal classification comprising:

an input layer, having a first plurality of nodes with associated activation values, for receiving an input signal feature vector representing an input signal and for classifying at least part of said input signal according to a first signal feature;

a context layer, having a second plurality of nodes with associated activation values, for classifying at least part of said input signal according to a second signal feature, the second signal feature representing a context of said input signal;

means for providing interaction between said input layer and said context layer to update selected ones of the activation values for the nodes in said input layer and said context layer during classification, wherein said means for providing interaction includes relaxation means for updating said selected ones of the activation values through a controlled sequence of partial updates;

means for providing interaction between multilayer relaxation networks, wherein each multilayer relaxation network represents a step in time; and

means for generating a signal classification for said input signal from said input layer or said second context layer.

21. A multilayer temporal relaxation network apparatus for classifying a signal representing spoken sounds comprising:

a first layer, having a plurality of phoneme nodes with associated activation values, for receiving an input signal feature vector representing an input signal associated with the spoken sounds and for classifying at least a portion of said spoken sounds as one of a plurality of possible phonemes;

a second layer, having a plurality of word nodes with associated activation values, for classifying said spoken sounds as one of a plurality of possible words;

means for providing multilayer relaxation interaction between said first and second layers to update selected ones of the activation values for phoneme nodes and word nodes;

means for providing interaction between multilayer relaxation networks, wherein each multilayer relaxation network represents a step in time; and

means for classifying said spoken sounds according to selected ones of the activation values in said first layer or according to selected ones of the activation values in said second layer.

22. The apparatus of claim 21, wherein said first layer further includes

a phoneme node for each of the possible phonemes;

wherein said second layer further includes a word node for each word in a vocabulary; and

wherein said means for providing multilayer relaxation interaction further includes interconnections among phoneme nodes and word nodes.

23. The apparatus of claim 21, wherein said phoneme nodes and said word nodes include means for receiving inhibition input, activation input, and a bias value for controlling an activation value for each phoneme node and word node.

24. The apparatus of claim 23, further comprising an intermediate activation node connected between a phoneme node and a word node.

25. The apparatus of claim 21, further comprising:

a third layer, having a plurality of grammar nodes with associated activation values, for associating a grammatical part of speech with said word classified by said second layer;

means for providing interaction between said second and third layers to update selected ones of the word node activation values and grammar node activation values; and

means for classifying said spoken sounds according to selected ones of said phoneme node activation values, said word node activation values, or said grammar node activation values.

26. The apparatus of claim 25, further including:

means for providing interaction between multilayer networks, each representing a different step of time.

27. The apparatus of claim 25, wherein said first layer further includes a phoneme node for each one of the possible phonemes;

wherein said second layer further includes a word node for each word in a vocabulary;

wherein said third layer further includes a grammar node for each grammatical part of speech;

wherein said means for providing multilayer relaxation interaction between said first and second layers further includes interconnections among phoneme nodes and word nodes; and

wherein said means for providing interaction between said second and third layers further comprises interconnections among word nodes and grammar nodes.

28. A multilayer temporal relaxation network apparatus for classifying a signal representing spoken sounds comprising:

a first layer of interconnected phoneme nodes, wherein each phoneme node has an activation value responsive to an input signal feature vector;

a second layer of interconnected word nodes connected to said phoneme nodes of said first layer, wherein each word node has an activation value responsive to a sequence of active phoneme nodes; and

means for providing interaction between multilayer relaxation networks, wherein each multilayer relaxation network represents a step in time.

29. The apparatus of claim 28, further comprising:

a phoneme match node connected between a phoneme node of said first layer and a word node of said second layer.

30. The apparatus of claim 28, further comprising:

a third layer of interconnected grammar nodes, wherein each grammar node has an activation value responsive to a sequence of active word nodes.

31. An apparatus for classifying a signal using a multilayer temporal relaxation network comprising:

means for receiving an input signal feature vector;

means for applying a multilayer relaxation process that updates an activation value of a node in the multilayer network for a current time step;

means for providing interaction between multilayer relaxation networks, wherein each multilayer relaxation network represents a step in time; and

means for outputting a classification.

32. A multilayer relaxation apparatus for classifying a signal comprising:

a first multilayer network representing a first time step, said first multilayer network including a first layer and a second layer;

a second multilayer network representing a second time step, said second multilayer network including a first layer and a second layer;

means for relaxing said first multilayer network connecting said first and second layers of said first multilayer network;

means for relaxing said second multilayer network connecting said first and second layers of said second multilayer network;

means for relaxing said first and second multilayer networks together; and

means for classifying said signal after relaxing said first and second multilayer networks.

33. A multilayer relaxation apparatus for classifying a signal comprising:

a first multilayer neural network representing a first time step, said first multilayer neural network including a first neural network layer and a second neural network layer;

a second multilayer neural network representing a second time step, said second multilayer neural network including a first neural network layer and a second neural network layer;

means for relaxing said first multilayer neural network connecting said first and second neural network layers of said first multilayer neural network;

means for relaxing said second multilayer neural network connecting said first and second neural network layers of said second multilayer neural network;

means for relaxing said first and second multilayer neural networks together; and

means for classifying said signal after relaxing said first and second multilayer neural networks.

34. A multilayer temporal relaxation network apparatus for classifying a signal representing spoken sounds comprising:

a first layer having a plurality of phoneme nodes with associated activation values, for receiving an input signal feature vector representing an input signal associated with the spoken sounds and for classifying at least a portion of said spoken sounds as one of a plurality of possible phonemes;

a second layer having a plurality of word nodes with associated activation values, for classifying said spoken sounds;

means for providing multilayer relaxation interaction between said first and second layers to update selected ones of the activation values for phoneme nodes and word nodes;

a third layer having a plurality of grammar nodes with associated activation values, for associating a grammatical part of speech with said spoken sounds classified by said second layer;

means for providing multilayer relaxation interaction between said second and third layers to update selected ones of the word node activation values and selected ones of the grammar node activation values;

means for providing interaction between multilayer relaxation networks, wherein each multilayer relaxation network represents a step in time;

means for classifying said spoken sounds according to selected ones of the activation values of the phoneme nodes, word nodes, or grammar nodes; and

a phoneme match node connecting said first layer and said second layer.

35. A multilayer temporal relaxation network apparatus for signal classification comprising:

an input layer, having a first plurality of nodes with associated activation values, for receiving an input signal feature vector representing an input signal and for classifying at least part of said input signal according to a first signal feature;

a context layer, having a second plurality of nodes with associated activation values, for classifying at least part of said input signal according to a second signal feature, the second signal feature representing a context of said input signal;

means for providing multilayer relaxation interaction between said input layer and said context layer to update selected ones of the activation values for the nodes in said input layer and said context layer during classification; and

means for providing interaction between multilayer relaxation networks, wherein each multilayer relaxation network represents a step in time; and

means for generating a signal classification for said input signal from said input layer or said second context layer.

36. The apparatus of claim 35, further comprising:

a third layer, having a third plurality of nodes with associated activation values, for classifying a third signal feature representing an additional context of said input signal;

means for providing interaction between said context layer and said third layer to update selected ones of the activation values in said context layer and said third layer; and

wherein the means for generating a signal classification includes means for generating a signal classification from said second or third signal features according to the activation value of one of the nodes in said context layer or said third layer.

37. The apparatus of claim 36, further including:

means for providing interaction between multilayer networks, each representing a different step of time.

38. The apparatus of claim 35, wherein said input layer or said context layer includes a node with an activation function f(t), where f(t)=k₀(t+1)−

1, when t<

−

1;

−

k₁L((−

t/k₂)−

2)+k₃, when −

1≦

t<

0;

0 when t=0;

k₁L((t/k₂)−

2)−

k₃, when 0<

t≦

1; and

k₀(t−

1)+1, when t>

1;

wherein operator L(t)=1/(1+e<sup>−

2t</sup>) and k₀, k₁, k₂, and k₃ are scaling constants.

39. The apparatus of claim 35, wherein said input layer or said context layer includes a node with an activation function representing scaled and translated logistic functions combined to produce a flat response near the middle of the activation function domain and a linear response in a clamping region.

40. A multilayer relaxation network apparatus for signal classification comprising:

an input layer, having a first plurality of nodes with associated activation values, for receiving an input signal feature vector representing an input signal and for classifying at least part of said input signal according to a first signal feature;

a context layer, having a second plurality of nodes with associated activation values, for classifying at least part of said input signal according to a second signal feature, the second signal feature representing a context of said input signal, wherein said input layer or said context layer includes a node with an activation function f(t), where f(t)=k₀(t+1)−

1, when t<

−

1;

−

k₁L((−

t/k₂)−

2)+k₃, when −

1≦

t<

0;

0 when t=0;

k₁L((t/k₂)−

2)−

k₃, when 0<

t≦

1; and

k₀(t−

1)+1, when t>

1;

wherein operator L(t)=1/(1+e<sup>−

2t</sup>) and k₀, k₁, k₂, and k₃ are scaling constants;

means for providing interaction between said input layer and said context layer to update selected ones of the activation values for the nodes in said input layer and said context layer during classification; and

means for generating a signal classification for said input signal from said input layer or said second context layer.

41. A multilayer relaxation network apparatus for signal classification comprising:

an input layer, having a first plurality of nodes with associated activation values, for receiving an input signal feature vector representing an input signal and for classifying at least part of said input signal according to a first signal feature;

a context layer, having a second plurality of nodes with associated activation values, for classifying at least part of said input signal according to a second signal feature, the second signal feature representing a context of said input signal, wherein said input layer or said context layer includes a node with an activation function representing scaled and translated logistic functions combined to produce a flat response near the middle of the activation function domain and a linear response in a clamping region;

means for providing interaction between said input layer and said context layer to update selected ones of the activation values for the nodes in said input layer and said context layer during classification; and

means for generating a signal classification for said input signal from said input layer or said second context layer.

42. A computer program product for use in a computer adapted for classifying a signal, the computer program product comprising a computer readable medium for storing computer readable code means, which when executed by a the computer, enables the computer to classify a signal, and wherein the computer readable code means includes computer readable instructions for causing the computer to execute a method comprising the steps of:

receiving an input signal feature vector in a first layer of a multilayer network;

updating an activation value of a node in the multilayer network for a current time step using multilayer relaxation;

providing interaction between multilayer relaxation networks, wherein each multilayer relaxation network represents a step in time;

determining whether there is a candidate classification for the current time step in a second layer;

outputting the candidate classification for the current time step; and

incrementing the current time step.