Rapidly trainable neural tree network

• US 5,634,087 A
• Filed: 11/29/1993
• Issued: 05/27/1997
• Est. Priority Date: 02/28/1991
• Status: Expired due to Term

• Current Assignee: Bank One Colorado NA As Agent
• Original Assignee: Rutgers University
• Inventors: Sankar, Ananth, Farrell, Kevin R., Mammone, Richard J.
• Primary Examiner(s): Davis, George B.
• Application Number:
  US08/159,397
• Time in Patent Office: 1,275 Days
• Field of Search: 395/21, 395/22, 395/23, 395/24, 395/27
• US Class Current: 706/25
• CPC Class Codes: G06F 18/24323 Tree-organised classifiers
• CPC Class Codes: G06N 3/063 using electronic means

Abstract

An apparatus and methods characterized by an electric neural network including a node having multipliers respectively receiving signals representing feature vector elements and signals representing weight vector elements to produce product signals, a summer to add the product signals with a bias signal and output a sum signal to a hard limiter, the hard limiter for outputting a preliminary output signal of polarity. In response to the output signal of polarity, one of at least two logic branches is enabled. In response to such enabling, weight elements are assigned to a next weight vector to be used in subsequent processing by the one of the at least two logic branches until a label is to be produced.

Claims

1. An electric neural tree network including:

a node including multipliers respectively receiving signals representing feature vector elements and signals representing weight vector elements to produce product signals, a summer to add the product signals with a bias signal and output a sum signal to a hard limiter, the hard limiter for outputting a preliminary output signal having a polarity;

gate means for enabling only one of a plurality of logic branches in response to the polarity of the preliminary output signal;

means for assigning, electrically connected to the one logic branch, to assign weight elements to a next weight vector to be used in a subsequent processing of the next weight vector; and

means for producing a label signal in response to the subsequent processing.

2. The electrical neural tree network of claim 1, wherein said means for assigning includes:

control logic to determine an address for the weight elements to the next weight vector corresponding to the one of the at least two logic branches, the control logic also to determine an address for the label signal; and

memory logic for storing data at the determined address, the data including the next weight vector and bias which are routed back to the node.

3. The electrical neural tree network of claim 1, wherein said means for assigning includes:

a bit to determine whether a weight enable signal should trigger loading the next weight vector and bias back to the node or whether a leaf enable signal should trigger a class signal to be output.

4. The electrical neural tree network of claim 1, further comprising:

gate means; and

wherein the at least two logic branches each include;

a child node electrically connected to the gate means such that only one of the child nodes is enabled by the gate means receiving a value of the preliminary output signal to permit the processing of the next weight vector and bias.

5. The electrical neural tree network of claim 4, wherein the means for producing includes:

hard limiters respectively connecting some of the child nodes to an encoder to produce the label signal.

6. The electrical neural tree network of claim 1, further including:

a feature extractor examining speech signals for producing the signals representing feature vector elements; and

wherein the label signal represents speaker recognition.

7. The electrical neural tree network of claim 1, further including:

a feature extractor examining face signals for producing the signals representing feature vector elements; and

wherein the label signal represents face recognition.

8. The electrical neural tree network of claim 1, further including:

a feature extractor examining chemical analysis for producing the signals representing feature vector elements; and

wherein the label signal represents chemical recognition.

9. The electrical neural tree network of claim 8, wherein:

the label signal represents recognition of an explosive.

10. The electrical neural tree network of claim 1, wherein:

the logic branches are organized hierarchically and limited in number by backward pruning.

61. The neural tree network of any one of claims 1 or 11 or the method of any one of claims 21, 31, 41, or 51, wherein:

the subsequent processing is carried out by means comprising one of at least three logic branches.

11. A neural tree network processing electrical signals, the network including:

a node means for processing input electrical signals representing a weight vector and a feature vector, the weight vector being comprised of weight elements and the feature vector being comprised of feature elements, to produce a preliminary output signal; and

gate means, electrically connected to the node means, for enabling only one of a plurality of logic branches in response to the polarity of the preliminary output signal; and

means, electrically connected to the means for enabling for assigning weight elements to a next weight vector to be used in a subsequent processing by the neural tree network.

12. The neural tree network of claim 11, further comprising:

child nodes, electrically connected to the means for assigning such that the enabling only one of at least two logic branches enables only one of the child nodes to perform the subsequent processing by receiving and processing the weight elements of the next weight vector and the feature elements.

13. The neural tree network of claim 11, wherein:

the means for assigning includes memory means for storing a lookup table containing a plurality of sets of weight elements such that the enabling only one of at least two logic branches enables only one of the sets of weight elements to be used in the next weight vector, and wherein said next weight vector and the feature elements are routed back to the node to perform the subsequent processing by receiving and processing the weight elements of the next weight vector and the feature elements.

14. The neural tree network of claim 11, wherein:

the means for enabling includes gate means to enable the one of the logic branches in response to the polarity of the preliminary output signal.

15. The neural tree network of claim 11, wherein:

the electric neural tree network is produced by training to produce a hierarchical organization of the weight vectors; and

wherein;

the processing includes implementing a linear sequential decision at each said node until a label is produced at a leaf node for training data.

16. The neural tree network of claim 15, wherein the hierarchical organization is formed by training the neural tree network by a method including backward pruning.

17. The neural tree network of claim 16, wherein the method of training includes determining a confidence measure for each said leaf node.

18. The neural tree network of claim 15, wherein the hierarchical organization is formed by training the neural tree network by a method including forward pruning.

19. The neural tree network of claim 18, wherein the method of training includes determining a confidence measure for each said leaf node.

20. The neural tree network of claim 11, further comprising a feature extractor for producing the signals representing feature vector elements, the feature elements from feature data produced by feature analysis performed by the feature extractor.

21. A method of using an electric neural tree network, the method including the steps of:

producing a preliminary output signal having a polarity by a node in which respective multipliers respectively receive signals representing feature vector elements and signals representing weight vector elements to produce product signals, a summer adds the product signals with a bias signal and outputs a sum signal to a hard limiter, and the hard limiter outputs the preliminary output signal;

enabling only one of a plurality of logic branches with the preliminary output signal;

assigning weight elements to a next weight vector to used in a subsequent processing of the next weight vector, the step of assigning being carried out until a label signal is to be produced; and

subsequent to the step of assigning, producing the label signal from the neural tree network.

22. The method of claim 21, wherein the step of assigning includes:

determining with control logic means an address for weight elements to the next weight vector corresponding to the one of the at least two logic branches; and

storing in a logic memory data at the determined address, the data including the next weight vector and bias which are routed back to the node.

23. The method of claim 21, wherein the step of assigning includes:

designating a bit value to determine whether a weight enable signal should to trigger loading the next weight vector and bias back to the node or whether a leaf enable signal should to trigger a class signal to be output.

24. The method of claim 21, wherein the step of assigning includes:

routing the preliminary output signal to gate means, and triggering the gate means with the polarity of the preliminary output signal; and

wherein the at least two logic branches each include;

child nodes electrically connected to the gate means such that only one of the child nodes is enabled by the gate means receiving a value of the preliminary output signal to permit the processing of the next weight vector and bias.

25. The method of claim 24, wherein the step of producing includes:

producing the label signal at an encoder by hard limiters respectively connecting some of the child nodes to the encoder.

26. The method of claim 21, further including:

producing the signals representing feature vector elements by means including a feature extractor examining signals representing speech; and

wherein the label signal represents speaker recognition.

27. The method of claim 21, further including the step of:

producing the signals representing feature vector elements by means including a feature extractor examining signals representing a face; and

wherein the label signal represents face recognition.

28. The method of claim 21, further including the step of:

producing the signals representing feature vector elements by means including a feature extractor examining signals representing at least one chemical; and

wherein the label signal represents chemical recognition.

29. The method of claim 28, wherein the step of producing the signals representing feature vector elements is carried out so that the label signal represents chemical recognition of an explosive.

30. The method of claim 21, further comprising the step of:

powering the logic branches that were organized hierarchically and limited in number by backward pruning.

31. A method of using a neural tree network processing electrical signals, the method including the steps of:

producing a preliminary output signal from a node having means for processing a weight vector and an input feature vector, the weight vector being comprised of weight elements and the feature vector being comprised of feature elements, to produce a preliminary output signal; and

enabling only one of a plurality of logic branches by the preliminary output signal; and

in the one logic branch, assigning weight elements to a next weight vector to be used in a subsequent processing by the neural tree network to produce a label.

32. The method of claim 31, further comprising the step of:

only one of at least two logic branches to enable only one of at least two child nodes to perform the subsequent processing by receiving and processing the weight elements of the next weight vector and the feature elements.

33. The method of claim 31, wherein the step of assigning includes:

storing in a memory a lookup table containing a plurality of sets of weight elements such that the enabling only one of at least two logic branches enables only one of the sets of weight elements for the next weight vector, and wherein said next weight vector and the feature elements are routed back to the node to perform the subsequent processing by receiving and processing the weight elements of the next weight vector and the feature elements.

34. The method of claim 31, wherein the preliminary output signal has a polarity, and wherein the step of enabling only one of at least two logic branches includes routing the preliminary output signal to gate means to be triggered by the polarity.

35. The method of claim 31, wherein the steps are carried out by an electric neural tree network structured by training to produce a hierarchical organization of the weight vectors;

and wherein method is limited to processing in accordance with a linear sequential decision at each said node until the label is produced at a leaf node for training data.

36. The method of claim 35, wherein the steps are carried out with the hierarchical organization having been formed by training the neural tree network by a method including backward pruning.

37. The method of claim 36, wherein the method of training includes a step of determining a confidence measure for each said leaf node.

38. The method of claim 35, wherein the steps are carried out with the hierarchical organization having been formed by training the neural tree network by a method including forward pruning.

39. The method of claim 38, wherein the method of training includes a step of determining a confidence measure for each said leaf node.

40. The method of claim 31, further comprising a step of:

producing the signals representing feature vector elements by a feature extractor, the feature elements being a subset of feature data produced by feature analysis performed by the feature extractor.

41. A method of making an electric neural tree network, the method including the steps of:

constructing a node by electrically connecting multipliers to respectively receive signals representing feature vector elements and signals representing weight vector elements and to produce product signals, electrically connecting a summer to add the product signals with a bias signal and output a sum signal, and electrically connecting a hard limiter to receive the sum signal and output a preliminary output signal having a polarity;

electrically connecting a gate means for enabling only one of a plurality of logic branches in response to the polarity of the preliminary output signal;

electrically connecting to the one logic branch a means for assigning weight elements to a next weight vector to be used in a subsequent processing; and

electrically connecting a means for producing the label signal from an output signal produced by the subsequent processing.

42. The method of claim 41, wherein the means for assigning includes:

control logic means for determining an address for weight elements to the next weight vector corresponding to the one of the at least two logic branches, the control logic also to determine an address for the label signal, said control logic means being electrically linked to memory logic for storing data at the determined address, the data including the next weight vector and bias which are routed back to the node.

43. The method of claim 41, wherein the step of electrically connecting the means for assigning includes:

designating a bit value to determine whether a weight enable signal should trigger loading the next weight vector and bias back to the node or whether a leaf enable signal should trigger a class signal to be output.

44. The method of claim 41, wherein the step of electrically connecting means for assigning includes:

providing an electrical rout for the preliminary output signal having the polarity to be directed to gate means to be triggered by the polarity; and

wherein the at least two logic branches include respective child nodes electrically connected to the gate means such that only one of the child nodes is enabled by the triggered gate means to permit the processing of the next weight vector and bias.

45. The method of claim 44, further comprising the step of:

locating an encoder to receive signals output by hard limiters respectively connected electrically between leaf nodes and the encoder to construct the label signal.

46. The method of claim 41, further including the step of:

electrically connecting means for producing to produce the signals representing feature vector elements from electrical signals produced by a feature extractor examining signals representing speech so that the label signal represents speaker recognition.

47. The method of claim 41, further including the step of:

electrically connecting means for producing to produce the signals representing feature vector elements from electrical signals produced by a feature extractor examining signals representing a face so that the label signal represents face recognition.

48. The method of claim 41, further including the step of:

electrically connecting means for producing to produce the signals representing feature vector elements from electrical signals produced by a feature extractor examining signals representing at least one chemical so that the label signal represents chemical recognition.

49. The method of claim 48, wherein the step of electrically connecting means for producing to produce the signals representing feature vector elements is carried out so that the label signal represents recognition of an explosive.

50. The method of claim 41, further comprising the step of:

organizing the logic branches hierarchically by backward pruning.

51. A method of making a neural tree network processing electrical signals, the method including the steps of:

providing electricity to a node to process a weight vector and an input feature vector, the weight vector being comprised of weight elements and the feature vector being comprised of feature elements, and to produce a preliminary output signal; and

providing the preliminary output signal to gate means to enable only one of a plurality of logic branches; and

electrically connecting to the one logic branch a means for assigning weight elements to a next weight vector to be used in a subsequent processing by the neural tree network to produce a label signal.

52. The method of claim 51, wherein the the means for assigning is electrically connected to

child nodes such that only one of the child nodes is enabled to perform the subsequent processing by receiving and processing the weight elements of the next weight vector and the feature elements.

53. The method of claim 51, wherein the means for assigning includes:

memory means storing a lookup table containing a plurality of sets of weight elements such that the enabling only one of at least two logic branches enables only one of the sets of weight elements for the next weight vector, and wherein said next weight vector and the feature elements are routed back to the node to perform the subsequent processing.

54. The method of claim 51, wherein the preliminary output signal has a polarity and wherein the step of providing the preliminary output signal includes:

routing the preliminary output signal to gate means trigger by the polarity of the preliminary output signal.

55. The method of claim 51, wherein the steps are carried out to structure the electric neural tree network by training to produce a hierarchical organization of the weight vectors;

and wherein the training includes a method of processing in accordance with a linear sequential decision at each said node until the label is produced at a leaf node for training data.

56. The method of claim 55, wherein the steps are carried out with the hierarchical organization having been formed by training the neural tree network by a method including backward pruning.

57. The method of claim 56, wherein the method of training includes a step of determining a confidence measure for each said leaf node.

58. The method of claim 55, wherein the steps are carried out with the hierarchical organization having been formed by training the neural tree network by a method including forward pruning.

59. The method of claim 58, wherein the method of training includes a step of determining a confidence measure for each said leaf node.

60. The method of claim 51, further comprising a step of:

producing the signals representing feature vector elements by a feature extractor, the feature elements being a subset of feature data produced by feature analysis performed by the feature extractor.