Perceptive system including a neural network

• US 5,835,901 A
• Filed: 07/30/1997
• Issued: 11/10/1998
• Est. Priority Date: 01/25/1994
• Status: Expired due to Fees

• Current Assignee: Lockheed Martin Corporation (Martin Marietta Corporation)
• Original Assignee: Martin Marietta Corporation
• Inventors: Bower, Mark, Beck, Hal E., Brown, Joe R., Duvoisin, Herbert III
• Primary Examiner(s): HAFIZ, TARIQ R
• Application Number:
  US08/903,345
• Time in Patent Office: 468 Days
• Field of Search: 706/15-18, 706/20, 706/21, 706/25, 706/26, 706/27, 706/19, 706/31
• US Class Current: 706/19
• CPC Class Codes: G06F 18/2413 based on distances to training or reference patterns
• CPC Class Codes: G06F 18/2433 Single-class perspective, e.g. one-against-all classification; Novelty detection; Outlier detection
• CPC Class Codes: G06N 3/042 Knowledge-based neural networks; Logical representations of neural networks
• CPC Class Codes: G06N 3/045 Combinations of networks
• CPC Class Codes: G06N 3/082 modifying the architecture, e.g. adding, deleting or silencing nodes or connections

Abstract

A real-time learning (RTL) neural network is capable of indicating when an input feature vector is novel with respect to feature vectors contained within its training data set, and is capable of learning to generate a correct response to a new data vector while maintaining correct responses to previously learned data vectors without requiring that the neural network be retrained on the previously learned data. The neural network has a sensor for inputting a feature vector, a first layer and a second layer. The feature vector is supplied to the first layer which may have one or more declared and unused nodes. During training, the input feature vector is clustered to a declared node only if it lies within a hypervolume defined by the declared node'"'"'s automatically selectable reject radius, else the input feature vector is clustered to an unused node. Clustering in overlapping hypervolumes is determined by a decision surface. During testing of the RTL network, the same strategy is applied to cluster an input feature vector to declared (existing) nodes. If clustering occurs, then a classification signal corresponding to the node is generated. However, if the input feature vector is not clustered to a declared node, then the second layer outputs a signal indicating novelty. The RTL neural network is used in a perceptive system which alternatively selects the RTL network novelty output or the output of a classifier trained on historical target data if the input vector is a subset of the historical target data.

Claims

1. A neural network, comprising:

input means for receiving an input feature vector;

a first layer, coupled to said input means, comprising;

a first declared node, including;

a first feature vector; and

a first hypervolume defined by said first feature vector and a first reject radius;

a second declared node, including;

a second feature vector; and

a second hypervolume defined by said second feature vector and a second reject radius, wherein a portion of said second hypervolume overlaps a portion of said first hypervolume, thereby defining an overlapping region; and

first means for clustering said input feature vector alternatively to only one of said first declared node and said second declared node if said input feature vector lies within said overlapping region.

2. The neural network of claim 1, wherein said first layer further comprises an unused node,

and wherein said neural network further comprises second means for clustering said input feature vector to said unused node only if said input feature vector is not located within either of said first and second hypervolumes.

3. The neural network of claim 1, wherein said first means determines which of said first or second declared nodes said input feature vector will be clustered to in accordance with maximum likelihood statistics.

4. The neural network of claim 1, wherein said first means determines which of said first or second declared nodes said input feature vector will be clustered to in correspondence with which of a first Euclidean distance and a second Euclidean distance is smaller, said first Euclidean distance being determined from said input feature vector to an arithmetic mean of said first declared node, and said second Euclidean distance being determined from said input feature vector to an arithmetic mean of said second declared node.

5. The neutral network of claim 1, further comprising means for automatically adjusting and optimizing the reject radius for the first declared node.

6. A perceptive system, comprising:

input means for receiving an input feature vector;

a first classifier, coupled to said input means, for generating a first classification signal corresponding to said input feature vector, said first classifier generating said first classification signal with the use of a training data set that includes a target feature vector;

first means, coupled to said input means, for generating a first signal indicating whether or not said input feature vector is in a same general region of feature space as said training data set;

a second classifier comprising real-time learning means, coupled to said input means, for generating a real-time classification signal based on said input feature vector and a real-time data set; and

selection means, coupled to said first classifier, said first means, and said real-time learning means, for using said first signal to select as a system output one of said first classification signal and said real-time classification signal.

7. The perceptive system of claim 6, wherein said selection means is an expert system.

8. The perspective system of claim 6 wherein said real-time learning means is a neural network comprising:

input means for receiving said input feature vector;

a first layer, coupled to said input means, comprising a node;

means for clustering said input feature vector to said node only if a distance between said input feature vector and a node feature vector is less than a reject radius associated with said node;

means for generating a classification signal in response to said input feature vector being clustered to said node; and

a second layer, coupled to said first layer, including means for outputting a novelty signal during testing of the real-time learning means if said means for clustering fails to cluster said input feature vector to said node.

9. The perspective system of claim 6, further comprising data feedback means, coupled to said selection means and said real-time learning means, for providing real-time correlation data to said real-time learning means,

wherein said real-time learning means adaptively optimizes nodes in said real-time data set during testing of the real-time learning means.

10. The perspective system of claim 6, wherein the selection means further includes means for establishing a level of confidence measure for the real-time classification signal generated by the real-time learning means and the first classification signal generated by the first classifier.

11. The perspective system of claim 6, wherein the first classifier, the first means, and the real-time learning means are independent neural networks operating in parallel with respect to each other.

12. The perspective system of claim 6, wherein the real-time learning means further includes:

first training means using a spatial novelty training strategy; and

second training means using a node density training strategy.

13. A method for classifying an object, comprising the steps of:

sensing said object to produce a feature vector;

receiving said feature vector in a neural network having a node;

in said neural network, clustering said received feature vector to said node only if a distance between said received feature vector and a node feature vector is less than a reject radius associated with said node;

in said neural network, generating a first classification signal only if said node has at least a predetermined number of feature vectors clustered to it.

wherein said predetermined number is greater than one.

14. A method for classifying an object, comprising the steps of:

sensing said object to produce a feature vector;

receiving said feature vector in a first classifier;

in said first classifier, using a training data set that includes a target feature vector to generate a first classification signal corresponding to said input feature vector;

generating a first signal indicating whether or not said feature vector is in same general region of feature space as the training data set;

receiving said feature vector in a second classifier comprising real-time learning means;

in said real-time learning means, generating a real-time classification signal based on said input feature vector and a real-time data set; and

using said first to select as a classification of said object one of said first classification signal and said real-time classification signal.

15. The method of claim 14, wherein said step of using said first signal further comprises the step of generating a feedback signal that includes real-time correlation data;

and

wherein said method of classifying an object further comprises the step of receiving the feedback signal, in said real-time learning means, for revising said real-time data set using evolutionary techniques.

16. The method of claim 14, wherein using the first signal to select one of said first classification signal and said real-time classification signal further includes establishing a level of confidence measure for the real-time classification signal and the first classification signal.

17. The method of claim 14, wherein generating the first classification signal using the training data set, generating the first signal, and generating the real-time classification signal are independent steps being executed in parallel with respect to each other.

18. The method of claim 14, wherein generating the real-time classification signal further includes:

training the real-time learning means using a spatial novelty training strategy; and

training the real-time learning using a node density training strategy.

19. A method of controlling a real-time learning neural network, said method comprising the steps of:

controlling the real-time learning neural network to detect whether a feature vector fails to cluster to an existing node, and in response to said detection then generating a new node based on the feature vector;

controlling the real-time learning neural network to select a reject radius for the new node from a number of stochastically generated reject radii; and

controlling the real-time learning neural network to detect whether the existing node has a statistically significant number of feature vectors clustered to it, and in response to said detection then designating the existing node as a valid classifier.

20. The method in accordance with claim 19, further comprising the step of:

controlling the real-time learning neural network to remove the existing node if the existing node fails to cluster a statistically significant number of feature vectors.

21. A method for controlling a real-time learning neural network, said method comprising the steps of:

controlling the real-time learning neural network to receive a feature vector;

controlling the real-time learning neural network to detect whether the feature vector falls within a stochastically generated reject radius surrounding an existing node, and in response to said detection then clustering the feature vector to the existing node;

controlling the real-time learning neural network to detect whether the feature vector fails to cluster to the existing node, and in response to said detection then generating a new node based on the feature vector;

controlling the real-time learning neural network to select a reject radius for the new node from a number of stochastically generated reject radii; and

controlling the real-time learning neural network to detect whether the existing node has a statistically significant number of feature vectors clustered to it, and in response to said detection, then optimizing the reject radius of the existing node and designating the existing node as a valid classifier,

wherein the feature vector can cluster to only one node in the real-time learning neural network.

22. The method in accordance with claim 21, further comprising the step of:

controlling the real-time learning neural network to remove nodes from the real-time learning network that fail to cluster a statistically significant number of feature vectors.

23. A method for classifying feature vectors in a real-time learning neural network, said method comprising the steps of:

in a first layer of the real-time learning network that contains a non-adaptive node, clustering a feature vector to the non-adaptive node if the feature vector falls within a reject radius surrounding the non-adaptive node;

if the feature vector clusters to the non-adaptive node in the first layer, then generating a classification signal, wherein the classification signal indicates that the feature vector belongs to a class associated with the non-adaptive node;

if the feature vector fails to cluster to the non-adaptive node in the first layer of the real-time learning network, then supplying the feature vector to a second layer of the real-time learning network and performing the steps of;

clustering the feature vector to an adaptive node in the second layer of the real-time learning network if the feature vector falls within a reject radius surrounding the adaptive node;

generating a new adaptive node in the second layer based on the feature vector if the feature vector fails to cluster to the adaptive node;

selecting a reject radius for the new adaptive node from a number of stochastically generated reject radii;

converting the adaptive node to a new non-adaptive node and removing the new non-adaptive node from the second layer and adding the new non-adaptive node to the first layer, if the adaptive node has a statistically significant number of feature vectors clustered to it; and

removing the adaptive node from the real-time learning network if it fails to cluster a statistically significant number of feature vectors,

wherein the feature vector can cluster to only one node in the real-time learning neural network.

24. The method in accordance with claim 23, further comprising the step of:

adjusting and optimizing the adaptive node.

25. The method of claim 24, wherein said step of adjusting and optimizing the adaptive node comprises the step of:

adjusting and optimizing the reject radius of the adaptive node.

26. A real-time learning neural network employing competitive training, said network comprising:

means for controlling the real-time learning neural network to generate a new node based on a feature vector, if the feature vector fails to cluster to an existing node;

selection means for controlling the real-time learning neural network to select a reject radius for the new node from a number of stochastically generated reject radii; and

means for controlling the real-time learning neural network to designate the existing node as a valid classifier, if the existing node has a statistically significant number of feature vectors clustered to it.

27. A real-time learning neural network for classifying feature vectors, said network comprising:

a first layer containing a non-adaptive node;

means for clustering a feature vector to the non-adaptive node in said first layer, if the feature vector falls within a reject radius surrounding the non-adaptive node;

means for generating a classification signal, if the feature vector clusters to the non-adaptive node in said first layer, wherein the classification signal indicates that the feature vector belongs to a class associated with the non-adaptive node;

means for supplying the feature vector to a second layer, if the feature vector fails to cluster to said one of the non-adaptive node in said first layer;

means for clustering the feature vector to an existing adaptive node in said second layer, if the feature vector falls within a reject radius surrounding the adaptive node;

means for generating a new adaptive node in said second layer based on the feature vector if the feature vector fails to cluster to the existing adaptive node;

selection means for selecting a reject radius for the new adaptive node from a number of stochastically generated reject radii;

means for converting the adaptive node to a new non-adaptive node and for removing the new non-adaptive node from the second layer and adding the new non-adaptive node to the first layer, if the existing adaptive node has a statistically significant number of feature vectors clustered to it; and

means for removing the adaptive node from the real-time network if the adaptive node fails to cluster a statistically significant number of feature vectors,

wherein the feature vector can cluster to only one node.