Clustered neural networks

• US 5,155,801 A
• Filed: 10/09/1990
• Issued: 10/13/1992
• Est. Priority Date: 10/09/1990
• Status: Expired due to Term

• Current Assignee: Raytheon Company (Rtx Corporation)
• Original Assignee: Hughes Aircraft Company (Rtx Corporation)
• Inventors: Lincoln, William P.
• Primary Examiner(s): Fleming, Michael R.
• Assistant Examiner(s): DOWNS, ROBERT W
• Application Number:
  US07/594,458
• Time in Patent Office: 735 Days
• Field of Search: 364/807, 364/513, 364/184, 364/187, 371/36, 395/21, 395/22
• US Class Current: 706/20
• CPC Class Codes: G06F 18/24317 Piecewise classification, i.e. whereby each classification requires several discriminant rules
• CPC Class Codes: G06F 18/254 of classification results, e.g. of results related to same input data
• CPC Class Codes: G06N 3/045 Combinations of networks
• CPC Class Codes: G06V 10/255 Detecting or recognising potential candidate objects based on visual cues, e.g. shapes

Abstract

A plurality of neural networks are coupled to an output neural network, or judge network, to form a clustered neural network. Each of the plurality of clustered networks comprises a supervised learning rule back-propagated neural network. Each of the clustered neural networks are trained to perform substantially the same mapping function before they are clustered. Following training, the clustered neural network computes its output by taking an "average" of the outputs of the individual neural networks that make up the cluster. The judge network combines the outputs of the plurality of individual neural networks to provide the output from the entire clustered network. In addition, the output of the judge network may be fed back to each of the individual neural networks and used as a training input thereto, in order to provide for continuous training. The use of the clustered network increases the speed of learning and results in better generalization. In addition, clustering multiple back-propagation networks provides for increased performance and fault tolerance when compared to a single unclustered network having substantially the same computational complexity. The present invention may be used in applications that are amenable to neural network solutions, including control and image processing applications. Clustering of the networks also permits the use of smaller networks and provides for improved performance. The clustering of multiple back-propagation networks provides for synergy that improves the properties of the clustered network over a comparably complex non-clustered network.

Claims

1. A clustered neural network architecture comprising:

(A) a plurality of individual neural networks, each neural network having an input and an output, wherein each of the inputs are connected in common, and each neural network comprising;

(a) means for receiving initial weights that are different from initial weights of others of the plurality of neural networks;

(b) means for training the neural network to implement a mapping function using the received initial weights, the mapping function being the same for all of the plurality of neural networks; and

(c) means for performing the mapping function in a different way than the other networks of the plurality of neural networks; and

(B) an output neural network having a plurality of inputs and a single output that provides an output of the clustered neural network, and wherein the respective outputs of the plurality of individual neural networks are individually coupled to the respective plurality of inputs of the output neural network.

2. The neural network architecture of claim 1 wherein each of the respective plurality of individual neural networks comprises a supervised, rule-based, back-propagation neural network.

3. The neural network architecture of claim 2 wherein the output of the output neural network is coupled to each of the respective plurality of neural networks.

4. The neural network architecture of claim 1 wherein the output of the output neural network is coupled to each of the respective plurality of neural networks.

5. A target detection system for use with a missile that incorporates an imaging system and a guidance system that are adapted to detect a target located in an image scene and guide the missile towards the target, said target detection system comprising:

a segmentor adapted to receive image data from the imaging system for identifying regions of the image scene that contain possible targets, and for providing output signals indicative of possible target images; and

a classifier adapted to process the output signals indicative of the possible targets provided by the segmentor to determine the presence of a target in the image scene and provide target location signals to the guidance system to direct the missile towards the target, said classifier comprising;

(A) a plurality of individual neural networks, each neural network having an input and an output, wherein each of the inputs are connected in common and are coupled to receive the output signals from the segmentor, and each neural network comprising;

(a) means for receiving initial weights that are different from initial weights of others of the plurality of neural networks;

(b) means for training the neural network to implement a mapping function using the received initial weights, the mapping function being the same for all the plurality of neural networks; and

(c) means for performing the mapping function in a different way that the other networks of the plurality of neural networks; and

(B) an output neural network having a plurality of inputs and a single output that provides an output of the classifier, and wherein the respective outputs of the plurality of individual neural networks are individually coupled to the respective plurality of inputs of the output neural network;

the classifier providing target location signals to the guidance system of the missile that are indicative of the location of the target to direct the missile towards the target.

6. The target detection system of claim 5 wherein each of the respective plurality of individual neural networks comprises a supervised, rule-based, back-propagation neural network.

7. The target detection system of claim 6 wherein the output of the output neural network is coupled to each of the respective plurality of neural networks.

8. The target detection system of claim 5 wherein the output of the output neural network is coupled to each of the respective plurality of neural networks.