Neural system of classification and classification method using such a system

• US 5,175,796 A
• Filed: 04/12/1991
• Issued: 12/29/1992
• Est. Priority Date: 04/24/1990
• Status: Expired due to Fees

• Current Assignee: Thomson-CSF (Thales SA)
• Original Assignee: Thomson-CSF (Thales SA)
• Inventors: Refregier, Philippe, Vallet, Francois
• Primary Examiner(s): Fleming, Michael R.
• Assistant Examiner(s): TRYON, JODI
• Application Number:
  US07/684,173
• Time in Patent Office: 627 Days
• Field of Search: 395/11, 395/21, 395/22
• US Class Current: 706/20
• CPC Class Codes: G06F 18/24155 Bayesian classification
• CPC Class Codes: G06F 18/24317 Piecewise classification, i.e. whereby each classification requires several discriminant rules
• CPC Class Codes: G06N 3/04 Architecture, e.g. interconnection topology

Abstract

The disclosure concerns neural networks designed specially for the classification of objects represented by vectors X. If the vectors X include several parameters and if the objects have to be classified in a large number N of classes, the end result is a very large number of interconnections which become difficult to set up physically, are slow in their operation and require lengthy learning phases. The disclosed neural classification system has the particular feature of being constituted on the basis of P neural networks each individually carrying out the classification of objects in only two classes or, at any rate, in a small number of classes only. These networks give probabilities P_i,j of membership in a class C_i among two classes C_i and C_j. The outputs of these networks are connected to a signal processing module which, through simple functions (implementing linear combinations of the outputs and non-linear standardization functions) establishes, on N outputs, results P_i (X) of classification among the N classes. The learning is done on classifications by pairs of classes, but the post-learning recognition gives classifications among N classes.

Claims

1. A neural system adapted to operate in a learning phase and a post-learning phase for performing object recognition by the classification of objects in N classes, comprising:

P distinct neural networks, wherein each neural network P includes input layer neurons for receiving a first input signal representing an object X to be classified and an output layer neuron comprising a first circuit for performing a weighted addition function on a received signal to produce a first output signal and a second circuit for performing a saturating non-linear function on said first output signal to produce a second output signal for use during the learning phase representing a probability that the object X belongs to a class C_i among of the N classes, n being greater than 1 and smaller than N, and

a signal processing module connected to the P neural networks for receiving during the post-learning phase of object recognition, the first output signals produced by the output layer neurons of the P neural networks, and for producing output signals representing probabilities that object X belongs to class i among N classes.

2. A neural system according to claim 1, wherein the second output signal produced by the output layer neurons of each of the P neural networks represents a probability of classifications of objects in n=2 possible classes, all the N classes being represented in the P networks.

3. A system according to claim 2, wherein the number of networks is P=N-1.

4. A system according to claim 3, wherein the second output signals produced by the output layer neurons of the P neural networks represent probabilities P_i,j (X) that an object X belongs to the class C_i among only two possible classes C_i and C_j, and the first output signals produced by the output layer neurons of the P neural networks represent values f_i,j (X) where

5. A system according to claim 1, wherein the saturating non-linear function of said second circuit is a sigmoid function having the form e^x /(1+e^x).

6. A neural system according to claim 1, wherein said processing module comprises a third circuit for performing linear combinations of signals and producing output signals representing the linear combinations obtained, and a fourth circuit for performing non-linear standardization on the output signals representing the linear combinations obtained and producing N outputs, the sum of which is constant, each of these N outputs representing a probability that the object X belongs to a respective class among N classes.

7. A neural system according to claim 6, wherein the linear combinations have the form g_i (X)=g_j (X)+f_i,j (X) where g_i (X) is a linear combination having an index i, g_j (X) is another linear combination having an index j and f_i,j (X) is a value directly related to the value of the second output signal of said output neuron representing the probability that the object X belongs to the class C_i among only two possible classes C_i and C_j.

8. A neural system according to claim 7, wherein the non-linear standardization function of said fourth circuit has the form:

space="preserve" listing-type="equation">P.sub.i (X)=e.sup.g i.sup.(x) /.

9. A method for the classification of objects in N classes by a neural system comprising the steps of:

learning a configuration of each of a plurality of neural networks P on the basis of input vectors representing an object X to be classified and results from each of the P neural networks of classification in n classes without using a processing module; and

performing operating phases after learning by using a processing module to obtain the results of classification into N classes from the results of classification in n classes by the P networks.

10. A method according to claim 9, wherein the learning phase of the P networks is followed by a phase for additional learning by the system, the step comprising connecting the processing module with a learning base.

11. A neural system adapted to operate in a learning phase and a post-learning phase for performing object recognition by the classification of objects in N classes, comprising:

a plurality of neural networks P including means for receiving an input signal representing an object X to be classified and means, connected to said first input signal receiving means, for producing an output signal for use during the learning phase representing a probability that the object X belongs to a class C_i among n of the N classes, n being greater than 1 and smaller than N, and means, connected to said output signal producing means, for processing said output signal during the post-learning phase to produce N output signals representing probabilities that the object X belongs to class i among N classes.

12. A method for the classification of objects in N classes by a neural system, said neural system having P configurable neural networks and a processing module to which each of the P neural networks is connected to process signals produced by the P neural networks, said method comprising the steps of:

configuring each of said P neural networks during a learning phase by using a set of input signals representing known objects and a set of corresponding output probability signals representing desired classification results for those known objects, wherein a probability signal issued from a respective neural network of the neural networks thus configured represents the probability that an object belongs to a given class among a respective subgroup of n classes selected among said N classes, with n greater than 1 and smaller than N, and with all of the N classes being represented in the P configured neural networks,

performing operating phases after learning by using said processing module, wherein input signals representing an object X to be classified are applied to the configured P neural networks, output signals are produced by the P neural networks, said output signals being applied to inputs of said processing module, and processing module signals are produced on outputs of the processing module, said processing module signals each presenting the probability that object X belongs to one class among the N classes.

13. A method according to claim 12, wherein said P neural networks each have a first output and a second output, said second output for supplying said probability signal during the learning phase and during the operating phases, and said first output for supplying a signal based on said probability signal, said first output being connected to said processing module during the operating phases.

14. A method according to claim 13, wherein said second output signal supplies a signal which is a saturated non-linear function of the signal present on the first output.

15. A method according to claim 14, wherein said saturated function is a sigmoid function.

16. A method for the classification of objects in N classes by a neural system, said neural system having P configurable neural networks each having first outputs and second outputs, and a processing module connected to the first outputs for processing signals produced by the P neural networks, said method comprising the steps:

configuring each of said P neural networks during a learning phase by using a set of input signals representing known objects and a set of corresponding output classification signals representing desired classification results for these known objects, wherein a classification signal produced on the second output of a respective neural network of the configured neural networks represents the probability that an object belongs to a given class among a respective subgroup of n classes selected among said N classes, with n greater than 1 and smaller than N, with all of the N classes being represented in the different subgroups corresponding to the different neural networks,

performing operating phases after learning by using said processing module, wherein input signals representing an object X to be classified are applied to said P neural networks, output signals are produced at the first outputs of the P neural networks and are applied to the processing module, the output signal issued from a respective neural network having a magnitude based on the probability that the object X belongs to one class among the respective selected group of n classes corresponding to the respective neural network, and end classification signals are produced on outputs of the processing module, said end classification signals representing the probability that the object X belongs to a given class among the N classes.

17. A method according to claim 16, wherein the learning phase of the P neural networks is followed by a phase for additional learning by the system, said additional learning phase comprising the step of connecting the processing module and further configuring the neural networks by using a learning base comprising input signals representing known objects and corresponding end classification signals, said input signals applied to the inputs of the neural networks, and said end classification signals produced on the outputs of the processing module.

18. A neural system adapted to operate in a learning phase and a post-learning phase for performing object recognition by the classification of objects in N classes, comprising

P distinct neural networks, each being configured individually and differently, wherein each neural network includes means for receiving a first input signal representing an object X to be classified, and specifically configured processing means connected to said first input signal receiving means, said specifically configured processing means adapted to produce a probability signal representing a probability that the object X belongs to a given class among a respective selected subgroup of n classes, said subgroup selected among the N classes, with n greater than 1 and smaller than N, the processing means of each of the P distinct neural networks being specifically configured so that each of the N classes is represented in the different subgroups corresponding to the P networks, each of said P neural networks further comprising an output for supplying an output signal having a value based on said probability signal,

and a signal processing module having P inputs connected to the outputs of the P neural networks for receiving said output signals during the post-learning phase of object recognition, said processing module having means for converting said output signals into N end classification signals representing probabilities that object X belongs to a respective class among the N classes.

19. A neural system according to claim 18, wherein n=2.

20. A neural system according to claim 18, wherein P=N-1.

21. A neural system according to claim 18, wherein said specifically configured processing means in each neural network comprises means for producing a signal having a value P_i,j (X) which is a function of the input signals representing object X, and wherein said output of each of the neural networks supplies an output signal f_i,j (X) having an amplitude of the form:

space="preserve" listing-type="equation">log(P.sub.i,j (X))-log(1-P.sub.i,j (X)).

22. A neural system according to one of claims 18 to 20, wherein each of said P neural networks comprises a first circuit for performing a weighted summation of signals, said first circuit having an output for supplying a sum signal, and a second circuit for performing a saturating non-linear function on the sum signal produced by the first circuit, and wherein the outputs of the first circuit of the different neural networks are connected to the inputs of said signal processing module.

23. A neural system according to claim 22, wherein said signal processing module comprises a third circuit for performing linear combinations of signals, said third circuit having inputs connected to the outputs of said P neural networks and having outputs for producing said linear combinations of signals, and a fourth circuit for performing non-linear standardization of said linear combinations of signals.

24. A neural system according to claim 23, wherein the linear combinations have the form g_i (X)=g_j (X)+f_i,j (X) where g_i (X) is a linear combination having an index i, where i=1 to N, g_j (X) is a linear combination having an index j, where j=1 to N, and f_i,j (X) is a signal produced on the output of the first circuit of a respective neural network.

25. A neural network according to claim 24, wherein said fourth circuit for performing non-linear standardization produces an output signal P_i (X) having the form:

space="preserve" listing-type="equation">P.sub.i (X)=e.sup.g.sub.i.sup.(X) /(sum of the e.sup.g.sub.i.sup.(X) for i=1 to N).

26. A neural system according to claim 22, wherein the saturating non-linear function of said second circuit is a sigmoid function having the form e^x /(1+e^x).