Unsupervised training method for a neural net and a neural net classifier device

• US 5,469,530 A
• Filed: 05/22/1992
• Issued: 11/21/1995
• Est. Priority Date: 05/24/1991
• Status: Expired due to Fees

• Current Assignee: US Philips Corporation (Koninklijke Philips N.V.)
• Original Assignee: US Philips Corporation (Koninklijke Philips N.V.)
• Inventors: Makram-Ebeid, Sherif
• Primary Examiner(s): Downs, Robert W.
• Assistant Examiner(s): HAFIZ, TARIQ R
• Application Number:
  US07/887,636
• Time in Patent Office: 1,278 Days
• Field of Search: 395/23, 395/24, 395/25, 395/22, 395/13, 395/27, 395/2.41, 364/413.01
• US Class Current: 706/25
• CPC Class Codes: G06F 18/232 Non-hierarchical techniques
• CPC Class Codes: G06N 3/063 using electronic means
• CPC Class Codes: G06N 3/088 Non-supervised learning, e.g. competitive learning

Abstract

Unsupervised training method for a neural net and a neural net classifier device wherein test vectors are supplied to the neural net whose operational parameters are classified in a stochastic labeling procedure by mutually correlating the net'"'"'s output activations for each test vector and on the basis thereof generating weighting factors that scale the probabilities when selecting a class at random. Once the test vectors are thus classified, the operational parameters of the net are modified in order to intensify the differences among the patterns of output activations for all test vectors.

Claims

1. A device for classifying in unsupervised training a pluralty of input vectors into classes comprising:

(a) neural net having an input for receiving the input vectors and an output for providing a pattern of output activations associated with the input vector, and a memory for storing values of operational parameters of the net;

(b) a stochastic labeling means, coupled to the output, for generating weighting factors for each of the classes upon mutually correlating the output activations of the patterns and for perfoming a stochastic selection using probabilities of a single class on the basis of probabilities weighted by the weighting factors;

(c) an intensifying means coupled to the memory for intensifying differences between the patterns by modifying the values of the operational parameters upon re-supplying the test vectors in random order and for storing the modified values in the memory.

2. The device of claim 1 wherein the stochastic labeling means comprises

a random number generator for producing a random number that lies within a predetermined range;

selection means, connected to the random number generator, for selecting the single class on the basis of a division of the range in a series of consecutive intervals, each interval being proportional to a weighting factor, the single class being specified by a particular interval wherein the random number falls.

3. An unsupervised training method for training a neural net having plural inputs to classify input vectors into a number of classes on the basis of a set of test vectors comprising the steps:

(a) initializing the neural net by assigning random values to operational parameters of the neural net,

(b) performing a stochastic labelling of classes by;

(i) supplying a set of test vectors to the inputs of the neural net,

(ii) for each test vector, obtaining a pattern of output activations at an output of the neural net,

(iii) assigning the test vectors to the classes, by mutually correlating the patterns of output activations by generating an associated weighting factor on the basis of said activations and selecting the class in a stochastic procedure having probabilities wherein the probabilities of assigning the test vectors to the classes are weighted with the associated weighting factors,

(c) said stochastic labelling step including an intensifying step, said intensifying step including;

(i) at least once;

(1) re-supplying the set of test vectors to the neural net in random order for generating further output activations;

(2) for each test vector modifying the operational parameters on the basis of the patterns of the further activations in order to increase the differences between the patterns.

4. The method of claim 3 wherein, upon termination of the intensifying step, each test vector is indicated as being assigned to a class by means of a respective pattern having at least one of the further output activations with a value that is more extreme than the other further output activations of the pattern.

5. The method of claim 3 wherein the test vectors of a first space are pretreated by means of subjecting the test vectors to a mapping onto a bounded region in a second space prior to being supplied to the neural net for enhancing clustering behaviour of the neural net.

6. An unsupervised training method for training a neural net to classify data, comprising the steps:

(a) initializing the neural net by assigning random values to operational parameters of the neural net,

(b) performing a stochastic labelling of classes by;

(i) supplying a set of test vectors to an input of the neural net,

(ii) for each test vector, obtaining a pattern of output activations at an output of the neural net,

(iii) assigning the test vectors to the classes, by mutually correlating the patterns of output activations,

(c) said stochastic labelling step including an intensifying step, said intensifying step including;

(i) modifying the operational parameters of the neural net, in response to a second pass through the test vectors, for intensifying differences between the patterns of output activations according to a predetermined criterion, so that each pattern represents a better-defined class,

(ii) said second pass including, at least once;

(1) re-supplying the set of test vectors to the neural net in random order for generating further output activations,

(2) for each test vector modifying the operational parameters on the basis of the patterns of the further activations in order to increase the differences between the patterns,

(iii) upon termination of the intensifying step, each test vector being indicated as being assigned to a class by the use of a pattern having at least one of the further output activations with a value that is more extreme than the other further output activations of the pattern,

(d) the differences between the patterns being evaluated on the basis of comparing each of the further output activations of each of the patterns to a predetermined threshold.