Method for constructing a neural device for classification of objects

• US 5,802,507 A
• Filed: 07/07/1997
• Issued: 09/01/1998
• Est. Priority Date: 12/16/1992
• Status: Expired due to Fees

• Current Assignee: US Philips Corporation (Koninklijke Philips N.V.)
• Original Assignee: US Philips Corporation (Koninklijke Philips N.V.)
• Inventors: Gentric, Philippe, Minot, Joel
• Primary Examiner(s): MacDonald, Allen R.
• Assistant Examiner(s): Smith, Jeffrey S.
• Application Number:
  US08/888,414
• Time in Patent Office: 421 Days
• Field of Search: 395/22, 395/23, 395/24
• US Class Current: 706/25
• CPC Class Codes: G06F 18/24137 Distances to cluster centroïds
• CPC Class Codes: G06N 3/082 modifying the architecture, e.g. adding, deleting or silencing nodes or connections
• CPC Class Codes: G06V 30/194 References adjustable by an adaptive method, e.g. learning

Abstract

On the basis of a device without hidden neurons, arbitrary samples are taken from a set of learning samples so as to be presented as objects to be classified. Each time if the response is not correct, a hidden neuron (H_i) is introduced with a connection to the output neuron (O_j) of the class of the sample, whereas if the response is correct, no neuron whatsoever is added. During this introduction phase for hidden neurons, the neurons are subdivided into groups by searching for each of the introduced neurons whether it falls within an existing group, in which case it is incorporated therein; otherwise a new group is created around this neuron. The association with a group is defined as a function of the distance from the "creator" neuron. Application: character recognition systems.

Claims

1. A method of constructing a neural device for the classification of objects, where the neural device is trained using a set of learning samples of objects having known classes, each object to be classified being defined by an input vector which is represented by a point in hyperspace, the device comprising:

a layer of input neurons, each of which corresponds to one of the dimensions of the hyperspace;

a layer of hidden neurons whose inputs are connected exclusively to the input neurons, activation of each hidden neuron being based on coordinates of a respective reference point of the hyperspace; and

a layer of output neurons, each of which corresponds to a respective class of the objects;

the method comprising;

a) applying an arbitrarily chosen sample from said set of learning samples to the neural device for classification;

b) subsequently placing a hidden neuron in the device, while defining the respective reference point associated with said hidden neuron as the point in hyperspace representing the sample;

c) establishing an excitatory connection of positive weight between said hidden neuron and the output neuron corresponding to a class of the sample;

d) choosing a new sample from said set of learning samples;

e) applying the new sample to the device for classification;

f) if, as a result of step e), a response of the device to the new sample does not give a correct classification, introducing into the device every time an incorrect classification occurs a new hidden neuron, corresponding to the new sample, by

I) defining the respective reference point associated with the new hidden neuron as being the point representing the new sample and;

II) establishing an excitatory connection of positive weight between the new hidden neuron and the output neuron corresponding to the class of the new sample;

g) if the response does give the correct classification, skipping step f);

h) treating all remaining samples according to steps e), f) and g) until there are no samples left in said set of learning samples; and

i) defining groups of neurons, with one neuron that is representative of each group and the group defined as all neurons located within a hypersphere of a predetermined radius centered in hyperspace upon the representative neuron, as the new hidden neurons are introduced, according to the following process;

I) determining, for each of the new hidden neurons, whether that new hidden neuron is located within the hypersphere of a previously defined group;

II) in response to a positive result from the determining step, incorporating that new hidden neuron into the group defined by the hypersphere in which that new hidden neuron is located; and

III) in response to a negative result from the determining step, forming a new group defined as all neurons located within a hypersphere of a predetermined radius centered upon the new hidden neuron, the new hidden neuron thus being defined as representative of the new group.

2. A method as claimed in claim 1, characterized in that a neuron is considered to form part of a given group if the distance between the reference point associated with this neuron and the reference point associated with the neuron which is representative of the given group is smaller than a given distance and further including the step of starting sample application without a hidden neuron in the neural device.

3. A method as claimed in claim 2, wherein, the groups being considered to be of a first level,

defining at least one second given distance which is greater than the first distance;

and

defining at least second groups of a second level based on the second given distance during the process.

4. A method as claimed in claim 1, characterized in that when two hidden neurons corresponding to two learning samples have representative points whose coordinates neighbour one another, without said samples belonging to the same class, an inhibitory synapse of negative weight is inserted between the hidden neuron corresponding to one of the samples and the output neuron corresponding to the class of the other sample.

5. A method as claimed in claim 1, characterized in that a supplementary hidden neuron which corresponds to a learning sample is inserted with a connection to the corresponding output neuron if for this sample, presented as an object to be classified, an output neuron is effectively activated for the desired class, but the difference between the activity of the output neuron with the highest activation level and that of the output neuron with the highest activation level but one is below a given threshold.