Method for learning data classification in two separate classes separated by a region divider of order 1 or 2

• US 6,219,658 B1
• Filed: 03/31/1999
• Issued: 04/17/2001
• Est. Priority Date: 10/01/1996
• Status: Expired due to Fees

• Current Assignee: Centre National De La Recherche Scientifique, Commisariat A L'Energie Atomique
• Original Assignee: Centre National De La Recherche Scientifique, Commisariat A L'Energie Atomique
• Inventors: Gordon, Mirta
• Primary Examiner(s): Powell, Mark R.
• Assistant Examiner(s): STARKS, WILBERT L
• Application Number:
  US09/269,204
• Time in Patent Office: 748 Days
• Field of Search: 706/15, 706/16, 706/17, 706/18, 706/19, 706/20, 706/14, 706/25, 706/40, 382/158
• US Class Current: 706/15
• CPC Class Codes: G06F 18/24137 Distances to cluster centroïds
• CPC Class Codes: G06F 18/245 relating to the decision surface
• CPC Class Codes: G06N 3/08 Learning methods

Abstract

A method for learning to classify data according to two distinct classes (c11, c12) separated by a separating surface (S), by means of a neurone of the binary type comprising a parameter describing the separating surface and whose inputs are weighted by a weight (w_i), and including the following steps:

a) defining a cost function C: $C_{\sigma }=\sum _{\underset{\left({\gamma }^{\mu }>o\right)}{\mu =1}}^P\left[A-B\tanh \frac{\sigma {\gamma }^{\mu }}{2T+}\right]+\sum _{\underset{\left({\gamma }^{\mu }\le 0\right)}{\mu =1}}^P\left[A-B\tanh \frac{\sigma {\gamma }^{\mu }}{2T-}\right]$

b) initializing the weights (W_i), the radii (rⁱ), the parameters (σ, T+, T−), the learning rate (ε) and speeds of the temperature decreasing (δT+, δT−);

c) minimizing the cost function C by successive iterations;

d) obtaining the values of the weights of the connections and radii of the neurone.

Application to the classification and recognition of shapes by a neural network.

Claims

1. A method for teaching a neurone with a quadratic activation function to classify data according to two distinct classes (c11, c12) separated by a separating surface (S), this neurone being a binary neurone having N connections coming from an input and receiving as an input N numbers representing a data item intended to be classified using a learning base containing a plurality of known data, each input of the neurone being affected by a weight (w_i) of the corresponding connection,

characterised in that it includes the following steps:

a) defining a cost function (C_σ

) by determining, as a function of a parameter describing the separating surface, a stability (γ

^μ

) of each data item (μ

) of the learning base, the cost function being the sum of all the costs determined for all the data in the learning base with;

$C_{\mathrm{\sigma <br><br>}}=\underset{\underset{\left({\mathrm{\gamma <br><br>}}^{\mathrm{\mu <br><br>}}><br><br>o\right)}{\mathrm{\mu <br><br>}=1}}{\overset{P}{\sum <br><br>}}<br><br>\left[A-B<br><br>\text{\hspace{1em}<br><br>}<br><br>\tanh <br><br>\text{\hspace{1em}<br><br>}<br><br>\frac{\mathrm{\sigma <br><br>}<br><br>\text{\hspace{1em}<br><br>}<br><br>{\mathrm{\gamma <br><br>}}^{\mathrm{\mu <br><br>}}}{2<br><br>T+}\right]+\underset{\underset{\left({\mathrm{\gamma <br><br>}}^{\mathrm{\mu <br><br>}}\le <br><br>0\right)}{\mathrm{\mu <br><br>}=1}}{\overset{P}{\sum <br><br>}}<br><br>\left[A-B<br><br>\text{\hspace{1em}<br><br>}<br><br>\tanh <br><br>\text{\hspace{1em}<br><br>}<br><br>\frac{\mathrm{\sigma <br><br>}<br><br>\text{\hspace{1em}<br><br>}<br><br>{\mathrm{\gamma <br><br>}}^{\mathrm{\mu <br><br>}}}{2<br><br>T-}\right]$

 

where A is any value, B is any positive real number, P is the number of data items in the learning base, γ

^μ

is the stability of the data item μ

, and T+, T− and

σ

are two parameters of the cost function;

b) initialising the weights (w_i), the radii (r_i), the parameters (T+ and T−

, with T+<

T−

), a learning rate ε and

speeds of the temperature decreasing (δ

T+ and δ

T−

);

c) minimising, with respect to the weight of the connections (W_i) and the radii (r_i), the cost function (Cσ

) by successive iterations during which the parameters (T+ and T−

) decrease at speeds of the temperature decreasing (δ

T+ and δ

T−

) as far as a predefined stop criterion;

d) obtaining values of the weights of the connections and the radii of the neurone.

2. A method according to claim 1, characterised in that the stability of each data item is:

${\mathrm{\gamma <br><br>}}^{\mathrm{\mu <br><br>}}=y^{\mathrm{\mu <br><br>}}<br><br>\ln <br><br>\left[\underset{i=1}{\overset{N}{\sum <br><br>}}<br><br>\text{\hspace{1em}<br><br>}<br><br>\frac{{\left(w_i-x_i^{\mathrm{\mu <br><br>}}\right)}^2}{r_i^2}\right],<br><br>\mathrm{with}<br><br>\text{\hspace{1em}<br><br>}<br><br>y=+1<br><br>\text{\hspace{1em}<br><br>}<br><br>\mathrm{or}<br><br>\text{\hspace{1em}<br><br>}-1,\mathrm{and}$

where μ

is the label of the pattern, x^μ

_i is the value of the pattern μ

for the i^th input, y^μ

is the class of the pattern μ

, N is the number of inputs and connections of the neurone, w_i is the weight of the connection between the input i and the neurone and r_j the radius parameter for the i^th input.

3. A method according to claim 1, characterised in that the stability of each data item is ${\gamma }^{}$

μ

=yμ



[∑

i=1N

[ 



(wi-xiμ

)2-ri2]]$\mathrm{with}<br><br>\text{\hspace{1em}<br><br>}<br><br>y=+1<br><br>\text{\hspace{1em}<br><br>}<br><br>\mathrm{or}<br><br>\text{\hspace{1em}<br><br>}-1,\mathrm{and}$

where μ

is the label of the pattern, x^μ

_i is the value of the pattern μ

for the i^th input, y^μ

is the class of the pattern μ

, N is the number of inputs and connections of the neurone, w_i is the weight of the connection between the input i and the neurone and r_i is the radius parameter for the i^th input.