Self-extending neural-network

• US 5,033,006 A
• Filed: 03/07/1990
• Issued: 07/16/1991
• Est. Priority Date: 03/13/1989
• Status: Expired due to Term

• Current Assignee: Sharp Kabushiki Kaisha (Hon Hai Precision Industry Co., Ltd.)
• Original Assignee: Sharp Kabushiki Kaisha (Hon Hai Precision Industry Co., Ltd.)
• Inventors: Ueda, Toru, Ishizuka, Yasushi, Togawa, Fumio
• Primary Examiner(s): MacDonald, Allen R.
• Assistant Examiner(s): Davis, George
• Application Number:
  US07/489,508
• Time in Patent Office: 496 Days
• Field of Search: 364/513, 307/201, 382/22
• US Class Current: 706/25
• CPC Class Codes: G06F 18/24137 Distances to cluster centroïds
• CPC Class Codes: G06N 3/082 modifying the architecture, e.g. adding, deleting or silencing nodes or connections

Abstract

A self-extending shape neural-network is capable of a self-extending operation in accordance with the studying results. The self-extending shape neural-network has initially minimum number of the intermediate layers and the number of the nodes (units) within each layer by the self-extension of the network construction so as to shorten the studying time and the discriminating time. This studying may be effected efficiently by the studying being directed towards the focus when the studying is not focused.

Claims

1. A self-extending neural-network having a multilayer neural-network composed of at least an input layer, an intermediate layer, and an output layer, which, during a studying operation, obtains inputted studying data and outputs output data according to a value of a coupling weight between nodes in the multilayer neural-network, comprising:

study progress judging portion means for judging whether or not the studying operation is progressing in accordance with the output data and the value of the coupling weight between the nodes and for outputting an extending instruction signal when the studying operation has been judged not to be progressing; and

self-extending portion means, responsive to said extending instruction signal, for providing a new node in accordance with said extending instruction signal from said study progress judging portion means;

said self-extending portion means setting a condition of a coupling between the nodes and said new node and an initial value of a coupling weight between the nodes and said new node so as to self-extend construction of the self-extending neural-network, thereby continuing the studying operation when the construction of the self-extending neural-network has been self-extended by said self-extending portion means.

2. The self-extending shape neural-network as claimed in claim 1, wherein the self-extending neural-network is a multilayer neural-network composed of an input layer, an intermediate layer and an output layer, and said new node is provided within said intermediate layer when said self-extending portion means self-extends the construction of the self-extending neural-network.

3. The self-extending shape neural-network as claimed in claim 2, wherein the self-extending neural-network is a multilayer neural-network composed of an input layer, an intermediate layer and an output layer;

and

said self-extending portion means self-extending the construction of the self-extending neural-network by adding a new intermediate layer to said intermediate layer, said new intermediate layer being composed of said new node.

4. The self-extending shape neural-network as claimed in claim 2, wherein the self-extending neural-network is a multilayer neural-network composed of an input layer, an intermediate layer and an output layer;

and

said self-extending portion means self-extending the construction of the self-extending neural-network by adding a new output layer as an upper layer of said output layer, said output layer being provided as a new intermediate layer, thereby adding to said intermediate layer.

5. The self-extending shape neural-network as claimed in claim 1, wherein the self-extending neural-network is a multilayer neural-network composed of an input layer, an intermediate layer and an output layer;

and

said self-extending portion means self-extending the construction of the self-extending neural-network by adding a new intermediate layer to said intermediate layer, said new intermediate layer being composed of said new node.

6. The self-extending shape neural-network as claimed in claim 5, wherein the self-extending neural-network is a multilayer neural-network composed of an input layer, an intermediate layer and an output layer;

and

said self-extending portion means self-extending the construction of the self-extending neural-network by adding a new output layer as an upper layer of said output layer, said output layer being provided as a new intermediate layer, thereby adding to said intermediate layer.

7. The self-extending shape neural-network as claimed in claim 1, wherein the self-extending neural-network is a multilayer neural-network composed of an input layer, an intermediate layer and an output layer;

and

said self-extending portion means self-extending the construction of the self-extending neural-network by adding a new output layer as an upper layer of said output layer, said output layer being provided as a new intermediate layer, thereby adding to said intermediate layer.

8. A self-extending neural-network comprising:

input portion means for receiving a studying sample;

calculating portion means, operatively connected to said input portion means, for calculating an output value by performing a study operation using a given algorithm upon said studying sample and a value of a coupling weight among respective nodes in said calculating portion means;

output portion means, operatively connected to said calculating portion means, for externally outputting said output value;

study progress judging portion means, operatively connected to said output portion means, for determining whether said study operation is progressing in accordance with said output value and said value of said coupling weight between said nodes; and

said study progress judging portion means outputting an extend instruction signal when said study operation is not progressing;

self-extending portion means, responsive to said extend instruction signal, for providing a new node;

said self-extending portion means establishing a condition of coupling between said nodes and said new node and establishing an initial value for a coupling weight between said nodes and said new node, thereby enabling further study operations after self-extension.

9. The self-extending neural-network as claimed in claim 8 further comprising:

focusing judging portion means, operatively connected to said output portion means, for determining whether said study operation is focused to a result; and

said focusing judging portion means outputting a focus signal when said study operation is not focused;

study control means, operatively connected to said focus judging portion means and said calculating portion means, for controlling said calculating portion means to renew said value of said coupling weight between said nodes and recalculate said output value when said focus signal is received.

10. The self-extending shape neural-network as claimed in claim 9, wherein the self-extending neural-network is a multilayer neural-network composed of an input layer, an intermediate layer, and an output layer;

and

said new node is provided within said intermediate layer when said self-extending portion means self-extends construction of the self-extending neural-network.

11. The self-extending shape neural-network as claimed in claim 9, wherein the self-extending neural-network is a multilayer neural-network composed of an input layer, an intermediate layer, and an output layer;

and

said self-extending portion means self-extending construction of the self-extending neural-network by adding a new intermediate layer to said intermediate layer, said new intermediate layer, being composed of said new node.

12. The self-extending shape neural-network as claimed in claim 9, wherein the self-extending neural-network is a multilayer neural-network composed of an input layer, an intermediate layer, and an output layer;

and

said self-extending portion means self-extending construction of the self-extending neural-network by adding a new output layer as an upper layer of said output layer, said output layer upstream of said upper layer being provided as a new intermediate layer, thereby adding an additional intermediate layer to said intermediate layer.

13. The self-extending shape neural-network as claimed in claim 8, wherein the self-extending neural-network is a multilayer neural-network composed of an input layer, an intermediate layer, and an output layer;

and

said new node is provided within said intermediate layer when said self-extending portion means self-extends construction of the self-extending neural-network.

14. The self-extending shape neural-network as claimed in claim 8, wherein the self-extending neural-network is a multilayer neural-network composed of an input layer, an intermediate layer, and an output layer;

and

said self-extending portion means self-extending construction of the self-extending neural-network by adding a new intermediate layer to said intermediate layer, said new intermediate layer being composed of said new node.

15. The self-extending shape neural-network as claimed in claim 8, wherein the self-extending neural-network is a multilayer neural-network composed of an input layer, an intermediate layer, and an output layer;

and

said self-extending portion means self-extending construction of the self-extending neural-network by adding a new output layer as an upper layer of said output layer, said output layer upstream of said upper layer being provided as a new intermediate layer, thereby adding an additional intermediate layer to said intermediate layer.

16. A method for self-extending a neural-network comprising the steps of:

(a) receiving a studying sample

(b) calculating an output value by performing a study operation using a given algorithm upon the studying sample and a value of a coupling weight among respect nodes;

(c) externally outputting the output value;

(d) determining whether the study operation is progressing in accordance with the output value and the value of the coupling weight between the nodes;

(e) outputting an extend instruction signal when said step (d) has determined that the study operation is not progressing;

(f) providing a new node in response to the extend instruction signal;

(g) establishing a condition of coupling between the nodes and the new node; and

(h) establishing an initial value for a coupling weight between the nodes and the new node, thereby enabling further study operations after self-extension.

17. The method as claimed in claim 16 further comprising the steps of:

(i) determining whether the study operation is focused to a result;

(j) outputting a focus signal when said step (i) determines that the study operation is not focused;

(k) renewing the value of the coupling weight between the nodes when the focus signal is produced; and

(l) recalculating the output value with the renewed value when the focus signal is produced.

18. The method as claimed in claim 16, wherein said step (f) provides the new node within an intermediate layer, the intermediate layer performing the calculations of said step (b).