Apparatus for configuring neural network and pattern recognition apparatus using neural network

• US 5,546,503 A
• Filed: 08/13/1991
• Issued: 08/13/1996
• Est. Priority Date: 11/09/1990
• Status: Expired due to Fees

• Current Assignee: Hitachi, Ltd.
• Original Assignee: Hitachi, Ltd.
• Inventors: Takenaga, Hiroshi, Morooka, Yasuo, Kawakami, Junzo, Kayama, Masahiro, Abe, Shigeo, Takatoo, Masao
• Primary Examiner(s): Downs, Robert W.
• Application Number:
  US07/744,263
• Time in Patent Office: 1,827 Days
• Field of Search: 395/23, 395/24
• US Class Current: 706/25
• CPC Class Codes: G06F 18/24317 Piecewise classification, i.e. whereby each classification requires several discriminant rules
• CPC Class Codes: G06F 18/2451 linear, e.g. hyperplane
• CPC Class Codes: G06N 3/082 modifying the architecture, e.g. adding, deleting or silencing nodes or connections

Abstract

In a neural network having neurons connected in a multi-layer, firstly, input signal sets are sequentially entered to statistically process the outputs of hidden neurons and determine the optimum number of hidden neurons. Secondly, while changing the input signal entered to each input neuron to the maximum change limit, the change of output values of the other input neurons are checked to thereby determine an unnecessary input neuron. Thirdly, the weights between input neurons and hidden neurons are set to be in correspondence with a hyperplane to enable pattern recognition.

Claims

1. An apparatus for configuring a multi-layered neural network comprising:

neurons in each layer of the multi-layered configuration and synapses connecting said neurons between the layers for storing input/output training data sets, each including input data signals and corresponding desired output data signals; and

means for optimizing said neural network in accordance with said training data sets to satisfy given conditions which include a designated learning time period, an optimized neural network scale and minimized erroneous operations for input data other than said training data sets.

2. An apparatus for configuring a neural network according to claim 1, further comprising means for setting weights between input and hidden neurons to separate input patterns of different classes, in order to configure a pattern recognition network having an optimized neural network scale.

3. An apparatus for configuring a neural network according to claim 2, further comprising means for solving inequalities expressing the relationship of said weights between hidden and output neurons.

4. An apparatus for configuring a neural network according to claim 2, further comprising neural network optimizing design means for optimizing the design of said neural network.

5. An apparatus for configuring a neural network according to claim 4, wherein said neural network optimizing design means comprises hidden neuron number optimizing means for setting an optimum number of said hidden neurons.

6. An apparatus for configuring a neural network according to claim 5, wherein said hidden neuron number optimizing means operates to sequentially input said input data signals to said neural network which has stored said relative input/output training data sets to obtain the outputs of said hidden neurons, and in accordance with the analysis results of said outputs, to determine the number of said hidden neurons.

7. An apparatus for configuring a network according to claim 5, wherein said neural network optimizing designing means comprises said input data signal optimizing means and said hidden neuron number optimizing means, and said hidden neuron number optimizing means optimizes the number of said hidden neurons in accordance with said input data signals determined by said input data signal optimizing means.

8. An apparatus for configuring a neural network according to claim 4, wherein said neural network optimizing design means minimizes a product of the number of said input neurons and the number of said hidden neurons.

9. An apparatus for configuring a neural network according to claim 1, further comprising means for satisfying said given condition by minimizing the number of said neurons.

10. An apparatus for configuring a neural network according to claim 1, further comprising means for satisfying said given condition by minimizing the number of input neurons.

11. An apparatus for configuring a neural network according to claim 1, further comprising means for satisfying said given condition by minimizing the number of hidden neurons among hidden layers of said multi-layered neural network.

12. An apparatus for configuring a neural network according to claim 1, further comprising input data signal optimizing means for detecting an input data signal which has the least influence upon the performance of said network, from said input data signals, and setting the number of input neurons to the number obtained by subtracting the number of said detected input data signals from the number of said input data signals.

13. An apparatus for configuring a neural network according to claim 12, wherein said input data signal optimizing means includes means for evaluating the correlation between said input data signal and said output data signals and identifying an input data signal which does not contribute to a determination of an output data signal, and further means for evaluating the linear relationship between said input data signals and identifying an input data signal as a redundant input signal which can be linearly approximated by other input data signals.

14. An apparatus for configuring a neural network according to claim 12, wherein said input data signal optimizing means includes means for determining said output data signals while changing the value of each input data signal and identifying an input data signal which does not influence said output data signals.

15. An apparatus for configuring a neural network according to claim 12, wherein said training data sets and combinations of said input and output data signals used for the training of said neural network are prepared, a relationship between said input and output data signals set by said input data signal optimizing means is stored in said neural network, and in accordance with an evaluation result of said neural network, said input data signal optimizing means further detects an input data signal having least influence upon the performance of said network to determine the number of input neurons and deletes said input data signal having least influence upon the performance of said network.

16. An apparatus for configuring a neural network according to claim 1, further comprising means for terminating the learning of said neural network within a designated learning time period.

17. An apparatus for configuring a neural network according to claim 16, further comprising means for determining the number of input data signals and the number of learning cycles so as to terminate the learning within said designated learning time period.

18. An apparatus for configuring a neural network according to claim 1, further comprising means for minimizing the output errors for input data signals other than said training data set.

19. An apparatus for configuring a neural network according to claim 18, further comprising means for setting the weights between input and hidden neurons so as to separate input patterns, while satisfying said condition of minimizing said output errors.

20. An apparatus for configuring a neural network according to claim 1, wherein said means for optimizing said scale of said network includes means for minimizing the number of input neurons, hidden neurons and product of the number of input neurons and hidden neurons.