Neural network having an optimized transfer function for each neuron
First Claim
1. A neural network comprising a plurality of neurons each performing a signal processing corresponding to a neural element, said neurons being hierarchically connected in the order of at least one input layer, at least one hidden layer and an output layer, and said input layer, hidden layer and output layer each consisting of at least one neuron, individual neurons of said hidden layer and output layer being operative to correct the data weighted by multiplying the outputs of the preceding input layer or hidden layer by predetermined weighting data with a predetermined threshold value, respectively, and to substitute the data after the correction in a predetermined transfer function to calculate output data, said neural network including:
- error data calculation means operative to use a predetermined cost function to calculate error data from the output data of the output layer,weighting data correction means operative to partially differentiate said cost function with each of a plurality of weighting variables to obtain partial differentiated coefficients, and to correct said weighting data according to said partial differential coefficients.threshold value correction means operative to partially differentiate said cost function with each of a plurality of threshold variables to obtain partial differentiated coefficients, and to correct each said threshold value according to said partial differential coefficients, andcharacteristic data correction means for partially differentiating said cost function, to obtain partial differentiated coefficients, with at least one characteristic variable that determines the characteristics of the respective transfer functions of said hidden layer and said output layer, and operative to correct said characteristic data according to said partial differential coefficients.
1 Assignment
0 Petitions
Accused Products
Abstract
The characteristic data for determining the characteristics of the transfer functions (for example, sigmoid functions) of the neurons of the hidden layer and the output layer (the gradients of the sigmoid functions) of a neural network are learned and corrected in a manner similar to the correction of weighting data and threshold values. Since at least one characteristic data which determines the characteristics of the transfer function of each neuron is learned, the transfer function characteristics can be different for different neurons in the network independently of the problem and/or the number of neurons, and be optimum. Accordingly, a learning with high precision can be performed in a short time.
24 Citations
11 Claims
-
1. A neural network comprising a plurality of neurons each performing a signal processing corresponding to a neural element, said neurons being hierarchically connected in the order of at least one input layer, at least one hidden layer and an output layer, and said input layer, hidden layer and output layer each consisting of at least one neuron, individual neurons of said hidden layer and output layer being operative to correct the data weighted by multiplying the outputs of the preceding input layer or hidden layer by predetermined weighting data with a predetermined threshold value, respectively, and to substitute the data after the correction in a predetermined transfer function to calculate output data, said neural network including:
-
error data calculation means operative to use a predetermined cost function to calculate error data from the output data of the output layer, weighting data correction means operative to partially differentiate said cost function with each of a plurality of weighting variables to obtain partial differentiated coefficients, and to correct said weighting data according to said partial differential coefficients. threshold value correction means operative to partially differentiate said cost function with each of a plurality of threshold variables to obtain partial differentiated coefficients, and to correct each said threshold value according to said partial differential coefficients, and characteristic data correction means for partially differentiating said cost function, to obtain partial differentiated coefficients, with at least one characteristic variable that determines the characteristics of the respective transfer functions of said hidden layer and said output layer, and operative to correct said characteristic data according to said partial differential coefficients. - View Dependent Claims (2, 3, 4, 5, 6, 7, 8, 9, 10, 11)
-
Specification