Method for Computer-Aided Learning of a Neural Network and Neural Network

• US 20100088263A1
• Filed: 09/20/2006
• Published: 04/08/2010
• Est. Priority Date: 09/29/2005
• Status: Active Grant

• Current Assignee: Siemens AG
• Original Assignee: Siemens AG
• Inventors: Szabo, Miruna, Deco, Gustavo, Stetter, Martin
• Granted Patent: US 8,423,490 B2
• Time in Patent Office: Days
• US Class Current: 706/25
• CPC Class Codes: G06F 18/29 Graphical models, e.g. Bayesian networks
• CPC Class Codes: G06N 3/08 Learning methods

Abstract

There is described a method for computer-aided learning of a neural network, with a plurality of neurons in which the neurons of the neural network are divided into at least two layers, comprising a first layer and a second layer crosslinked with the first layer. In the first layer input information is respectively represented by one or more characteristic values from one or several characteristics, wherein every characteristic value comprises one or more neurons of the first layer. A plurality of categories is stored in the second layer, wherein every category comprises one or more neurons of the second layer. For one or several pieces of input information, respectively at least one category in the second layer is assigned to the characteristic values of the input information in the first layer. Input information is entered into the first layer and subsequently at least one state variable of the neural network is determined and compared to the at least one category of this input information assigned in a preceding step. The crosslinking between the first and second layer is changed depending on the comparison result from a preceding step.

Claims

1-18. -18. (canceled)

19. A method for computer-aided learning of a neural network, comprising:

providing a plurality of neurons;

dividing the neurons of the neural network into at least two layers, a first layer and a second layer crosslinked with the first layer,

wherein the crosslinking between the first layer and the second layer is represented by synaptic connections between the neurons,

wherein a strength of the connection is reflected by a weight, and

wherein the synaptic connections between a first neuron and a second neuron have a forwardly-directed connection from the first neuron to the second neuron and a backwardly-directed connection from the second neuron to the first neuron;

representing input information in the first layer respectively by one or more characteristic values from one or a plurality of characteristics, wherein at least one of the characteristic values comprises one or more neurons of the first layer, and wherein a plurality of categories are stored in the second layer, wherein the categories have one or more neurons of the second layer;

assigning at least one category in the second layer to the characteristic values of the input information in the first layer respectively for one or several pieces of input information;

entering an input information into the first layer;

determining at least one state variable of the neural network subsequent to the entering of the input information into the first layer;

comparing the determined state variable of the neural network with the assigned category in the second layer to the characteristic values of the input information in the first layer, wherein it is determined in the comparison if a conformity is present for the input information between the at least one state variable of the neural network and the assigned at least one category of the input information;

determining an activity of the neurons in the neural network;

classifying the neurons respectively as active or inactive as a function of their activity;

strengthening the weights of the synaptic connections between active neurons of the first layer and active neurons of the second layer, when a conformity is present for the input information between the at least one state variable of the neural network and the assigned at least one category of the input information;

weakening the weights of the forwardly-directed synaptic connections of first active neurons from one of the first and the second layers to second inactive neurons from the other one of the first and the second layers are weakened, when a conformity is present for the input information between the at least one state variable of the neural network and the assigned at least one category of the input information; and

weakening the weights of the synaptic connections between active neurons of the first layer and active neurons of the second layer are weakened, when a nonconformity is present for the input information between the at least one state variable of the neural network and the assigned at least one category of the input information.

20. The method as claimed in claim 19, wherein the categories of the second layer represent solutions of a task, wherein the solution of the task depends on the input information.

21. The method as claimed in claim 20, wherein the characteristics are divided into diagnostic characteristics relevant for the solution of the task, and into non-diagnostic characteristics not relevant for the solution of the task.

22. The method as claimed in claim 20, wherein every at least one category assigned of an input information represents a right solution of the task.

23. The method as claimed in claim 19, wherein a conformity between the at least one state variable of the neural network for an input information and the assigned at least one category of this input information is present, if the number of the active neurons which belong to the assigned at least one category of this input information exceeds a predetermined number as a function of the total number of the neurons in the assigned at least one category and/or the number of active neurons in other categories.

24. The method as claimed in claim 19, wherein when a conformity is present, the weights of all synaptic connections not strengthened or weakened, are not changed.

25. The method as claimed in claim 19, wherein when a conformity is not present the weights of all synaptic connections not weakened, are not changed.

26. The method as claimed in claim 19, wherein an evaluation of conformity and the assignment are carried out several times in an iterative manner.

27. The method as claimed in claim 26, wherein the iteration is terminated after reaching a convergence criterion.

28. The method as claimed in claim 26, wherein a normalization of the crosslinking of the neural network is carried out after carrying out the determination of the activity of the neurons in the neural network, classifying the neurons as active or inactive as a function of their activity and configuring the weights.

29. The method as claimed in claim 27, wherein a normalization of the crosslinking of the neural network is carried out after carrying out the determination of the activity of the neurons in the neural network, classifying the neurons as active or inactive as a function of their activity and configuring the weights.

30. The method as claimed in claim 19, wherein the crosslinked neurons of the first and second layer of the neural network are exciting pulsed neurons.

31. The method as claimed in claim 30, wherein the exciting pulsed neurons of the first layer are grouped at least partially into input pools, wherein at least one input pool is assigned to every characteristic value.

32. The method as claimed in claim 31, wherein the input pools cooperate with one another.

33. The method as claimed in claim 31, wherein activities of the input pool in each case represent a characteristic value.

34. The method as claimed in one of claim 30, wherein the exciting pulsed neurons of the second layer are grouped at least partially into category pools, wherein at least one category pool is assigned to every category.

35. The method as claimed in claim 34, wherein the category pools compete with one another, and wherein an active category pool prevails in the competition.

36. The method as claimed in claim 19, wherein the neural network contains inhibiting pulsed neurons, which form at least an inhibiting pool in the first and/or second layer, wherein the inhibiting pool exerts a global inhibition on the input and/or category pools.

37. A neural network, comprising:

providing a plurality of neurons;

a first layer having a plurality of neurons;

a second layer having a plurality of neurons, wherein the first layer and the second layer are crosslinked,

wherein the crosslinking between the first layer and the second layer is represented by synaptic connections between the neurons,

wherein a strength of the connection is reflected by a weight, and

wherein the synaptic connections between a first neuron and a second neuron have a forwardly-directed connection from the first neuron to the second neuron and a backwardly-directed connection from the second neuron to the first neuron; and

an input information in the first layer represented respectively by one or more characteristic values from one or a plurality of characteristics, wherein the characteristic value comprises one or more neurons of the first layer, and wherein a plurality of categories are stored in the second layer, wherein the category has one or more neurons of the second layer, wherein for one or several pieces of input information, respectively at least one category in the second layer is assigned to the characteristic values of the input information in the first layer, wherein an input information is entered into the first layer and subsequently at least one state variable of the neural network is determined and compared to the at least one category of the input information assigned to, wherein it is determined in comparison if a conformity is present for the input information between the at least one state variable of the neural network and the assigned at least one category of the input information, wherein the activity of the neurons in the neural network is determined and the neurons are respectively classified as active or inactive as a function of their activity, where, in the case that a conformity is present, the weights of the synaptic connections between active neurons of the first layer and active neurons of the second layer are strengthened and the weights of the forwardly-directed synaptic connections of first active neurons from one of the first and the second layers to second inactive neurons from the other one of the first and the second layers are weakened, and in the case that a conformity is not present, the weights of the synaptic connections between active neurons of the first layer and active neurons of the second layer are weakened.