Method for training a neural network

• US 6,968,327 B1
• Filed: 08/24/2000
• Issued: 11/22/2005
• Est. Priority Date: 08/26/1999
• Status: Expired due to Term

• Current Assignee: Wilex AG
• Original Assignee: Wilex AG
• Inventors: Schmitt, Manfred, Harbeck, Nadia, Kates, Ronald
• Primary Examiner(s): Hirl, Joseph P.
• Application Number:
  US10/049,650
• Time in Patent Office: 1,916 Days
• Field of Search: 706/21, 706/15, 706/16
• US Class Current: 706/21
• CPC Class Codes: A61B 5/00 Measuring for diagnostic purposes radiation diagnosis A61B6/00; diagnosis by ultrasonic, sonic or infrasonic waves A61B8/00; Identification of persons
• CPC Class Codes: A61B 5/415 the glands, e.g. tonsils, adenoids or thymus
• CPC Class Codes: A61B 5/418 lymph vessels, ducts or nodes
• CPC Class Codes: A61B 5/7267 involving training the classification device
• CPC Class Codes: G06N 3/08 Learning methods
• CPC Class Codes: G16H 50/70 for mining of medical data, e.g. analysing previous cases of other patients

Abstract

A method for training a neural network in order to optimize the structure of the neural network includes identifying and eliminating synapses that have no significant influence on the curve of the risk function. First and second sending neurons are selected that are connected to the same receiving neuron by respective first and second synapses. It is assumed that there is a correlation of response signals from the first and second sending neurons to the same receiving neuron. The first synapse is interrupted and a weight of the second synapse is adapted in its place. The output signals of the changed neural network are compared with the output signals of the unchanged neural network. If the comparison result does not exceed a predetermined level, the first synapse is eliminated, thereby simplifying the structure of the neural network.

Claims

1. A method for training a neural network in order to identify a patient risk function such that the structure of the neural network is simplified, wherein the neural network includes

an input layer having a plurality of input neurons that receive input data,

at least one intermediate layer having a plurality of intermediate neurons,

an output layer having a plurality of output neurons that provide output signals, wherein the output signals define the patient risk function following a first occurrence of a disease on the basis of given training data records including objectifiable and metrologically captured data relating to the medical condition of a patient, and

a multiplicity of synapses, wherein each said synapse interconnects a first neuron of a first layer with a second neuron of a second layer, defining a data sending and processing direction from the input layer toward the output layer,

wherein the method comprises:

identifying and eliminating synapses of the multiplicity of synapses that have an influence on the curve of the risk function that is less than a predetermined significance including

determining pre-change output signals of the neural network,

selecting first and second sending neurons that are connected to the same receiving neuron by respective first and second synapses,

assuming a correlation of response signals from said first and second sending neurons to the same receiving neuron,

interrupting the first synapse and adapting in its place the weight of the second synapse,

determining post-change output signals of the neural network,

comparing the post-change output signals with the pre-change output signals, and

eliminating the first synapse if the comparison result does not exceed a predetermined level.

2. The method of claim 1, wherein the first and second selected sending neurons are located on the same layer.

3. The method of claim 1, wherein interrupting the first synapse and adapting in its place the weight of the second synapse further includes adapting a value of a bias of the receiving neuron.

4. The method of claim 1, wherein identifying and eliminating synapses of the multiplicity of synapses that have an influence on the curve of the risk function that is less than a predetermined significance further includes

selecting a synapse, after determining the pre-change output signals of the neural network,

assuming that the selected synapse does not have a significant influence on the curve of the risk function,

interrupting the selected synapse, before determining the post-change output signals of the neural network and comparing the post-change output signals with the pre-change output signals, and

eliminating the selected synapse if the comparison result does not exceed the predetermined level.

5. The method of claim 4, further comprising

repeating the identifying and eliminating actions n times;

wherein comparing the post-change output signals with the pre-change output signals includes

comparing the post-change output signals with the pre-change output signals prior to performing the first identifying and eliminating actions, to provide a first comparison result; and

comparing the post-change output signals with the pre-change output signals after performing the n-1st identifying and eliminating actions, to provide a second comparison result;

wherein the comparison result is a cumulative comparison result including the first comparison result and the second comparison result.

6. The method of claim 1, further comprising

repeating the identify and eliminating actions n times;

wherein comparing the post-change output signals with the pre-change output signals includes

comparing the post-change output signals with the pre-change output signals prior to performing the first identifying and eliminating actions, to provide a first comparison result; and

comparing the post-change output signals with the pre-change output signals after performing the n-1st identifying and eliminating actions, to provide a second comparison result;

wherein the comparison result is a cumulative comparison result including the first comparison result and the second comparison result.

7. The method of claim 1, further comprising calculating a value of a likelihood function for the neural network to represent an expected output of the neural network.

8. The method of claim 1, further comprising comparing structure variants of the neural network using a significance test.

9. The method of claim 8, wherein the structure variants of the neural network are compared using a CHI-SQUARED test.

10. The method of claim 8, wherein the structure variants of the neural network are compared using a BOOT-STRAPPING method.

11. The method of claim 8, further comprising:

calculating a value of a likelihood function for the neural network;

wherein comparing structure variants of the neural network includes calculating a ratio of values of the likelihood functions for the structure variants.

12. The method of claim 1, further comprising optimizing strengths of connections between connected pairs of the neurons according to a simplex method.

13. A method for training a neural network in order to identify a patient risk function such that the structure of the neural network is simplified, wherein the neural network includes

an input layer having a plurality of input neurons that receive input data,

at least one intermediate layer having a plurality of intermediate neurons,

an output layer having a plurality of output neurons that provide output signals, wherein the output signals define the patient risk function following a first occurrence of a disease on the basis of given training data records including objectifiable and metrologically captured data relating to the medical condition of a patient, and

a multiplicity of synapses, wherein each said synapse interconnects a first neuron of a first layer with a second neuron of a second layer, defining a data sending and processing direction from the input layer toward the output layer,

wherein the method comprise:

identifying and eliminating synapses of the multiplicity of synapses that have an influence on the curve of the risk function that is less than a predetermined significance, including

determining pre-change output signals of the neural network,

selecting a synapse,

assuming that the selected synapse does not have a significant influence on the curve of the risk function,

interrupting the selected synapse,

determining posts-change output signals of the neural network,

comparing the post-change output signals with the pro-change output signals, and

eliminating the selected synapse if the comparison result does not exceed a predetermined level.

14. The method of claim 13, further comprising

repeating the identifying and eliminating actions n times;

wherein comparing the post-change output signals with the pre-change output signals includes

comparing the post-change output signals with the pre-change output signals prior to performing the first identifying and eliminating actions, to provide a first comparison result; and

comparing the post-change output signals with the pre-change output signals after performing the n-1st identifying and eliminating actions, to provide a second comparison result;

wherein the comparison result is a cumulative comparison result including the first comparison result and the second comparison result.