Adaptive neural learning system

• US 6,424,961 B1
• Filed: 12/06/1999
• Issued: 07/23/2002
• Est. Priority Date: 12/06/1999
• Status: Active Grant

• Current Assignee: Francisco José Ayala
• Original Assignee: Francisco José Ayala
• Inventors: Ayala, Francisco José
• Primary Examiner(s): Davis, George B.
• Application Number:
  US09/455,251
• Time in Patent Office: 960 Days
• Field of Search: 706/25, 706/16
• US Class Current: 706/25
• CPC Class Codes: G06N 3/08 Learning methods

Abstract

An adaptive integration network includes a plurality of interconnected neurons that are configured to fire when their excitation level, which is responsive to weighted input signals, is greater than or equal to a threshold. When two neurons fire in close temporal proximity, the weight of the connection is strengthened. Adaptive learning is induced by increasing the activity of the adaptive integration network, such as by lowering the threshold level.

Claims

1. A method of training a neural network, comprising:

presenting an input data set to the neural network to produce an output; and

increasing a number of firing neurons in the neural network until the output changes to a desired output.

2. The method according to claim 1, wherein:

the neural network includes a plurality of neurons interconnected by connections for transferring signals, each of said connections being associated with a weight; and

said increasing the network activity includes scaling the weight associated with the connections by a positive factor.

3. The method according to claim 1, wherein:

the neural network includes a plurality of neurons interconnected by connections for transferring signals having a magnitude in a firing state; and

said increasing the network activity includes increasing the magnitude of the signal in the firing state.

4. The method according to claim 1, wherein:

the neural network includes a plurality of interconnected neurons, said interconnected neurons including a plurality of data input neurons adapted to receive respective external signals; and

said increasing the network activity includes increasing a magnitude of the external signals.

5. The method according to claim 1, wherein:

the neural network includes a plurality of interconnected neurons, each of said interconnected neurons being configured to fire when a corresponding excitation level thereof is greater than or equal to a threshold; and

said increasing the network activity includes lowering the threshold.

6. The method according to claim 5, further comprising:

determining the excitation level of one of the neurons based on a weighted sum of input signals received over respective connections, said connections being associated with respective weights; and

adjusting each of the weights when said one of the neurons and a corresponding one of the others of the neurons fire within a prescribed time interval.

7. The method according to claim 6, wherein said adjusting includes adjusting said each of the weights to asymptotically converge to a line w′

=w for higher absolute values of the weight.

8. The method according to claim 1, further comprising increasing the network activity in response to a signal.

9. The method according to claim 8, further comprising:

providing the desired output data; and

comparing the desired output data and the output to generate the signal in response if the desired output data is not equal to the output.

10. A neural network comprising:

means for receiving an input data set;

means for producing an output based on the input data set; and

means for increasing a number of firing neurons in the neural network until the output changes to a desired output.

11. The neural network according to claim 10, wherein:

the neural network includes a plurality of neurons interconnected by connections for transferring signals, each of said connections being associated with a weight; and

said means for increasing the network activity includes means for increasing the weight associated with the connections.

12. The neural network according to claim 10, wherein:

the neural network includes a plurality of neurons interconnected by connections for transferring a signal in a firing state having a magnitude; and

said means for increasing the network activity includes means for increasing the magnitude of the signal in the firing state.

13. The method according to claim 10, wherein:

the neural network includes a plurality of interconnected neurons, said interconnected neurons including a plurality of data input neurons adapted to receive respective external signals; and

said means for increasing the network activity includes means for increasing a magnitude of the external signals.

14. The neural network according to claim 10,

wherein the neural network includes a plurality of interconnected neurons, each of said interconnected neurons being configured to fire when a corresponding excitation level thereof is greater than or equal to a threshold;

and

said means for increasing the network activity includes means for lowering the threshold.

15. The neural network according to claim 10, further comprising means for increasing the network activity in response to a signal.

16. The neural network according to claim 15, further comprising:

means for providing the desired output data; and

means for comparing the desired output data and the output to generate the signal in response if the desired output data is not equal to the output.

17. The neural network according to claim 10, further comprising:

a plurality of neurons; and

a plurality of excitatory and inhibitory connections interconnecting the neurons;

wherein some of the neurons are interconnected in a closed circuit of neurons linked by some of the excitatory connections in a same circular direction;

wherein the neurons of the closed circuit are configured to fire successively in cycles.

18. The neural network according to claim 17, wherein a number of the inhibitory connections I_opt is approximately:

$I_{\mathrm{opt}}=\frac{\mathrm{nk}}{L<br><br>\left(n,k\right)+1},$

wherein n represents a number of the neurons, k represents a number of connections per node, and L(n, k) is a function that indicates an expected average loop length.

19. A computer-readable medium bearing instructions for training a neural network, said instructions being arranged to cause one or more processors upon execution thereby to perform the steps of:

presenting an input data set to the neural network to produce an output; and

increasing a number of firing neurons in the neural network until the output changes to a desired output.

20. The computer-readable medium according to claim 19, wherein:

the neural network includes a plurality of neurons interconnected by connections for transferring signals, each of said connections being associated with a weight; and

said increasing the network activity includes scaling the weight associated with the connections by a positive factor.

21. The computer-readable medium according to claim 19, wherein:

the neural network includes a plurality of neurons interconnected by connections for transferring signals having a magnitude in a firing state; and

said increasing the network activity includes increasing the magnitude of the signal in the firing state.

22. The computer-readable medium according to claim 19, wherein:

the neural network includes a plurality of interconnected neurons, said interconnected neurons including a plurality of data input neurons adapted to receive respective external signals; and

said increasing the network activity includes increasing a magnitude of the external signals.

23. The computer-readable medium according to claim 19, wherein:

the neural network includes a plurality of interconnected neurons, each of said interconnected neurons being configured to fire when a corresponding excitation level thereof is greater than or equal to a threshold; and

said increasing the network activity includes lowering the threshold.

24. The computer-readable medium according to claim 23, wherein said instructions are further arranged to execute the steps of:

determining the excitation level of one of the neurons based on a weighted sum of input signals received over respective connections, said connections being associated with respective weights; and

adjusting each of the weights when said one of the neurons and a corresponding one of the others of the neurons fire within a prescribed time interval.

25. The computer-readable medium according to claim 24, wherein said adjusting includes adjusting said each of the weights to asymptotically converge to a line w′

=w for higher absolute values of the weight.

26. The computer-readable medium according to claim 19, wherein said instructions are further arranged to execute the step of increasing the network activity in response to a signal.

27. The computer-readable medium according to claim 26, wherein said instructions are further arranged to execute the steps of:

receiving the desired output data; and

comparing the desired output data and the output to generate the signal in response if the desired output data is not equal to the output.

28. A method of operating a neural learning system including a plurality of interconnected neurons, said interconnected neurons configured to fire when respective excitation levels thereof are greater than or equal to respective thresholds, said method comprising the steps of:

operating the neural learning system to produce a first output in response to an input;

lowering the thresholds from the respective thresholds and operating the neural learning system with the lowered threshold; and

restoring the thresholds to the respective thresholds, wherein further operation of the neural learning system after restoring the thresholds produces, in response to the input, a second output having a different value than the first output.

29. The method according to claim 28, further comprising the step of:

adjusting a weight associated with a connection between a pair of the interconnected neurons when the pair of interconnected neurons fired within a prescribed time interval.

30. A method of operating a neural learning system including a plurality of interconnected neurons, said interconnected neurons for transferring signals having a first magnitude in a firing state, said method comprising the steps of:

operating the neural learning system to produce a first output in response to an input;

increasing the magnitude in the firing state from the first magnitude to a second magnitude and operating the neural learning system with the increased magnitude in the firing state; and

restoring the magnitude in the firing state to the first magnitude, wherein further operation of the neural learning system after restoring the magnitude in the first state, in response to the input, a second output having a different value than the first output.