Self-organizing circuits

• US 4,774,677 A
• Filed: 09/24/1986
• Issued: 09/27/1988
• Est. Priority Date: 08/06/1981
• Status: Expired due to Fees

• Current Assignee: Bruce S. Buckley
• Original Assignee: Bruce S. Buckley
• Inventors: Buckley, Bruce S.
• Primary Examiner(s): Goldberg, E. A.
• Assistant Examiner(s): DONOVAN, LINCOLN D
• Application Number:
  US06/911,144
• Time in Patent Office: 734 Days
• Field of Search: 382/14-15, 382/33, 382/35-37, 382/49, 364/105, 364/513, 364/148, 364/421, 367/135
• US Class Current: 706/23
• CPC Class Codes: G09B 25/02 of industrial processes; of machinery

Abstract

Self-organizing circuits to receive input signals from transducers and the like and operable, under guidance from modifying inputs thereto, to operate in a learning mode and systems embodying such self-organizing circuits. The circuits include mechanisms to change the modifying inputs based upon Boolean functions and further mechanisms to change the modifying inputs based upon competition among the input signals.

Claims

1. A system that comprises:

transducer means to sense an input thereto and operable to provide a first output signal representative of said input;

self-organizing circuit means connected to receive said first output signal and operable to effect analysis of constituent elements of said input and provide a further output signal related to said input, each self-organizing circuit of said selforganizing circuit means including a plurality of sub-circuits or levels which are connected to receive messages from some other sub-circuit or level and are operable to deliver messages to some other sub-circuit or level, there being voting in that each subcircuit or level weights relative amounts of negative and positive inputs thereto and votes whether or not to pass along information to the further sub-circuits or levels;

means connected to apply modifying inputs to each subcircuit or level of the self-organizing circuit means, which modifying inputs serve to correct errors in the further output signal occasioned by accurate or inaccurate analysis of the input by the self-organizing circuit means;

means connected to change said modifying inputs based upon cumulative Boolean functions between said messages; and

means connected to change said modifying inputs based upon competition amongst said messages.

2. A system as claimed in claim 1 in which the transducer means is a sensor operable to accept light containing information as an electrical analog signal and adapted to convert that analog signal to a binary digital signal.

3. A system as claimed in claim 1 in which each subcircuit or level provides path reinforcement to the other sub-circuits or levels.

4. A system as claimed in claim 1 in which the selforganizing circuit further includes a preprocessor circuit connected to receive said first output signal as input thereto, which preprocessor is adapted to process said input to provide better prior organization of said first output signal to improve or simplify the analysis of constituent elements thereof, the thus-organized signal being connected as input to said plurality of sub-circuits or levels, the connection being such that the inputs to the successive sub-circuits or levels are serial in that a signal to a particular sub-circuit or level is modified by that particular sub-circuit or level and then passed along to successive sub-circuits or levels, or the signal is passed along to successive sub-circuits or levels without modifications.

21. A system according to claim 1 in which each selforganizing circuit of the self-organizing circuit means is an analog circuit and in which voting is implemented by a multipleinput voting circuit that receives input electrical currents and produces an output that is proportional to the sum of the input currents, the output of the analog circuit being a function of said sum.

22. A system according to claim 21 in which each selforganizing circuit includes means to effect path reinforcement.

23. A system according to claim 1 in which each sub-circuit or level comprises a plurality of DACs and associated voting circuits combined to form a sub-circuit or level, said DACs including path-related DACs which add electric current to the voting circuit associated therewith and threshold DACs which draw current from the associated voting circuit, the outputs of the plurality of voting circuits being the output of the sub-circuit or level and being combined with like outputs from other subcircuits or levels to form said further output signal.

24. A system according to claim 23 in which the means connected to apply modifying inputs to the self-organizing circuit means is a computer connected to receive said further output signal and programmed to interact with the self-organizing circuit means in a learning mode, said computer being operable to provide learning signals, recognize signals, punish signals and reward signals appropriately applied as input to the self-organizing circuit means.

25. A system according to claim 24 that includes a preprocessor that is connected to receive said first output signal as input thereto, which preprocessor is adapted to process said input to provide better prior organization of said first output signal to improve or simplify the analysis of constituent elements thereof.

26. A system according to claim 23 in which the outputs of a particular level circuit are monitored by the computer, said outputs being representative of said first output signal and, ultimately, the input to the transducer means, any error in said further output signal being corrected by punishing the subcircuit or level from which the error emanated and a desired output in said further output signal being rewarded.

27. A system according to claim 26 wherein each pathrelated DAC comprises a binary up-down counter, punishment being affected by reducing the value of the binary numbers in the counter and a reward being effected by increasing the value of the binary numbers in the counter, thereby respectively to decrease or increase electric current flow into the voting circuit associated with the particular DAC.

28. A system according to claim 27 wherein each threshold DAC comprises a binary up-down counter whose count is increased or decreased respectively to draw less or more current from the voting circuit associated therewith in the course of a learning experience.

29. A system according to claim 1 in which each sub-circuit or level comprises a plurality of positive-negative DACs, each positive-negative DAC being associated with a voting circuit and being adapted to apply and modify electric current to the associated voting circuit.

30. A system according to claim 29 in which each said DAC includes a binary up-down counter which applies positive current flow for positive values in the counter and negative flow for negative values in the counter, which system includes a computer to apply, during a learning mode, reward signals by adding binary bits to increase the value of the binary numbers stored in the counter and punishment signals by subtracting binary bits from the binary number stored in the counter.

42. A system according to claim 1 wherein feedback means is provided to permit feedback signals to flow from the output of a higher sub-circuit or level to the input of a lower sub-circuit or level to enhance analysis of said first output signal.

43. A system according to claim 42 in which initial weights at each sub-circuit or level are randomly applied and in which said modifying inputs include punish and reward signals, said system including means to apply said punish and reward signals.

5. A system that comprises:

transducer means to sense as input thereto a signal containing information, said transducer means being operable to convert that input to a first output signal representative of said input;

self-organizing circuit means connected to receive said first output signal and operable to effect analysis of the pattern of constituent elements thereof and provide a second output signal shaped, at least in part, by said information, said selforganizing circuit means comprising multi-levels with punish and reward signals to all levels, feedback signals at all levels and random initial output weighting;

means connected to apply modifying inputs to the selforganizing circuit means at all levels, which modifying inputs serve to correct errors in the second output signal pattern occasioned by inaccurate analysis by the self-organizing circuit means, which modifying inputs include said punish and reward signals;

means connected to change said modifying inputs based upon cumulative Boolean functions among intermediate signals and said first output signal; and

means connected to change said modifying inputs based upon competition among said intermediate signals and said first output signal.

34. A system according to claim 5 wherein the self-organizing circuit means includes a plurality of DACs to permit weighting, weighting being achieved by introducing binary numbers to the DACs.

35. A system according to claim 34 wherein the DACs are positive/negative DACs in that each DAC comprises a positive/negative counter whose count is affected by said binary numbers.

6. A system that comprises:

self-organizing circuit means connected to receive as input information in the form of a signal and operable to effect analysis of the pattern of constituent elements thereof to extract said information from the input signal and provide an output signal related to said information, said self-organizing circuit means comprising multi-level circuits wherein, the information flows from a lower-level circuit to a higher-level circuit; and

means connected to apply modifying inputs to the self-organizing circuit means, which modifying inputs serve to correct errors in the output signal pattern occasioned by inaccurate analysis of the self-organizing circuit means, which modifying circuit means include punish and reward signals;

means connected to change said modifying inputs based upon cumulative Boolean functions between the input signals to each said multilevel circuit and its outputs; and

means connected to change said modifying inputs based upon competition among said input signals.

7. A system as claimed in claim 6 in which said signals are an electrical signals.

36. A system according to claim 6 wherein feedback paths are provided to permit feedback signals to flow from the output of the higher-level circuit to the input to the lower-level circuit to enhance analysis of the pattern.

37. A system according to claim 36 wherein the multi-level circuits consist of several circuits that proceed from a lowest level circuit to an Nth or highest level circuit, wherein each level circuit includes a binary-weighted DAC which comprises a binary digital counter wherein weighting of said output signal is achieved by controlling a binary-digital input to said counter.

38. A system according to claim 37 wherein initial weighting is achieved by means that loads the counter with small, but random, binary numbers, the punish and reward signals being applied by either adding or subtracting numbers from the counter to achieve learning.

39. A system according to claim 38 that includes an array of optical sensors to receive an optical pattern and operable to convert the optical pattern to an electrical signal representative of the optical pattern, a preprocessor circuit connected to receive the electrical signal and operable to convert the raw sensor data into useful information, that, in turn, is inputted to the self-organizing circuit means.

40. A system according to claim 39 wherein the sensor comprises pixels and associated electrical circuitry that obey the nearest neighbor rule in that if a pixel has an active output and one of its nearest neighbors also has an active output, then an active signal appears at a predetermined connection from the preprocessor.

41. A system according to claim 40 in which the associated circuitry includes AND-gate means and OR-gate means to permit identification of primitive components of the optical pattern and the general locations of the primitive components with respect to each other.

8. A system that comprises:

self-organizing circuit means connected to receive a plurality of input signals representing constituent elements of input information and operable to effect identification of the pattern of constituent elements by combining the influence which each constituent element has on the pattern of constituent elements, said self-organizing circuit means comprising multi-levels with feedback from higher levels to lower levels to aid in identification of said pattern, said self-organizing circuit means being operable to provide an output signal related to said pattern;

correction means connected to modify said self-organizing circuit means by varying the influence which each constituent element has on the pattern of constituent elements as occasioned by accurate and inaccurate identification of said input information;

means connected to vary said influence based upon cumulative Boolean functions between the input signals to each said multilevel circuit and its outputs; and

means connected to vary said influence based upon competition among said input signals.

9. A system as in claim 8 in which said self-organizing circuit means comprises:

primarily analog means for combining the influence which each constituent element has on the pattern of constituent elements.

10. A system as in claim 8 in which said self-organizing circuit circuit means effects identification among a plurality of possible patterns of constituent elements of input information and provides a plurality of output signals based on said identification.

11. A system as in claim 8 in which said self-organizing circuit means uses primarily digital means for combining the influence which each constituent element has on the pattern of constituent elements.

12. A system as in claim 8 in which self-organizing circuit means receives time-varying input information and whose identification of the pattern of constituent elements of said input information also varies with time.

13. A system as in claim 8 in which said self-organizing circuit means effects said identification of said pattern by simultaneously combining the influence of constituent elements of said input informtion.

14. A system as in claim 8 in which said self-organizing circuit means effects said identification of said pattern by sequentially combining the influence of constituent elements of said input information.

15. A system as in claim 8 in which said corrective means by which the influence of each constitutive element has on the pattern of constituent elements is varied by non-electrical means including but not limited to pneumatic, hydraulic, chemical, optical, magnetic or electrostatic means.

16. A system as in claim 8 in which the input information is non-electrical and is tranduced to electrical input information by elecronic transducer means.

17. A system as in claim 8 in which a plurality of said self-organizing means are organized such that the output of one self organizing circuit becomes part of the input information for another self-organizing circuit and said correction means varies the influence which each constituent element has on the pattern of constituent elements of each said self-organizing circuit based on the accurate and inaccurate identification of input information of any of the plurality of self-organizing circuits.

18. A system as in claim 8 which is incorporated into a device which uses said identification of the pattern of constituent elements of input information to alter the input information received by the device.

19. A system as in claim 8 in which the initial influence which each constituent element has on the pattern of constituent elements is randomly chosen.

20. A system as in claim 8 in which the self-organizing circuit further includes preprocessor circuit connected to receive said plurality of input signals as input thereto, which preprocessor is adapted to process said input signals to provide constituent elements of said input information which improve or simplify the identification of said input information.

31. A system according to claim 8 in which the correction means includes means to deliver punish and reward signals to the self-organizing circuit means.

32. A system according to claim 31 wherein the punish and reward signals are universal punish and reward signals, that is, signals that are delivered simultaneously to all levels of said multi-levels.

33. A system according to claim 8 that includes weighting means that includes a plurality of DACs, wherein initial weighting is achieved by randomly loading the DACs and that includes reward and punish circuit means to add to and subtract from the binary numbers in the DACs on the basis of said identification.

44. A system according to claim 8 that includes acoustic transducer means to receive sound waves and operable to convert the same to an electrical signal, a preprocessor connected to receive the electric signal and operable to process the same to provide said plurality of input signals to the self-organizing circuit means which serves to recognize a sound pattern in the sound waves, said self-organizing circuit means including a mechanism by which electric current to a summing junction is modified during a learning process adapted to permit recognition of the sound pattern.

45. A system according to claim 8 in which said combining the influence which each constituent element has occurs at fixed time intervals.

46. A system according to claim 8 in which said each constituent element is delayed in time before combining the influence which each constituent element has on the pattern of constituent elements.

47. A system according to claim 8 in which said competition among input signals includes only some of all the possible input signals available from either a lower level circuit or fedback from a higher level circuit.

56. A system according to claim 8 in which said competition among input signals includes only some of all the possible input signals available from either a lower level circuit or fed back from a higher level circuit.

57. A system according to claim 8 in which the input signals represent at least one of static and time-varying constituent elements of the input information and in which the identification of the said pattern of constituent elements is achieved by combining the influence which each constituent element has on least one of a combinational and sequential pattern of the constituent elements

48. A system that comprises:

self-organizing circuit means connected to receive an input signal and operable to effect analysis of constituent elements of said signal and provide an output signal related to said input signal, each self-organizing circuit of said self-organizing circuit means including a plurality of sub-circuits or levels which are connected to receive messages from some other subcircuit or level and are operable to deliver messages to some other sub-circuit or level, there being voting in that each subcircuit or level weights relative amounts of negative and positive inputs thereto and votes whether or not to pass along information to the further sub-circuits or levels;

altering means connected to apply modifying inputs to each sub-circuit or level of the self-organizing circuit means, which modifying inputs serve to correct errors in the further output signal occasioned by accurate or inaccurate analysis of the input by the self-organizing circuit means;

Boolean logic means connected to change said modifying inputs based upon cumulative Boolean functions between said messages; and

means connected to change said modifying inputs based upon competition amongst said messages.

49. A system according to claim 48 in which each sub-circuit or level includes a plurality of neuron circuits (77a . . . 77n) which receive messages along first connections (25a . . . 25n, respectively), and deliver messages along second connections (26a . . . 26n), respectively)), said Boolean functions being generally of the form

50. A system according to claim 49 in which the Boolean logic means comprises two-input logic gates.

51. A system according to claim 50 in which the two-input logic gates include gates that provide AND functions, OR functions, NOR functions, NAND functions, XOR functions, EQV functions, IMP functions, NOT functions, IF functions, CON functions and TAUT functions.

52. A system that comprises:

transducer means to sense an input thereto and operable to provide a first output signal representative of said input;

self-organizing circuit means connected to receive said first output signal and operable to effect analysis of constituent elements of said input and provide a further output signal related to said input, each self-organizing circuit of said self-organizing circuit means including a plurality of sub-circuits or levels which are connected to receive messages from some other sub-circuit or level and are operable to deliver messages to some other sub-circuit or level, there being voting in that each sub-circuit or level weights relative amounts of negative and positive inputs thereto and votes whether or not to pass along information to the further sub-circuits or levels; and

altering means connected to apply modifying inputs to each subcircuit or level of the self-organizing circuit means, which modifying inputs serve to correct errors in the further output signal occasioned by accurate or inaccurate analysis of the input by the self-organizing circuit means;

said altering means including

Boolean logic means connected to change said modifying inputs based upon cumulative Boolean functions between said messages;

said altering means further including

maximizing means connected to change said modifying inputs based upon competition amongst said messages.

53. A method of self-organizing input information in the form of a signal and operable to effect analysis of the pattern of constituent elements thereof to extract said information from the signal and provide an output signal relating to the input information that comprises:

analyzing the constituent elements of said input signal and providing an output signal related to said input signal in a selforganizing circuit that includes a plurality of sub-circuits or levels which are connected to receive messages from some other sub-circuit or level and are operable to deliver messages to some other sub-circuit or level, there being voting in that each sub-circuit or level weights relative amounts of negative and positive inputs thereto and votes whether or not to pass along information to further subcircuits or levels; and

applying modifying inputs to each sub-circuit or level of the self-organizing circuit, which modifying inputs serve to correct errors in the output signal occasioned by accurate or inaccurate analysis of the input signal by the self-organizing circuit, said modifying inputs being based, in part, upon cumulative Boolean functions between said messages and said modifying inputs being based, in part, upon competition amongst said messages.

54. A method according to claim 53 in which said voting occurs at fixed time intervals.

55. A method according to claim 53 in which said received messages from some other sub-circuit or level is delayed in time before each sub-circuit or level weights relative amounts of negative and positive inputs thereto and votes whether or not to pass along information to further subcircuits or levels