Self-organizing data driven learning hardware with local interconnections

• US 20030055799A1
• Filed: 06/18/2002
• Published: 03/20/2003
• Est. Priority Date: 06/19/2001
• Status: Active Grant

• Current Assignee: Ohio University
• Original Assignee: Ohio University
• Inventors: Starzyk, Janusz A.
• Granted Patent: US 7,293,002 B2
• Time in Patent Office: Days
• US Class Current: 706/27
• CPC Class Codes: G06N 3/088 Non-supervised learning, e.g. competitive learning

Abstract

A method for organizing processors to perform artificial neural network tasks is provided. The method provides a computer executable methodology for organizing processors in a self-organizing, data driven, learning hardware with local interconnections. A training data is processed substantially in parallel by the locally interconnected processors. The local processors determine local interconnections between the processors based on the training data. The local processors then determine, substantially in parallel, transformation functions and/or entropy based thresholds for the processors based on the training data.

Claims

1. A data-driven self-organizing hardware system, comprising:

a plurality of self-organizing processors arranged in a learning array; and

a plurality of local interconnections that connect subsets of the plurality of self-organizing processors, where the local interconnections are chosen, substantially in parallel, by the self-organizing processors.

2. The system of claim 1, a self-organizing processor comprising:

a neuronal processor that facilitates partitioning an input data space;

one or more entropy based evaluators (EBEs) for determining the entropy in an input data space to support the neuronal processor partitioning the input data space; and

a memory for storing a value for access by one or more of the neuronal processor and the EBEs.

3. The system of claim 2, where the neuronal processor is a reduced instruction set processor.

4. The system of claim 2, a self-organizing processor comprising:

a first multiplexor for selecting between two or more data inputs to provide to one or more of the neuronal processor and the EBEs; and

a second multiplexor for selecting between two or more threshold-controlled inputs to provide to one or more of the neuronal processor and the EBEs, where the threshold-controlled inputs can selectively control the operation of one or more of the neuronal processor and the EBEs.

5. The system of claim 4, where the neuronal processor can selectively perform a transformation function on a data input to produce one or more of a data output and a threshold-controlled output, where the neuronal processor selectively performs the transformation function based, at least in part, on the data input.

6. The system of claim 5, where the transformation function is one of a linear, threshold, sigmoidal, exponential, logarithmic, and synaptic function.

7. The system of claim 6, where the self-organizing processor is programmed to locally select the transformation function.

8. The system of claim 4, where the self-organizing processor is programmed to locally select a subset of the local interconnections to retain and a subset of the local interconnections to release.

9. The system of claim 4, where the self-organizing processor is programmed to locally select an entropy based threshold to store in the memory.

10. The system of claim 4, where an entropy based evaluator comprises:

a lookup table;

an entropy calculating unit that determines an entropy value based, at least in part, on a value in the lookup table;

a threshold storage for storing the entropy value;

a maximum entropy storage for storing a maximum entropy value; and

a comparator unit that produces a comparator result by comparing the entropy value and the maximum entropy value.

11. The system of claim 10, where the neuronal processor selectively performs a transformation function on the data input to produce a data output based, at least in part, on the comparator result.

12. The system of claim 10, where the neuronal processor receives an information deficiency input.

13. The system of claim 12, where the neuronal processor selectively performs a transformation function on the data input based, at least in part, on the information deficiency input.

14. The system of claim 10, where the self-organizing processor produces an information deficiency output.

15. The system of claim 1, where the plurality of local interconnections are initially pseudo-randomly formed in the learning array with favoritism shown to local connections.

16. The system of claim 1, where the local interconnections convey into a self-organizing processor one or more of a data input, a threshold-controlled input, and an input information deficiency value.

17. The system of claim 1, where the local interconnections convey out of a self-organizing processor one or more of a data output, a threshold-controlled output, and an output information deficiency value.

18. A method for organizing artificial neurons to facilitate performing artificial neural network tasks, comprising:

pseudo randomly connecting a plurality of artificial neurons;

receiving a training data; and

locally determining, substantially in parallel, for an artificial neuron, one or more of, local interconnections between two or more artificial neurons, a transformation function, and an entropy based threshold, based, at least in part, on the training data.

19. The method of claim 18, where the local interconnection conveys into an artificial neuron one or more of, a data input, a threshold-controlled input, and an input information deficiency.

20. The method of claim 18, where the local interconnection conveys out of an artificial neuron one or more of, a data output, a threshold-controlled output, and an output information deficiency.

21. The method of claim 18, where the transformation function is one of, a linear function, a threshold function, an exponential function, a sigmoidal function, a logarithmic function, and a synaptic function.

22. The method of claim 18, comprising:

forming one or more associations between one or more partitions of the training data and one or more artificial neurons.

23. The method of claim 18, where the local interconnection is chosen in a manner that increases the likelihood that artificial neurons will be connected in local neighborhoods.

24. A method for classifying a data point in an input data space using a self organized, locally interconnected learning array, comprising:

receiving an input vector;

identifying one or more artificial neurons that code for the input vector; and

providing an indicator of the one or more associations.

25. A system for organizing a data-driven learning array, comprising:

means for establishing an initial pseudo random array of locally connected self-organizing processors; and

means for training the array of self-organizing processors to facilitate performing artificial neural network functions.

26. The system of claim 25, the means for training comprising:

means for locally selecting local interconnections between self-organizing processors;

means for locally selecting a transformation function for a self-organizing processor; and

means for locally selecting an entropy based threshold for a self-organizing processor.

27. A method for organizing artificial neurons to facilitate performing artificial neural network tasks, comprising:

pseudo randomly connecting a plurality of artificial neurons in a first array;

training up the artificial neurons to perform an artificial neural network task;

identifying a plurality of artificial neurons that code for the artificial neural network task;

identifying a plurality of connections between the identified artificial neurons; and

designing a second array that comprises the identified artificial neurons, the identified connections, and a pseudo random connection of a plurality of artificial neurons.