Method and apparatus for neural networking using semantic attractor architecture

• US 6,009,418 A
• Filed: 09/13/1996
• Issued: 12/28/1999
• Est. Priority Date: 05/02/1996
• Status: Expired due to Term

• Current Assignee: David L. Cooper
• Original Assignee: David L. Cooper
• Inventors: Cooper, David L.
• Primary Examiner(s): Hafiz, Tariq R.
• Assistant Examiner(s): RHODES, JASON W
• Application Number:
  US08/713,470
• Time in Patent Office: 1,201 Days
• Field of Search: 395/21, 395/22, 395/23, 395/24, 706/15, 706/16, 706/25, 706/26, 706/27, 706/29, 706/31, 706/44
• US Class Current: 706/15
• CPC Class Codes: G06N 3/045 Combinations of networks
• CPC Class Codes: G06V 10/768 using context analysis, e.g. recognition aided by known co-occurring patterns

Abstract

A semantic attractor memory uses an evolving neural network architecture and learning rules derived from the study of human language acquisition and change to store, process and retrieve information. The architecture is based on multiple layer channels, with random connections from one layer to the next. One or more layers are devoted to processing input information. At least one processing layer is provided. One or more layers are devoted to processing outputs and feedback is provided from the outputs back to the processing layer or layers. Inputs from parallel channels are also provided to the one or more processing layers. With the exception of the feedback loop and central processing layers, the network is feedforward unless it is employed in a hybrid back-propagation configuration. The learning rules are based on non-stationary statistical processes, such as the Polya process or the processes leading to Bose-Einstein statistics, again derived from considerations of human language acquisition. The invention provides rapid, unsupervised processing of complex data sets, such as imagery or continuous human speech, and a means to capture successful processing or pattern classification constellations for implementation in other networks.

Claims

1. A method for processing information ti an unsupervised manner, comprising:

using a feedforward neural network having a plurality of parallel multiple-layer channels, each comprising a multiple-layered set of nodes, with random connections from one layer to the next, to transform input arrays from prior layers or the environment into output arrays with fractal dimension for subsequent layers or output devices;

using at least one layer of said multiple-layer channels to process input information;

using a plurality of processing layers to process inputs from a plurality of said parallel multiple-layer channels;

feeding back information from at least one of said plurality of processing layers to a prior one of said processing layers;

using at least one output layer in an output channel of said plurality of multiple-layer channels to process outputs;

feeding back information from said at least one output channel back to said at least one processing layer;

using lateral connections from said parallel channels as inputs to said at least one processing layer; and

,

applying selective excitation and inhibition to nodes of said channels and using said nodes to apply learning rules which are based upon non-stationary statistical processes to inputs of said channels to create constellations of activations having fractal dimension.

2. The method according to claim 1, wherein said non-stationary statistical processes comprise a Polya process.

3. The method according to claim 1, wherein said non-stationary statistical processes comprise a process which employs Bose-Einstein statistics.

4. A neural network system for processing information in an unsupervised manner, comprising:

a feedforward architecture having a plurality of channel means, each comprising a plurality of layers of nodes with random connections from one layer to the next, for transforming input arrays from prior layers or the environment into output arrays having fractal dimension for subsequent layers or output devices, said plurality of channels further comprising;

at least one input layer means devoted to processing input information;

a plurality of processing layer means;

at least one feedback connection from one of said processing layer means to a prior one of said processing layer means;

at least one output layer means, in an output channel means of said plurality of channels, devoted to processing outputs;

means for providing feedback from said at least one output channel means back to said at least one processing layer means;

inputs from parallel channels to said at least one processing layer means; and

,

said channel means comprising means for applying selective excitation and inhibition to nodes and means for applying learning rules which are based upon non-stationary statistical processes to inputs of said channel means to create constellations of activations having fractal dimension.

5. The neural network system according to claim 4, wherein said non-stationary statistical process comprises a Polya process.

6. The neural network system according to claim 4, wherein said non-stationary statistical process comprises a process which employs Bose-Einstein statistics.

7. A neural network system for processing information in an unsupervised manner, comprising:

a feedforward architecture having a plurality of self-adjusting variable-width multiple-layer channel means, each comprising a multiple-layered set of nodes, for transforming input arrays from prior layers or the environment into output arrays of fractal dimension for subsequent layers or output devices, with random connections from one layer to the next;

at least one input layer means devoted to processing input information;

a plurality of processing layer means;

feedback connections from at least one of said processing layer means back to a prior processing layer means;

at least one output layer means, within an output channel means, devoted to processing outputs;

means for providing feedback from said at least one output channel means back to at least one of said plurality of processing layer means; and

,

inputs from parallel channels to said at least one processing layer means.

8. A method for processing information in an unsupervised manner, comprising the steps of:

using a feedforward neural network having a plurality of parallel multiple-layer channels, each comprising a multiple-layered set of nodes, with random connections from one layer to the next, to transform input arrays from prior layers or the environment into output arrays of fractal dimension for subsequent layers or output devices; and

,

applying selective excitation and inhibition to nodes of said channels and using said nodes to apply, to inputs of said channels, learning rules which are based upon non-stationary statistical processes, whereby constellations of activations having fractal dimension are created.

9. The method according to claim 8, wherein said non-stationary statistical process comprises a Polya process.

10. The method according to claim 8, wherein said non-stationary statistical process comprises a process which employs Bose-Einstein statistics.

11. A process for using a first neural network to program a second neural network, comprising:

using a plurality of parallel multiple-layer channels, each comprising a multiple-layered set of nodes, to transform input arrays from prior layers or the environment into output arrays of fractal dimension;

using at least one input layer to process inputs;

using outputs from at least one of said plurality of parallel multiple-layer channels as inputs to at least one processing layer having lateral connections;

using at least one output layer to process outputs;

using feedback connections to connect said at least one output layer and said at least one processing layer;

assigning connection weights to said feedback connections and said lateral connections;

providing target data to said input layers;

adjusting said connection weights;

varying said feedback connections and said lateral connections;

applying selective excitation and inhibition to nodes of said channels and using said nodes to apply learning rules which are based upon non-stationary statistical processes to inputs of said channels to create constellations of activations having fractal dimension;

recording at least one successful combination of connection weights, feedback connections, and lateral connections, which together comprise a set having fractal dimension; and

,

using said at least one successful combination to program a second neural network.

12. The method according to claim 11, wherein said successful combination comprises a feedforward combination.

13. The method according to claim 11, wherein said step of using said successful combination comprises using said successful combination to program a pattern classifier.

14. The method according to claim 11, wherein said step of using said successful combination comprises using said successful combination to program a microprocessor.

15. A system for using a first neural network to program a second neural network, comprising:

a first neural network, further comprising;

a plurality of multiple-layer channels, each comprising a multiple-layered set of nodes, for transforming input arrays from prior layers or the environment to output arrays of fractal dimension for subsequent layers or output devices;

at least one input layer;

at least one processing layer having lateral connections;

at least one output layer in at least one output channel;

feedback connections between said at least one output channel and said at least one processing layer;

means for assigning connection weights to said feedback connections and said lateral connections;

means for providing target data to said input layers;

means for evolving said connection weights;

means for varying said feedback connections and said lateral connections;

means for applying selective excitation and inhibition to nodes of said channels and using said nodes to apply learning rules which are based upon non-stationary statistical processes to inputs of said channels to create constellations of activations having fractal dimension;

means for recording at least one set of fractal dimension, said set comprising a successful combination of connection weights, feedback connections, and lateral connections; and

,

means for using said at least one successful combination to program a second neural network.

16. The system according to claim 15, wherein said second neural network comprises a pattern classifier.

17. The system according to claim 15, wherein said second neural network comprises a microprocessor.

18. The system according to claim 15, wherein said second neural network is complex relative to said first neural network, and wherein said at least one successful combination is used to program at least one capability among a larger set of capabilities for which said second neural network is programmed.

19. The system according to claim 15, wherein said at least one successful combination comprises a cascade.

20. A method for processing information, comprising:

using a feedforward neural network system to process said information in an unsupervised manner, further comprising the steps of;

providing a plurality of multiple-layer channels, each comprising a multiple-layered set of nodes, for transforming input arrays from prior layers or the environment to output arrays of fractal dimension for subsequent layers or output devices;

using at least one layer to process input information;

using at least one processing layer to process outputs of said input layer;

using at least one output layer in at least one output channel to process outputs;

feeding back data from said at least one output channel back to said at least one processing layer;

using outputs from parallel channels as inputs to said at least one processing layer; and

,

applying selective excitation and inhibition to nodes of said channels and using said nodes to apply learning rules which are based upon non-stationary statistical processes to inputs of said channel means to create constellations of activations with fractal dimensions.