Intelligent control with hierarchical stacked neural networks

• US 10,417,563 B1
• Filed: 04/07/2017
• Issued: 09/17/2019
• Est. Priority Date: 09/30/2002
• Status: Expired due to Term

• Current Assignee: Michael Lamport Commons, Mitzi Sturgeon White
• Original Assignee: Michael Lamport Commons, Mitzi Sturgeon White
• Inventors: Commons, Michael Lamport, White, Mitzi Sturgeon
• Primary Examiner(s): Smith, Paulinho E
• Application Number:
  US15/482,229
• Time in Patent Office: 893 Days
• CPC Class Codes: G06N 3/045 Combinations of networks
• CPC Class Codes: G06N 3/08 Learning methods

Abstract

An intelligent control system based on an explicit model of cognitive development (Table 1) performs high-level functions. It comprises up to O hierarchically stacked neural networks, N_m, . . . , N_m+(O−1), where m denotes the stage/order tasks performed in the first neural network, N_m, and O denotes the highest stage/order tasks performed in the highest-level neural network. The type of processing actions performed in a network, N_m, corresponds to the complexity for stage/order m. Thus N₁ performs tasks at the level corresponding to stage/order 1. N₅ processes information at the level corresponding to stage/order 5. Stacked neural networks begin and end at any stage/order, but information must be processed by each stage in ascending order sequence. Stages/orders cannot be skipped. Each neural network in a stack may use different architectures, interconnections, algorithms, and training methods, depending on the stage/order of the neural network and the type of intelligent control system implemented.

Claims

1. An artificial neural network system, comprising:

a sequentially stacked plurality of artificial neural network layers, implemented by at least one programmable automated processor, receiving an input and producing an output;

each respective artificial neural network layer having an architecture comprising an array of neurons receiving weighted input data from below the respective artificial neural network layer, a weighting of the input data to each respective neuron of the respective neural network layer being trained dependent on at least a respective layer training data set, to achieve a transformation based on at least the respective layer training data set, between a respective layer input from below the respective artificial neural network layer and a respective layer output of the respective artificial neural network layer, the respective layer training data set for each respective artificial neural network layer being different to achieve a different transformation;

a successively higher respective artificial neural network layer receiving inputs based on at least the transformation achieved by a respectively lower artificial neural network layer;

wherein the stacked plurality of artificial neural network layers operate sequentially to achieve successive stages of transformation, to achieve a plurality of levels of abstraction, each level of abstraction being dependent on the respective layer training data set and the architecture of the respective artificial neural network layer.

2. The artificial neural network system according to claim 1, wherein the input comprises image information.

3. The artificial neural network system according to claim 1, wherein the input comprises video information.

4. The artificial neural network system according to claim 1, wherein the input comprises speech information.

5. The artificial neural network system according to claim 1, wherein the input comprises natural language information.

6. The artificial neural network system according to claim 1, wherein the input comprises information that changes over time, and the output changes over time corresponding to changes in the input.

7. The artificial neural network system according to claim 1, wherein each of the stacked plurality of artificial neural network layers have a plurality of respective layer outputs that each represent a nonarbitrary organizations of actions.

8. The artificial neural network system according to claim 1, wherein at least one of the stacked plurality of artificial neural network layers is configured as a feed forward artificial neural network.

9. The artificial neural network system according to claim 1, wherein at least one of the stacked plurality of artificial neural network layers receives a feedback signal feedback from at least one respectively higher stacked plurality of artificial neural network layer.

10. The artificial neural network system according to claim 9, wherein the at least one of the stacked plurality of artificial neural network layers which receives the feedback signal is effective to persistently modify a weighting of the input data in a the respective at least one of the stacked plurality of artificial neural network layers.

11. The artificial neural network system according to claim 1, wherein at least one respective artificial neural network layer produces a respective layer output that corresponds to at least one of a centroid, a physical dimension, and a physical coordinate.

12. The artificial neural network system according to claim 1, wherein the stacked plurality of artificial neural network layers is trained with the training data set according to a backward-propagation algorithm.

13. The artificial neural network system according to claim 1, wherein the stacked plurality of artificial neural network layers comprises an adaptive resonance neural network.

14. The artificial neural network system according to claim 1, wherein the stacked plurality of artificial neural network layers is configured to cluster information patterns.

15. The artificial neural network system according to claim 1, wherein the stacked plurality of artificial neural network layers has at least one respective artificial neural network layer in which a weighting of the input data is derived in an unsupervised manner.

16. The artificial neural network system according to claim 1, wherein the plurality of levels of abstraction comprise at least three different orders of hierarchical complexity.

17. A method of processing data with a hierarchically stacked plurality of artificial neural network layers, comprising:

receiving an input to the stacked plurality of artificial neural network layers, the stacked plurality of artificial neural network layers being implemented on at least one automated processor;

processing the received input by the stacked plurality of artificial neural network layers, each respective artificial neural network layer having an architecture comprising an array of neurons receiving weighted input data from a hierarchical level below the respective artificial neural network layer, a weighting of the input data to each respective neuron of the respective artificial neural network layer being trained dependent on at least a training data set, to achieve a transformation based on at least the training data set, between a respective layer input from the hierarchical level below the respective artificial neural network layer and a respective layer output of the respective artificial neural network layer, a successively hierarchically higher respective artificial neural network layer receiving inputs based on at least a respectively hierarchically lower artificial neural network layer, wherein the stacked plurality of artificial neural network layers operate sequentially to achieve a plurality of levels of abstraction, the respective layer training data set for each respective artificial neural network layer being different to achieve a different transformation; and

producing an output the stacked plurality of artificial neural network layers which differs from the input by a plurality of levels of abstraction, each level of abstraction being dependent on the respective layer training data set and the architecture of the respective artificial neural network layer.

18. The method according to claim 17, further comprising:

producing at least one external prompt for a human, from the stacked plurality of artificial neural network layers;

receiving a natural language response to the prompt from the human, to the stacked plurality of artificial neural network layers;

processing the received natural language response in the stacked plurality of artificial neural network layers; and

producing the output selectively in dependence on at least the received natural language response and the input.

19. An artificial neural network, comprising:

a plurality of artificial neural network layers implemented by at least one automated processor, each artificial neural network layer having an architecture comprising at least at least one hidden layer comprising an array of hidden layer neurons respectively receiving information and a set of connection weights;

the set of connection weights for each respective artificial neural network layer being generated based on at least a respective set of training information, each of the plurality of artificial neural network layers being implemented to produce an output which achieves a predefined level of abstraction for the respective level, selectively based on at least the respective set of training information and the respective architecture;

the plurality of artificial neural network layers each having a respectively different level of abstraction from preceding or succeeding layers, acting sequentially, having information transferred based on a processing of at least one first array of hidden layer neurons to at least one distinct second array of hidden layer neurons.

20. The artificial neural network according to claim 19, further comprising:

an input layer configured to receive information from outside of the plurality of artificial neural network layers; and

an output layer configured to externally communicate abstracted information selectively dependent on at least the received information from outside of the plurality of artificial neural network layers and the sets of connection weights of the respective plurality of artificial neural network layers.