Intelligent control with hierarchical stacked neural networks

• US 9,619,748 B1
• Filed: 09/03/2015
• Issued: 04/11/2017
• Est. Priority Date: 09/30/2002
• Status: Expired due to Term

• Current Assignee: Michael Lamport Commons, Mitzi Sturgeon White
• Original Assignee: Michael Lamport Commons, Mitzi Sturgeon White
• Inventors: Commons, Michael Lamport, White, Mitzi Sturgeon
• Primary Examiner(s): Vincent, David
• Assistant Examiner(s): Bharadwaj, Kalpana
• Application Number:
  US14/844,849
• Time in Patent Office: 586 Days
• CPC Class Codes: G06N 3/045 Combinations of networks
• CPC Class Codes: G06N 3/08 Learning methods

Abstract

An intelligent control system based on an explicit model of cognitive development (Table 1) performs high-level functions. It comprises up to O hierarchically stacked neural networks, N_m, . . . , N_m+(O−1), where m denotes the stage/order tasks performed in the first neural network, N_m, and O denotes the highest stage/order tasks performed in the highest-level neural network. The type of processing actions performed in a network, N_m, corresponds to the complexity for stage/order m. Thus N₁ performs tasks at the level corresponding to stage/order 1. N₅ processes information at the level corresponding to stage/order 5. Stacked neural networks begin and end at any stage/order, but information must be processed by each stage in ascending order sequence. Stages/orders cannot be skipped. Each neural network in a stack may use different architectures, interconnections, algorithms, and training methods, depending on the stage/order of the neural network and the type of intelligent control system implemented.

Claims

1. An artificial neural network system configured to receive input data and produce an abstracted output in dependence on the received input data, comprising:

a plurality of successive artificial neural network layers, each respective successive artificial neural network layer being implemented by at least one automated processor and comprising an array of hidden layer neurons and a respective set of weights in a stacked architecture, the array of hidden layer neurons of a respective artificial neural network layer having a state dependent on at least a state of a preceding artificial neural network layer, and a respective set of connection weights to the preceding artificial neural network layer;

at least one artificial neural network layer further automatically receiving feedback from at least one succeeding artificial neural network layer;

each respective set of connection weights being dependent on at least training information, wherein the training information comprises a relationship of the received input data and abstract information represented in the respective received input data, wherein the feedback received from the at least one succeeding artificial neural network layer acts to modify at least one connection weight;

the artificial neural network system being implemented to achieve a predefined level of abstraction based on at least the training information modified based on the feedback;

wherein the arrangement of at least a respective array of hidden layer neurons, and the respective sets of connection weights define an architecture of the artificial neural network layer; and

the plurality of artificial neural network layers each having a different respective architecture and operating sequentially to achieve a plurality of levels of abstraction between the received input data and an artificial neural network output.

2. The artificial neural network system according to claim 1, wherein the received input data comprises image information.

3. The artificial neural network system according to claim 1, wherein the received input data comprises video information.

4. The artificial neural network system according to claim 1, wherein the received input data comprises speech information.

5. The artificial neural network system according to claim 1, wherein the received input data comprises natural language information.

6. The artificial neural network system according to claim 1, wherein the received input data comprises information that changes over time.

7. The artificial neural network system according to claim 1, wherein at least one of the plurality of artificial neural network layers is configured to selectively respond to at least one joint domain pattern.

8. The artificial neural network system according to claim 1, wherein at least one of the plurality of artificial neural network layers is configured to receive supervised training.

9. The artificial neural network system according to claim 1, wherein a plurality of neurons within the array of hidden layer neurons of at least one layer of the artificial neural network system each receive an input dependent on a respective difference between outputs of other respective neurons and the respective set of weights.

10. The artificial neural network system according to claim 1, wherein the artificial neural network system is configured to perform processing actions of the input data of at least a sentential order of hierarchical complexity.

11. The artificial neural network system according to claim 1, wherein the artificial neural network system is configured to produce a set of information in the neural network output that corresponds to at least one of a centroid, a physical dimension, and a physical coordinate of an object within a space.

12. The artificial neural network system according to claim 1, wherein at least one of the plurality of successive artificial neural network layers is trained according to a backward-propagation algorithm.

13. The artificial neural network system according to claim 1, further comprising a plurality of feedback signals from a plurality of different succeeding artificial neural network layers directed to a respective artificial neural network layer, configured to modify at least two connection weights of the respective artificial neural network layer.

14. The artificial neural network system according to claim 1, wherein at least one of the plurality of artificial neural network layers comprises an adaptive resonance neural network.

15. The artificial neural network system according to claim 1, wherein at least one of the plurality of artificial neural network layers is configured to map excitation patterns present in the input data into clusters.

16. The artificial neural network system according to claim 1, wherein at least one of the plurality of artificial neural network layers has the respective set of connection weights derived from at least the training information in an unsupervised manner.

17. The artificial neural network system according to claim 1, wherein the plurality of levels of abstraction comprise at least three different orders of hierarchical complexity at or above the sentential order.

18. The artificial neural network system according to claim 1, wherein:

the input data comprises speech information from a caller;

at least one artificial neural network layer is configured to generate the artificial neural network output to selectively prompt a caller according to predetermined scripts; and

at least one artificial neural network layer is configured to generate the artificial neural network output to selectively match the caller to a destination, based on the speech information.

19. An artificial neural network method, comprising:

receiving input data through an input layer, comprising an array of input neurons;

providing a plurality of stacked artificial neural network layers, each neural network layer comprising a hidden layer, comprising an array of hidden layer neurons respectively receiving information dependent on activity of a respectively lower layer of the plurality of stacked artificial neural network layers and a set of connection weights,

at least one respective hidden layer receiving feedback from at least one respectively higher artificial neural network layer of the plurality of stacked artificial neural network layers,

each respective hidden layer propagating an output through the respective set of connection weights to a respectively higher artificial neural network layer of the plurality of stacked artificial neural network layers,

a lowest respective artificial neural network layer of the plurality of stacked artificial neural network layers receiving an input from the array of input neurons,

a highest respective artificial neural network layer of the plurality of stacked artificial neural network layers producing an output through an output layer comprising an array of output neurons; and

computing the set of connection weights based on at least training information, each of the plurality of stacked artificial neural network layers being implemented to achieve a predefined level of abstraction of information from a preceding neural network layer based on at least the training information modified by the feedback;

wherein the arrangement of the input layer, hidden layer, and the set of connection weights define an architecture of each respective artificial neural network;

the plurality of stacked artificial neural layers each having a different respective architecture defined by the respective hidden layer and the set of connection weights, and operating sequentially to achieve a plurality of different levels of abstraction.

20. An artificial neural network method, comprising:

providing a plurality of stacked artificial neural networks, each artificial neural network comprising;

an input layer, comprising an array of input neurons receiving input data; and

at least one hidden layer, comprising an array of hidden layer neurons respectively receiving information dependent on activity of the array of input neurons and a set of connection weights to an adjacent artificial neural network of the plurality of stacked artificial neural networks, and producing a respective output data pattern, the respective output data pattern of an inferior artificial neural network being received processed in accordance with the set of connection weights and passed to a superior artificial neural network, and at least one superior neural network providing a feedback signal to a respective inferior artificial neural network effective to modify an effect of the set of connection weights;

the set of connection weights for each respective artificial neural network being defined based on at least training information, each of the plurality of stacked artificial neural networks being implemented to achieve a predefined level of abstraction based on at least the training information and modified by the feedback;

wherein the arrangement of the input layer, hidden layer, and the set of connection weights define an architecture of each respective artificial neural network;

the plurality of stacked artificial neural networks each having have a different respective architecture defined by the respective array of neurons and the respective set of connection weights, and operating operate sequentially to achieve ascending levels of abstraction through the plurality of stacked artificial neural networks;

receiving an external input to a most inferior of the plurality of stacked artificial neural networks;

processing the external input by the plurality of stacked artificial neural networks; and

conveying the respective output data pattern from the plurality of stacked artificial neural networks having a plurality of levels of abstraction from the external input according to the predetermined levels of abstraction for each respective artificial neural network and the feedback conveying an output from the second of the plurality of artificial neural networks at the second predefined level of abstraction.