Intelligent control with hierarchical stacked neural networks
First Claim
1. An artificial neural network system configured to receive input data and produce an abstracted output in dependence on the received input data, comprising:
- a plurality of successive artificial neural network layers, each respective successive artificial neural network layer being implemented by at least one automated processor and comprising an array of hidden layer neurons and a respective set of weights in a stacked architecture, the array of hidden layer neurons of a respective artificial neural network layer having a state dependent on at least a state of a preceding artificial neural network layer, and a respective set of connection weights to the preceding artificial neural network layer;
at least one artificial neural network layer further automatically receiving feedback from at least one succeeding artificial neural network layer;
each respective set of connection weights being dependent on at least training information, wherein the training information comprises a relationship of the received input data and abstract information represented in the respective received input data, wherein the feedback received from the at least one succeeding artificial neural network layer acts to modify at least one connection weight;
the artificial neural network system being implemented to achieve a predefined level of abstraction based on at least the training information modified based on the feedback;
wherein the arrangement of at least a respective array of hidden layer neurons, and the respective sets of connection weights define an architecture of the artificial neural network layer; and
the plurality of artificial neural network layers each having a different respective architecture and operating sequentially to achieve a plurality of levels of abstraction between the received input data and an artificial neural network output.
0 Assignments
0 Petitions
Accused Products
Abstract
An intelligent control system based on an explicit model of cognitive development (Table 1) performs high-level functions. It comprises up to O hierarchically stacked neural networks, Nm, . . . , Nm+(O−1), where m denotes the stage/order tasks performed in the first neural network, Nm, and O denotes the highest stage/order tasks performed in the highest-level neural network. The type of processing actions performed in a network, Nm, corresponds to the complexity for stage/order m. Thus N1 performs tasks at the level corresponding to stage/order 1. N5 processes information at the level corresponding to stage/order 5. Stacked neural networks begin and end at any stage/order, but information must be processed by each stage in ascending order sequence. Stages/orders cannot be skipped. Each neural network in a stack may use different architectures, interconnections, algorithms, and training methods, depending on the stage/order of the neural network and the type of intelligent control system implemented.
-
Citations
20 Claims
-
1. An artificial neural network system configured to receive input data and produce an abstracted output in dependence on the received input data, comprising:
-
a plurality of successive artificial neural network layers, each respective successive artificial neural network layer being implemented by at least one automated processor and comprising an array of hidden layer neurons and a respective set of weights in a stacked architecture, the array of hidden layer neurons of a respective artificial neural network layer having a state dependent on at least a state of a preceding artificial neural network layer, and a respective set of connection weights to the preceding artificial neural network layer; at least one artificial neural network layer further automatically receiving feedback from at least one succeeding artificial neural network layer; each respective set of connection weights being dependent on at least training information, wherein the training information comprises a relationship of the received input data and abstract information represented in the respective received input data, wherein the feedback received from the at least one succeeding artificial neural network layer acts to modify at least one connection weight; the artificial neural network system being implemented to achieve a predefined level of abstraction based on at least the training information modified based on the feedback; wherein the arrangement of at least a respective array of hidden layer neurons, and the respective sets of connection weights define an architecture of the artificial neural network layer; and the plurality of artificial neural network layers each having a different respective architecture and operating sequentially to achieve a plurality of levels of abstraction between the received input data and an artificial neural network output. - View Dependent Claims (2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18)
-
-
19. An artificial neural network method, comprising:
-
receiving input data through an input layer, comprising an array of input neurons; providing a plurality of stacked artificial neural network layers, each neural network layer comprising a hidden layer, comprising an array of hidden layer neurons respectively receiving information dependent on activity of a respectively lower layer of the plurality of stacked artificial neural network layers and a set of connection weights, at least one respective hidden layer receiving feedback from at least one respectively higher artificial neural network layer of the plurality of stacked artificial neural network layers, each respective hidden layer propagating an output through the respective set of connection weights to a respectively higher artificial neural network layer of the plurality of stacked artificial neural network layers, a lowest respective artificial neural network layer of the plurality of stacked artificial neural network layers receiving an input from the array of input neurons, a highest respective artificial neural network layer of the plurality of stacked artificial neural network layers producing an output through an output layer comprising an array of output neurons; and computing the set of connection weights based on at least training information, each of the plurality of stacked artificial neural network layers being implemented to achieve a predefined level of abstraction of information from a preceding neural network layer based on at least the training information modified by the feedback; wherein the arrangement of the input layer, hidden layer, and the set of connection weights define an architecture of each respective artificial neural network; the plurality of stacked artificial neural layers each having a different respective architecture defined by the respective hidden layer and the set of connection weights, and operating sequentially to achieve a plurality of different levels of abstraction.
-
-
20. An artificial neural network method, comprising:
-
providing a plurality of stacked artificial neural networks, each artificial neural network comprising;
an input layer, comprising an array of input neurons receiving input data; and
at least one hidden layer, comprising an array of hidden layer neurons respectively receiving information dependent on activity of the array of input neurons and a set of connection weights to an adjacent artificial neural network of the plurality of stacked artificial neural networks, and producing a respective output data pattern, the respective output data pattern of an inferior artificial neural network being received processed in accordance with the set of connection weights and passed to a superior artificial neural network, and at least one superior neural network providing a feedback signal to a respective inferior artificial neural network effective to modify an effect of the set of connection weights;the set of connection weights for each respective artificial neural network being defined based on at least training information, each of the plurality of stacked artificial neural networks being implemented to achieve a predefined level of abstraction based on at least the training information and modified by the feedback; wherein the arrangement of the input layer, hidden layer, and the set of connection weights define an architecture of each respective artificial neural network;
the plurality of stacked artificial neural networks each having have a different respective architecture defined by the respective array of neurons and the respective set of connection weights, and operating operate sequentially to achieve ascending levels of abstraction through the plurality of stacked artificial neural networks;receiving an external input to a most inferior of the plurality of stacked artificial neural networks; processing the external input by the plurality of stacked artificial neural networks; and conveying the respective output data pattern from the plurality of stacked artificial neural networks having a plurality of levels of abstraction from the external input according to the predetermined levels of abstraction for each respective artificial neural network and the feedback conveying an output from the second of the plurality of artificial neural networks at the second predefined level of abstraction.
-
Specification