High order information processing method by means of a neural network and minimum and maximum searching method therefor

• US 5,153,923 A
• Filed: 11/22/1989
• Issued: 10/06/1992
• Est. Priority Date: 03/25/1988
• Status: Expired due to Fees

• Current Assignee: Hitachi, Ltd.
• Original Assignee: Hitachi, Ltd.
• Inventors: Minami, Keiko, Matsuba, Ikuo
• Primary Examiner(s): Boudreau, Leo H.
• Application Number:
  US07/445,840
• Time in Patent Office: 1,049 Days
• Field of Search: 382/14, 382/15, 364/513
• US Class Current: 382/158
• CPC Class Codes: G06N 3/04 Architecture, e.g. interconnection topology

Abstract

In order to improve the problematical points concerning the structure and the processing speed of a prior art neural network, the optimum structure of the neural network, in which a synapse structure constructed on the basis of living body physiological knowledge or presumed therefrom is determined to make it possible to realize high level information processing functions such as feature extraction, feature unification, memory, etc. Applications to an image recognition, a movement control, etc. making the most of the robust recognizing power thereof, or application to an optimum problem, a large scale numerical analysis, etc. making the most of the parallel processing power thereof are made possible.

Claims

1. A high order information processing method, as performed by a neural network, comprising the steps of:

performing at least one or a combination of a plurality of various information processings including a feature extracting processing, a feature unifying processing, a memory processing, a recognizing processing, and a control information generating processing by arranging an internal structure of said neural network based on physiological knowledge information of a living body;

arranging said internal structure of said neural network according to an intended information processing based on said physiological knowledge information; and

effecting parallel concurrent and competitive actions of a group of neural elements constituting said neural network internally arranged by said arranging step.

2. A high order information processing method, as performed by a neural network, according to claim 1, wherein when said feature extracting processing is performed said method further comprises the steps of:

causing a number of neural elements in said neural network to be arranged in a layered structure having a plurality of layers; and

eliminating noises, which are unnecessary for feature propagation, by making a value proportional to an average value of a state of a group of neural elements in each of the layers propagate to a group of neural elements located in a next higher layer at the same time features are extracted hierarchically one after another.

3. A high order information processing method, as performed by a neural network, according to claim 2, wherein said value proportional to the average value is used to extract the features hierarchically one after another;

and

wherein said method further comprises the steps of;

defining a probability distribution for general synapse couplings between the neural elements having a linearity or a non-linearity in each of the layers;

executing an operation by integrating a high frequency component of the probability distribution; and

keeping probability of coupling between adjacent neural elements invariant.

4. A high order information processing method, as performed by a neural network, according to claim 1, wherein when said feature unifying processing is performed said method comprises the steps of:

matching inputted features with high order information or concept information through concurrent and competitive actions;

arranging neural layers constituted by groups of neural elements corresponding to features extracted by feature extracting processing in lower layers;

arranging a group of middle layers having groups of neural elements corresponding to information combining for the features extracted; and

arranging neural elements corresponding to a feature to be recognized or a concept thereof from combined features in the uppermost neural layer;

wherein depending on a degree of relation between neural elements within each layer or between different layers, a large positive value is assigned to a synapse coupling between the neural elements, if the relation is strong, and a negative value is assigned thereto, if there is no relation.

5. A high order information processing method, as performed by a neural network, according to claim 4, wherein said feature unifying processing comprises the steps of:

processing, including solving a problem of minimizing energy consisting of a product of a synapse coupling between neural elements and a second order term of the state of said neural elements and a product of a threshold value and the state of said neural elements, the neural elements having a function that if all information inputted in one of said neural elements is greater than a second threshold value, the one neural element is ignited and if the information is smaller, then the one element is placed in pause state.

6. A high order information processing method, as performed by a neural network, according to claim 5, wherein said processing of solving said problem of minimizing energy includes the step of:

processing, by a hypothetical neural network, in which synapse coupling between neural elements is constant and the threshold value depends on a square root of an original synapse coupling, of introducing a new variable taking a continuous value and deducing a new energy consisting of a second order term of the new variable and a product of the new variable, a square root of the synapse coupling and an original state of the neural elements.

7. A high order information processing method, as performed by a neural network, according to claim 1, wherein when said memory processing is performed said method comprises the steps of:

processing, by which high order processing is stored for a long term by coding it as a value of a synapse coupling between neural elements, by memorizing the high order processing necessary for feature unifying processing by learning;

arranging neural elements corresponding to high order information in an input layer, the neural elements arranged in a number of intermediate layers for transmitting information through the synapse coupling; and

modifying the synapse coupling, depending on the difference between output information from an output layer and target information.

8. A high order information processing method, as performed by a neural network, according to claim 7, wherein said memory processing is executed to thereby provide memory functions having different mechanisms corresponding to a long term memory and a short term memory in accordance with physiological knowledge information.

9. A high order information processing method, as performed by a neural network, according to claim 8, wherein said long term memory is based on a synapse coupling structure which is based on said physiological knowledge information including statistical experimental physiological facts which includes information indicating dependence of a diameter of a tree-shaped protrusion on a distance of a position of attachment of the protrusion from a center of a cell body and information indicating dependence of a number of synapses attached to the tree-shaped protrusion on the diameter of the tree-shaped protrusion at the position of the attachment.

10. A high order information processing method, as performed by a neural network, according to claim 9, wherein said structure of synapse coupling is so determined that an amount of information, which can be transmitted by the coupling between a predetermined neural element and the closest adjacent neural element decreases successively through each succeeding neural element, in such a manner that a coupling between the predetermined neural element and a second adjacent element, after the closest adjacent neural element, transmits 1/2 of the amount of the information transmitted between the predetermined neural element and the closest adjacent neural element, and a coupling between said predetermined neural element and a third adjacent element, after the closest adjacent neural element transmits 1/4 of the amount of the information transmitted between the predetermined neural element and the closest adjacent neural element, on the basis of an optimization principle, by which a sum of energy consumption necessary for transmitting information through axons serving as information transmitting medium and a space occupied by the axons is minimized.

11. A high order information processing, as performed by a neural network, according to claim 8, wherein in order to determine suitable values for the synapse, said long term memory processing is executed by forming the synapse coupling between the input layer of the neural network and the next upper layer by a product of the input information and a random L variable, forming the synapse coupling between the output layer and the next lower layer by a product of the output information and a random variable, and forming the synapse coupling between the other layers by a product of two kinds of independent random variables, or means for generating the synapse couplings from two kinds of random variables and determining said random variables for a synapse coupling of the lower layer one after another, starting from a synapse coupled with the output layer, so that the difference between output information and target information is decreases.

12. A high order information processing method, as performed by a neural network, according to claim 8, wherein said short term memory processing is executed by making the memory correspond to minimum values of energy for the neural network or stationary states of dynamic equations deduced from said energy.

13. A high order information processing method, as performed by a neural network, according to claim 12, wherein said short term memory processing includes a processing for identifying physiological knowledge information that make stationary states of said dynamic equation appear by setting random values for all synapse couplings and make it possible to increase a number of stationary states to a certain finite number by increasing a number of synapse couplings coupled with neural elements, a state being chaotic, where no stable states can be realized, even if the number of couplings is increased, exceeding said finite number.

14. A high order information processing method, as performed by a neural network, according to claim 1, wherein said recognizing processing comprises the steps of:

processing, by which primitive features such as figures consisting of lines or edges of an image are extracted hierarchically by said feature extracting processing;

unifying said primitive features to obtain information of high order in a degree necessary for recognition in said feature unifying processing; and

matching said information obtained by said unifying step with high order information stored by previous learning in said memory processing.

15. A high order information processing method, as performed by a neural network, according to claim 5, wherein said step of processing includes the steps of:

processing an initial visual sensation such as movement direction recognition, depth recognition; and

processing to minimize a sum of energy based on a system of equations deduced spontaneously from a fact that a brightness of an image is kept unchanged with respect to movement and energy based on apriori limiting conditions for determining a solution.

16. A high order information processing method, as performed by a neural network, according to claim 1, wherein in a movement control such as the control of a robot manipulator by means of said neural network, in order to control an object following a dynamic equation structurally determined so that the trajectory thereof follows a target trajectory, at first a neural network for identifying the behavior of the object is newly constructed and the object is controlled by the neural network at a point of time, where the output of is equal to the output of the object.

17. A high order information processing method, as performed by a neural network, according to claim 4, wherein said processing steps comprise the steps of:

processing, by which a negative solution method is used for time finite differences, for which stability of a solution of non-stationary partial differential equations is secured;

effecting for a space a finite differentiation by using the finite difference-finite element method;

assigning finite-differentiated variables to neural elements constituting the neural network; and

executing minimization of energy deduced from equations by a network multiply layered, corresponding to the time finite difference.