Self-organizing sequential memory pattern machine and reinforcement learning method

• US 8,504,493 B2
• Filed: 02/15/2011
• Issued: 08/06/2013
• Est. Priority Date: 02/15/2011
• Status: Active Grant

• Current Assignee: Intergraph Corporation (Hexagon AB)
• Original Assignee: Sigma Space Corp. (Hexagon AB)
• Inventors: Jones, Robert Linzey III
• Primary Examiner(s): Chaki, Kakali
• Assistant Examiner(s): Zidanic, Michael
• Application Number:
  US13/027,458
• Publication Number:
  US 20120209794A1
• Time in Patent Office: 903 Days
• Field of Search: None
• US Class Current: 706/12
• CPC Class Codes: G06N 20/00 Machine learning
• CPC Class Codes: G06N 20/20 Ensemble learning

Abstract

A self-organizing computing machine utilizes a method for mapping from a plurality of patterns contained within provided inputs to an invariant perception, distinguishable by a name or a label. The self-organizing computing machine includes a network of at least three nodes arranged in at least two hierarchical levels, at least one feature extractor, and at least one output unit arranged to interface the invariant perception. The nodes may include a reinforcement learning sub-network combined with an ensemble learning sub-network. The reinforcement learning sub-network may be arranged to receive at least two correlants, to determine a plurality of output values and to output the output values to the nodes of the higher level and the nodes of the lower level. Also, the ensemble learning sub-network may be arranged to receive and to combine output values from nodes of the higher level and nodes of the lower level.

Claims

1. A self-organizing computing machine for mapping from a plurality of patterns contained within at least one predetermined set of provided inputs to at least one invariant perception distinguishable by a name or a label among a plurality of categories, wherein the self-organizing computing machine comprises:

at least one network of at least three nodes interconnected by variable connections into at least two hierarchical node levels including at least a lower node level and a higher node level;

at least one feature extractor arranged to receive the at least one predetermined set of provided inputs, to process the at least one predetermined set of provided inputs to determine at least one hierarchical set of at least two correlants commensurate with at least two hierarchical correlant levels including at least a lower correlant level and a higher correlant level, and to communicate the determined hierarchical sets of at least two correlants to the at least two distinct nodes of the at least two distinct hierarchical node levels commensurate with the at least two correlants of the at least two distinct correlant levels such that the correlants of the lower correlant level communicate to the corresponding nodes of the lower node level and that the correlants of the higher correlant level communicates to the corresponding nodes of the higher node level; and

at least one output unit arranged to interface the at least one invariant perception distinguishable by a name, or a label, among the plurality of categories;

wherein, the at least one node at each hierarchical node level incorporate at least one reinforcement learning sub-network combined with at least one ensemble learning sub-network;

wherein, the at least one reinforcement learning sub-network has been arranged to receive the commensurate correlants of the hierarchical sets of at least two correlants, to determine a plurality of output values and to output the output values from the determined plurality of output values to the nodes of the higher node level and the nodes of the lower node level; and

wherein, the at least one ensemble learning sub-network has been arranged to receive and to combine at least one output value from the at least one node of the higher node level and to receive and to combine at least one output value from the at least one node of the lower node level.

2. The self-organizing computing machine of claim 1, wherein the at least one reinforcement learning sub-network incorporates a correlant input sub-node, an output sub-node, and a state-of-mind sub-node.

3. The self-organizing computing machine of claim 1, wherein the at least one ensemble learning sub-network incorporates a local input sub-node, a hidden state sub-node, and a state-of-mind sub-node.

4. The self-organizing computing machine of claim 1, wherein the at least one ensemble learning sub-network and the at least one reinforcement learning sub-network incorporate a common state-of-mind sub-node.

5. The self-organizing computing machine of claim 1, wherein the at least one hierarchical node level incorporates at least two proximal nodes arranged for a direct information exchange between the at least two proximal nodes.

6. The self-organizing computing machine of claim 5, wherein the at least two proximal nodes have been arranged for the direct information exchange between the pertinent state-of-mind sub-nodes of the at least two proximal nodes.

7. The self-organizing computing machine of claim 5, wherein the at least two proximal nodes have been arranged for the direct information exchange between the at least one state-of-mind sub-nodes and the at least one local input sub-node of the at least two proximal nodes.

8. The self-organizing computing machine of claim 5, wherein the at least two proximal nodes have been arranged for the direct information exchange between the at least one state-of-mind sub-nodes and the at least one output sub-node of the at least two proximal nodes.

9. The self-organizing computing machine of claim 1, wherein each of the at least two proximal hierarchical node levels incorporates at least one proximal node arranged for direct information exchange with the at least another proximal node incorporated in the different proximal hierarchical node level.

10. The self-organizing computing machine of claim 3, wherein the at least one ensemble learning sub-network incorporates the hidden state sub-node arranged to derive and preserve information based on past information previously contained in the state-of-mind sub-node.

11. The self-organizing computing machine of claim 4, wherein the at least one node at each hierarchical node level comprises the ensemble learning sub-network arranged for an iterative exchange of information between a hidden state sub-node and the common state-of-mind sub-node, and the reinforcement learning sub-network arranged for an iterative exchange of information between an output sub-node and the common state-of-mind sub-node.

12. The self-organizing computing machine of claim 2, wherein the at least one reinforcement learning sub-network incorporates a reward feed-back sub-node arranged to receive process and communicate a reward information arranged to modulate the output value.

13. The self-organizing computing machine of claim 12, wherein the reward feed-back sub-node process and communicate reward information arranged to modulate the reinforcement learning sub-network output as a function of previous output values.

14. A self-organizing computing process for mapping from a plurality of patterns contained within at least one predetermined set of provided inputs to at least one invariant perception distinguishable, by a name or a label, among a plurality of categories, the self-organizing computing process comprises:

a) providing at least one self-organizing computing machine incorporating at least one network of at least three nodes arranged in at least two hierarchical node levels including at least a lower node level and a higher node level;

at least one feature extractor for receiving the at least one predetermined set of provided inputs, processing the at least one predetermined set of provided inputs to determine a hierarchical set of at least two correlants commensurate with the at least two hierarchical correlant levels including at least a lower correlant level and a higher correlant level, and communicating the determined hierarchical sets of at least two correlants to the at least two distinct nodes of the at least two distinct hierarchical node levels commensurate with the at least two correlants of the at least two distinct correlant levels such that the correlants of the lower correlant level communicates to the corresponding nodes of the lower node level and that the correlants of the higher correlant level communicates to the corresponding nodes of the higher node level;

at least one output unit for interfacing the at least one output one invariant perception distinguishable, by a name or a label, among categories;

wherein, the at least one node at each hierarchical node level includes at least one reinforcement learning sub-network combined with at least one ensemble learning sub-network;

wherein, the at least one reinforcement learning sub-network have been arranged to receive the commensurate correlants of the hierarchical sets of at least two correlants, to determine a plurality of output values and to output the output values from the determined plurality of output values to the nodes of the higher node level nodes and the nodes of the lower node level; and

wherein, the at least one ensemble learning sub-network has been arranged to receive and to combine at least one output value from the at least one node of the higher node level and to receive and combine at least one output value from the at least one node of the lower node level;

b) providing at least one predetermined initial set of inputs, to the at least one feature extractor and determining the hierarchical set of at least two correlants commensurate with the at least two hierarchical correlant levels;

c) communicating the determined hierarchical sets of at least two correlants to the at least two distinct nodes of the at least two distinct hierarchical node levels commensurate with the at least two correlants such that the correlants of the lower correlant level communicate to the corresponding nodes of the lower node level and that the correlants of the higher correlant level communicate to the corresponding nodes of the higher node level;

d) determining at least one output value from each of the at least two distinct nodes and providing the determined output values from each node to proximal nodes of the at least one network of the at least one self-organizing computing machine;

e) after a predetermined time period providing at least another subsequent set of inputs, to the at least one feature extractor and determining the hierarchical set of at least two subsequent correlants commensurate with the at least two hierarchical correlant levels;

f) communicating the determined hierarchical sets of at least two subsequent correlants to the at least two distinct nodes of the at least two distinct hierarchical node levels commensurate with the at least two subsequent correlants such that the correlants of the lower correlant level communicates to the corresponding nodes of the lower node level and that the correlants of the higher correlant level communicates to the corresponding nodes of the higher node level;

g) determining at least one subsequent output value from each of the at least two distinct nodes and providing the determined subsequent output values from each node to proximal nodes of the at least one network of the at least one self-organizing computing machine;

h) determining, based on the at least one subsequent output value of the at least one updated invariant perception distinguishable, by a name or a label, among categories;

i) repeating sequentially steps c)-h) for another predetermined time period, or for a duration of time necessary to achieve a predetermined convergence of the at least one subsequent output value of a preselected node of the at least one network;

j) interfacing the at least one updated invariant perception distinguishable, by a name or a label, among categories.

15. The self-organizing computing process of claim 14, wherein providing the at least one self-organizing computing machine includes providing the at least one reinforcement learning sub-network incorporating a correlant input sub-node, an output sub-node, and a state-of-mind sub-node.

16. The self-organizing computing process of claim 14, wherein providing the at least one self-organizing computing machine includes providing the at least one ensemble learning sub-network incorporates a local input sub-node, a hidden state sub-node, and a state-of-mind sub-node.

17. The self-organizing computing process of claim 14, wherein providing the at least one self-organizing computing machine includes providing the at least one ensemble learning sub-network and the at least one reinforcement learning sub-network incorporate a common state-of-mind sub-node.

18. The self-organizing computing process of claim 14, wherein providing the at least one self-organizing computing machine includes providing the at least one hierarchical node level incorporates at least two proximal nodes arranged for a direct information exchange between the at least two proximal nodes.

19. The self-organizing computing process of claim 18, wherein providing the at least one self-organizing computing machine includes providing the at least two proximal nodes arranged for the direct information exchange between the pertinent state-of-mind sub-nodes of the at least two proximal nodes.

20. The self-organizing computing process of claim 18, wherein providing the at least one self-organizing computing machine includes providing the at least two proximal nodes arranged for the direct information exchange between the at least one state-of-mind sub-nodes and the at least one local input sub-node of the at least two proximal nodes.

21. The self-organizing computing process of claim 18, wherein providing the at least one self-organizing computing machine includes providing the at least two proximal nodes arranged for the direct information exchange between the at least one state-of-mind sub-nodes and the at least one output sub-node of the at least two proximal nodes.

22. The self organizing computing process of claim 17, wherein each of the at least two proximal hierarchical node levels incorporates at least one proximal node arranged for direct information exchange with the at least another proximal node incorporated in the different proximal hierarchical node level.

23. The self-organizing computing process of claim 17, wherein the at least one ensemble learning sub-network incorporates the hidden state sub-node arranged to derive and preserve information based on past information previously contained in the state-of-mind sub-node.

24. The self-organizing computing process of claim 17, wherein the at least one node at each hierarchical node level comprises the ensemble learning sub-network arranged for an iterative exchange of information between the hidden state sub-node and the state-of-mind sub-node, and the reinforcement learning sub-network arranged for an iterative exchange of information between the output sub-node and the state-of-mind sub-node.

25. The self-organizing computing process of claim 14, wherein providing at least one predetermined initial set of inputs to the at least one feature extractor and determining the hierarchical set of at least two correlants commensurate with the at least two hierarchical node levels includes decomposing the at least one predetermined initial set of inputs into at least two substantially self-similar hierarchical structures and determining at least one correlant for each self-similar hierarchical structure.

26. The self-organizing computing process of claim 25, wherein communicating the determined hierarchical sets of at least two correlants to the at least two distinct nodes of the at least two distinct hierarchical node levels includes inputting the at least one determined correlant into the at least one reinforcement learning sub-network of the at least one node of the higher node level of the at least one network.

27. The self-organizing computing process of claim 14, wherein determining the updated output value from each of the at least two distinct nodes and providing the determined output values from each node to proximal nodes of the at least one network of the at least one self-organizing computing machine includes determining a value of at least one reward parameter and providing the determined value at least one reward parameter to proximal nodes of the at least one network of the at least one self-organizing computing machine.

28. The self-organizing computing process of claim 14, wherein repeating sequentially steps c)-e) for another predetermined time period or for a duration of time necessary to achieve a predetermined convergence of the output value of a preselected node of the at least one network includes sequential assembling in at least one hidden state sub-node of sequentially updated sets of statistical data corresponding with an evolution of information in the at least one hidden state sub-node of at least one node and utilizing the set of statistical data in combination with the determined output values from each node to proximal nodes to determine an updated value stored in the state-of-mind sub-node of the at least one node of the at least one network of the at least one self-organizing computing machine.

29. The self-organizing computing process of claim 14, wherein the determined output values from each node provided to proximal nodes of the at least one network of the at least one self-organizing computing machine include a reward information for definition of a policy determinative for calculation of the output values provided from each node to proximal nodes of the at least one network of the at least one self-organizing computing machine.