System and method for a recursive cortical network

• US 9,373,085 B1
• Filed: 05/15/2013
• Issued: 06/21/2016
• Est. Priority Date: 05/15/2012
• Status: Active Grant

• Current Assignee: Intrinsic Innovation LLC (Alphabet Inc.), Intrinsic I. Llc
• Original Assignee: Vicarious FPC, Inc. (Alphabet Inc.)
• Inventors: George, Dileep, Kansky, Kenneth Alan, Phoenix, David Scott, Marthi, Bhaskara, Laan, Christopher, Lehrach, Wolfgang
• Primary Examiner(s): Hill, Stanley K
• Assistant Examiner(s): Chubb, Mikayla
• Application Number:
  US13/895,225
• Time in Patent Office: 1,133 Days
• Field of Search: None
• US Class Current: 1/1
• CPC Class Codes: G06N 20/00 Machine learning
• CPC Class Codes: G06N 3/04 Architecture, e.g. interconnection topology
• CPC Class Codes: G06N 3/045 Combinations of networks
• CPC Class Codes: G06N 3/049 Temporal neural networks, e.g. delay elements, oscillating neurons or pulsed inputs
• CPC Class Codes: G06N 3/082 modifying the architecture, e.g. adding, deleting or silencing nodes or connections
• CPC Class Codes: G06N 5/04 Inference or reasoning models
• CPC Class Codes: G06N 7/01 Probabilistic graphical models, e.g. probabilistic networks

Abstract

A system and method for generating and inferring patterns with a network that includes providing a network of recursive sub-networks with a parent feature input node and at least two child feature output nodes; propagating node selection through the network layer hierarchy in a manner consistent with node connections of sub-networks of the network, the propagation within the sub-network including enforcing a selection constraint on at least a second node of a second pool according to a constraint node of the sub-network; and compiling the state of final child feature nodes of the network into a generated output.

Claims

1. A method for inferring patterns with a network comprising:

providing a recursive network of sub-networks with a parent feature node and at least two child feature nodes;

configuring nodes of the sub-networks with posterior distribution component;

receiving data feature input at the final child feature nodes;

propagating node activation through network layer hierarchy in a manner consistent with node connections of sub-networks of the network and the posterior prediction of child nodes, wherein propagating node activation comprises child feature nodes messaging a likelihood score to connected parent-specific child feature (PSCF) nodes;

at a pool node of a sub-network, generating a likelihood score from the posterior distribution component and the likelihood score of connected PSCF nodes;

at a parent feature node of the sub-network, generating a likelihood score from the posterior distribution component and the likelihood score of pool nodes connected to the parent feature node;

enforcing an activation constraint between at least two nodes of a sub-network, wherein enforcing an activation constraint between at least two nodes comprises enforcing an activation constraint between a first PSCF node connected to a first pool node and a second PSCF node connected to a second pool node;

and outputting parent feature node selection to an inferred output.

2. The method of claim 1, wherein enforcing an activation constraint between at least two nodes comprises enforcing an activation constraint between a first PSCF node in a first sub-network and a second PSCF node in a second sub-network.

3. The method of claim 1, wherein enforcing an activation constraint between at least two nodes comprises enforcing an activation constraint between a first PSCF node in network of a first time instant and a second PSCF in network of a second time instant.

4. The method of claim 1, wherein receiving data feature input comprises setting activation of final child feature nodes according to image features of an image;

and wherein the inferred output is a detected object in the image.

5. The method of claim 1, wherein receiving data feature input comprises setting activation of final child feature nodes according to audio signal features;

and wherein the inferred output is a detected audio pattern in the audio signal.

6. The method of claim 1, wherein the child feature nodes of a first layer sub-network are the parent feature nodes for at least two second layer sub-networks.

7. A method for constructing a neural network comprising:

recursively architecting a plurality of sub-networks in a network hierarchy that comprises communicatively coupling each of the child feature nodes of a higher layer sub-network to the parent feature nodes of sub-networks in a lower layer;

setting a selection function of the parent feature node of the sub-networks, wherein the selection function is defined by selection options of at least two pools in the sub-network;

setting a selection function of the pool nodes, wherein the selection function of a pool node is defined by selection options of at least two parent-specific child feature (PSCF) nodes;

linking at least a pair of nodes with a constraint node; and

propagating node selection through the network layer hierarchy in a manner consistent with node connections of sub-networks of the network, the selection functions, and the linked constraint nodes.

8. The method of claim 7, wherein linking at least a pair of nodes with a constraint node comprises linking a first PSCF node connected to a first pool node with a second PSCF node connected to a second pool.

9. The method of claim 7, wherein linking at least a pair of nodes with a constraint node comprises linking a first PSCF node of a first sub-network with a second PSCF node of a second sub-network.

10. The method of claim 7, wherein linking at least a pair of nodes with a constraint node comprises linking a first PSCF node in a first network of a first instance with a second PSCF node in a second network of a first instance.

11. The method of claim 7, further comprising setting posterior parameters;

wherein propagating node selection through the network layer hierarchy is further performed in a manner consistent with belief propagation according to the set posterior parameters.

12. The method of claim 7, further comprising connecting the PSCF nodes with child feature nodes, wherein at least one child feature node is connected with at least two PSCF nodes.

13. A system comprising:

A recursively architected network of sub-networks organized into a plurality of hierarchical layers;

the sub-networks comprising at least a parent feature node, a pool node, a parent-specific child feature node, and a child feature node;

the parent feature node of at least one sub-network configured with a selection function actionable on at least two pool nodes connected to the parent feature node of the at least one sub-network;

the pool node of the at least one sub-network configured with a selection function actionable on at least two PSCF nodes connected to the pool node of the at least one sub-network;

the PSCF node of the at least one sub-network configured to activate a connected child feature node;

the child feature node connectable to at least a parent feature node of a second sub-network at a lower hierarchical layer; and

a constraint node with at least two connections from at least two PSCF nodes, with a selection function to augment selection by the pool node.