Method and apparatus for recognizing objects visually using a recursive cortical network

• US 9,262,698 B1
• Filed: 05/15/2013
• Issued: 02/16/2016
• Est. Priority Date: 05/15/2012
• Status: Active Grant

• Current Assignee: Intrinsic Innovation LLC (Alphabet Inc.)
• Original Assignee: Vicarious FPC, Inc. (Alphabet Inc.)
• Inventors: George, Dileep, Kansky, Kenneth Alan, Phoenix, David Scott, Laan, Christopher, Lehrach, Wolfgang, Marthi, Bhaskara
• Primary Examiner(s): Chawan, Sheela
• Application Number:
  US13/895,058
• Time in Patent Office: 1,007 Days
• Field of Search: 382/159, 382/224, 382/156, 382/103, 706/14, 706/16, 706/20
• US Class Current: 1/1
• CPC Class Codes: G06N 3/044 Recurrent networks, e.g. Hopfield networks
• CPC Class Codes: G06N 3/047 Probabilistic or stochastic networks
• CPC Class Codes: G06N 3/08 Learning methods
• CPC Class Codes: G06V 10/454 Integrating the filters into a hierarchical structure, e.g. convolutional neural networks [CNN]

Abstract

A computer-implemented method for object recognition using a recursive cortical network comprising receiving an input image at an input module, applying a trained recursive cortical network (RCN) to the image using an inference module to activate child features of the RCN, selecting pools of the RCN containing the activated child features, propagating the selection of the pools to identify probabilities of one or more high-level features matching one or more objects in the input image.

Claims

1. A computer-implemented method for object recognition using a recursive cortical network comprising:

receiving an input image at a training module;

applying a trained recursive cortical network (RCN) to the image using an inference module to activate child features of the RCN;

selecting pools of the RCN containing the activated child features; and

propagating the selection of the pools to identify probabilities of one or more higher-level features matching one or more objects in the input image.

2. The method of claim 1 further comprising:

selecting a second pool when a child feature in a first pool has a lateral constraint with a child feature in the second pool; and

repeating applying the trained RCN, selecting pools of the RCN and propagating the selection of the pools, where outputs of child pools in the RCN are input to parent pools of the RCN.

3. The method of claim 2 further comprising:

during training of the RCN, reducing a pooling radius of a pool in the RCN to zero;

determining whether a higher level feature is memorized using its component pools; and

modifying the pooling to include more features if the higher level feature has been memorized to increase invariance to a predetermined threshold.

4. The method of claim 1 further comprising:

assigning a probability to the activated child feature corresponding to the probability that this feature exists in the input image; and

selecting the pools of the RCN based on whether the probability is greater than a threshold value.

5. The method of claim 1 further comprising:

backtracking down the RCN from each pool to locate miscounted features from a set of winning features;

notifying a parent of the double counted feature of the miscounting; and

propagating the selection of the pools to one or more high-level features accounting for the miscounted features.

6. The method of claim 5 further comprising:

performing backtracking on each lateral connection for the winning features;

identifying overlapping features; and

correcting the probability of the overlapping features.

7. The method of claim 6 further comprising:

performing backtracking on all winning features simultaneously; and

correcting inference values for all miscounted features.

8. The method of claim 1 further comprising:

removing a portion of the input image which has been matched to a high-level feature representation; and

performing inference on the remaining portion of the image.

9. The method of claim 8 further comprising:

instead of removing a portion of the input image which has been matched, reducing the probability that there was a match in order to account for occluded features.

10. The method of claim 1 further comprising:

receiving a sequence of images at the training module; and

matching the sequence of images to high-level features using the RCN.

11. The method of claim 1, wherein the RCN comprises bi-directional links between nodes of the RCN.

12. A computer-implemented method for object generation using a recursive cortical network (RCN) comprising:

activating a high-level feature-node of the RCN according to user entered request;

selecting one of one or more pools associated with the high-level feature node at random;

selecting a winning feature from each of the one or more pools based on lateral connections; and

composing an object based on the selected winning features from each pool.

13. The method of claim 12, selecting a winning feature from each of the one or more pools further comprising:

select a first winning feature from the selected pool at random;

selecting a second winning feature from each of the other pools in the one or more pools, having a lateral constraint with the first winning feature; and

selecting a third winning feature from each of the other pools not containing lateral support, wherein the third winning feature has maximum support based on the first winning feature and second winning feature.

14. The method of claim 12, further comprising:

determining probabilities for each feature of each pool.

selecting one or more features based on the determined probabilities.

selecting other pools that are laterally consistent with the previously selected pool(s).

15. The method of claim 14 further comprising:

fixing expectation of portions of the composed object and only selecting random winning features on the non-fixed portions of the composed object.

16. An apparatus for object recognition using a recursive cortical network comprising:

a training module for receiving input data; and

a processor configured to;

apply a trained recursive cortical network (RCN) to the data using an inference module to activate child features of the RCN corresponding to features in the input data;

select pools of the RCN containing the activated child features; and

propagate the selection of the pools to identify probabilities of one or more high-level features matching one or more objects in the input data.

17. The apparatus of claim 16 wherein the image processor is further configured to:

select a second pool when a child feature in a first pool has a lateral constraint with a child feature in the second pool.

18. The apparatus of claim 17 wherein the image processor is further configured to:

during training of the RCN, reduce a pooling radius of a pool in the RCN to zero;

determine whether a higher level feature is memorized using its component pools; and

modify the pooling to include more features if the higher level feature has been memorized to increase invariance to a predetermined threshold.

19. The apparatus of claim 16 wherein the image processor is further configured to:

assign a probability to the activated child feature corresponding to the probability that this feature exists in the input image; and

select the pools of the RCN based on whether the probability is greater than a threshold value.

20. The apparatus of claim 16 wherein the image processor is further configured to:

backtrack down the RCN from each pool to locate miscounted features from a set of winning features;

notify a patent of the double counted feature of the miscounting; and

propagate the selection of the pools to one or more high-level features accounting for the miscounted features.

21. The apparatus of claim 16, further comprising wherein the input data may be one or more of an image, an image sequence, audio data, and video data.

22. The apparatus of claim 16, wherein the RCN comprises bi-directional links between nodes of the RCN.