Systems and methods for predictive/reconstructive visual object tracker

• US 10,282,849 B2
• Filed: 06/19/2017
• Issued: 05/07/2019
• Est. Priority Date: 06/17/2016
• Status: Active Grant

• Current Assignee: Brain Corporation
• Original Assignee: Brain Corporation
• Inventors: Piekniewski, Filip, Richert, Micah, Fisher, Dimitry, Laurent, Patryk, Petre, Csaba
• Primary Examiner(s): Tsai, Tsung Yin
• Application Number:
  US15/627,096
• Publication Number:
  US 20180018775A1
• Time in Patent Office: 687 Days
• CPC Class Codes: G06F 12/08 in hierarchically structured memory systems, e.g. virtual memory systems
• CPC Class Codes: G06F 2212/1016 Performance improvement
• CPC Class Codes: G06F 2212/401 Compressed data
• CPC Class Codes: G06T 1/60 Memory management
• CPC Class Codes: G06T 2207/20084 Artificial neural networks [ANN]
• CPC Class Codes: G06T 7/0002 Inspection of images, e.g. flaw detection
• CPC Class Codes: G06T 7/20 Analysis of motion motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51
• CPC Class Codes: G06T 9/00 Image coding bandwidth or redundancy reduction for static pictures H04N1/41; coding or decoding of static colour picture signals H04N1/64; methods or arrangements for coding, decoding, compressing or decompressing digital video signals H04N19/00
• CPC Class Codes: G06T 9/002 using neural networks
• CPC Class Codes: G06T 9/004 Predictors, e.g. intraframe, interframe coding
• CPC Class Codes: H04N 19/503 involving temporal prediction adaptive coding with adaptive selection between spatial and temporal predictive coding H04N19/107; adaptive coding with adaptive selection among a plurality of temporal predictive coding modes H04N19/109

Abstract

Systems and methods for predictive/reconstructive visual object tracking are disclosed. The visual object tracking has advanced abilities to track objects in scenes, which can have a variety of applications as discussed in this disclosure. In some exemplary implementations, a visual system can comprise a plurality of associative memory units, wherein each associative memory unit has a plurality of layers. The associative memory units can be communicatively coupled to each other in a hierarchical structure, wherein data in associative memory units in higher levels of the hierarchical structure are more abstract than lower associative memory units. The associative memory units can communicate to one another supplying contextual data.

Claims

1. A visual system for tracking an object in a scene over time, comprising:

a hierarchy of associative memory units, the hierarchy including levels progressing from lower levels to higher levels, the associative memory units in the same level of the hierarchy feed compressed predictions to other associative memory units in the same level of the hierarchy, and each of the associative memory unit is configured to;

receive a past signal and a present signal of the scene;

predict a future signal based at least on the present signal and an association between the past signal and the present signal;

send the compressed predictions to other associative memory units;

receive the compressed predictions from other associative memory units; and

produce a signal indicative of the presence of the tracked object based at least on the predictions.

2. The visual system of claim 1, wherein the produced signal indicative of the presence of the tracked object comprises a position of the tracked object.

3. The visual system of claim 1, wherein the associative memory units are part of an artificial neural network.

4. The visual system of claim 1, wherein predicting the future signal is further based at least on the compressed predictions from other associative memory units.

5. The visual system of claim 1, wherein,

the associative memory units of the higher levels of the hierarchy feed compressed predictions to the lower levels, and

the compressed predictions further comprise concatenations of present signals of those associative memory units.

6. The visual system of claim 1, wherein each associative memory unit has a lower layer, middle layer, and upper layer.

7. The visual system of claim 6, wherein the middle layer predicts the future signal.

8. The visual system of claim 6, wherein the middle layer compresses the prediction.

9. The visual system of claim 1, further comprising a sensor unit configured to generate signals based at least on the scene.

10. A visual system for tracking an object in a scene over time, comprising:

a hierarchy of associative memory units, the hierarchy including levels progressing from a lower layer to a high layer, the lower and higher layers including a middle layer in-between thereto, the middle layer predicts a future signal, and each of the associative memory unit is configured to;

receive a present signal of the scene;

reconstruct the present signal based at least on the present signal and an association relating the present signal to the reconstructed present signal;

compress the reconstruction;

send the compressed reconstruction to other associative memory units;

receive compressed reconstructions from other associative memory units; and

produce a signal indicative of the presence of the tracked object based on at least the reconstruction.

11. The visual system of claim 10, wherein the produced signal indicative of the presence of a tracked object also comprises the position of the tracked object.

12. The visual system of claim 10, wherein the associative memory units are part of an artificial neural network.

13. The visual system of claim 10, wherein the middle layer is configured to predict the future signal based at least on compressed predictions from other associative memory units.

14. The visual system of claim 13, wherein the middle layer compresses the prediction.

15. The visual system of claim 10, wherein,

the associative memory units of the higher levels of the hierarchy is further configured to feed compressed predictions to the lower levels, the compressed predictions further comprise concatenations of present signals of those associative memory units.

16. The visual system of claim 15, wherein associative memory units in the same level of the hierarchy feed compressed predictions to other associative memory units in the same level of the hierarchy.

17. The visual system of claim 10, further comprising a sensor unit configured to generate signals based at least on the scene.

18. A method for processing information for object tracking, comprising:

receiving a past signal and a present signal of a scene containing an object of interest, along with a context;

associating the past signal with the present signal;

predicting a future signal based at least on the present signal, said association, and the context;

compressing the prediction and sending the compressed prediction to one or more associative memory units, each of the one or more associative memory units including levels progressing from a lower layer to a high layer, the lower and higher layers including a middle layer in-between thereto, the middle layer predicts the future signal; and

producing a signal indicative of the presence of the object of interest based at least on the prediction.

19. A method for processing information for object tracking, comprising:

receiving a present signal of a scene containing an object of interest, along with a context;

reconstructing the present signal based at least on the present signal, an association relating the present signal to the reconstructed present signal, and the context;

compressing the reconstruction;

sending the compressed reconstruction to one or more associative memory units, each of the one or more associative memory units including levels progressing from a lower layer to a high layer, the lower and higher layers including a middle layer in-between thereto, the middle layer predicts a future signal; and

producing a signal indicative of the presence of the object of interest based at least on the reconstruction.