×

Systems and methods for a computer understanding multi modal data streams

  • US 9,378,455 B2
  • Filed: 02/07/2013
  • Issued: 06/28/2016
  • Est. Priority Date: 05/10/2012
  • Status: Active Grant
First Claim
Patent Images

1. A computer implemented method in a self-adaptive multi modal data stream processing system having at least one computer processor and at least one spatiotemporal associative memory coupled to the at least one computer processor, the method comprising:

  • receiving multi modal data streams by the computer processor from multiple data stream sources, the multi modal data streams representing an environment of the multi modal data stream processing system;

    constructing, by a construction module under control of a control module of the multi modal data stream processing system, a model of a situation built upon an underlying associative neural network that is partitioned into neuronal packets which are internally cohesive and externally weakly coupled subnetworks surrounded by energy barriers at a boundary of the subnetworks;

    storing the underlying associative neural network in the associative memory to establish situational understanding of the situation;

    associating neuronal packet groupings into stable (invariant) and changing (variable) entities and relationships between the entities;

    assigning a relationship type to the components based on their content and behavior thereby creating a model of the situation, wherein each entity is able to be nested by the control module by being comprised of lower level models and wherein the lower level models are formed of neuronal packets and are groups of neuronal packets;

    manipulating the lower level models by the control module of the multi modal data stream processing system, by manipulating neuronal packets while leaving the underlying associative neural network intact by not changing synaptic weights in the underlying associative neural network in the manipulation of the lower level models;

    reducing, by the multi modal data stream processing system, energy consumption and energy dissipation in the constructing and the manipulating of the models by the control module seeking progressively more general and adequate models persisting through various situations and wherein the reducing energy consumption and dissipation translates into entropy reduction, or system negentropy production in the system;

    based on a generated situational understanding of a situation, generating in real time by the multi modal data stream processing system appropriate output to facilitate one or more responses to the situation selected from the group consisting of an assessed threat level when objects or conditions in the situation constitute a threat when acting in coordination, identification of objects in an environment of a robotic vehicle or other robotic system, and automatic detection and evaluation of malware in a computer network;

    if the situation is an assessed threat level, facilitating an automated intelligent surveillance of the situation;

    if the situation is objects in an environment of a robotic vehicle or other robotic system, performing by the robotic vehicle or other robotic system adjusting pursuit of specified objectives and responding to obstacles; and

    if the situation is the automatic detection and evaluation of malware in a computer network, dynamically deploying countermeasures against the malware over time.

View all claims
  • 0 Assignments
Timeline View
Assignment View
    ×
    ×