SYSTEMS AND METHODS FOR A COMPUTER UNDERSTANDING MULTI MODAL DATA STREAMS
0 Assignments
0 Petitions
Accused Products
Abstract
Systems and methods for understanding (imputing meaning to) multi modal data streams may be used in intelligent surveillance and allow a) real-time integration of streaming data from video, audio, infrared and other sensors; b) processing of the results of such integration to obtain understanding of the situation as it unfolds; c) assessing the level of threat inherent in the situation; and d) generating of warning advisories delivered to appropriate recipients as necessary for mitigating the threat. The system generates understanding of the system by creating and manipulating models of the situation as it unfolds. The creation and manipulation involve “neuronal packets” formed in mutually constraining associative networks of four basic types. The process is thermodynamically driven, striving to produce a minimal number of maximally stable models. Obtaining such models is experienced as grasping, or understanding the input stream (objects, their relations and the flow of changes).
5 Citations
34 Claims
-
1-25. -25. (canceled)
-
26. A method of obtaining understanding of multimodal data streams by dynamically optimizing allocation of neuronal resources to data elements in the stream wherein resources are drawn from neuronal pool on which four temporal constraints are defined:
- pool'"'"'s longevity limit (life span), neuronal recuperation period, mobilization duration, and link decay period.
-
34. Neuronal pool, comprising groups of identical type neurons (minicolumns) such that
a) neuronal packets are formed of neurons mobilized from different minicolumns; -
b) functional distance in the neuronal space between neuronal packets A and B is determined by the degree of overlap accounting for the number of neuron types present in A but not in B, the number of neuron type present in B but not in A, and the number of neuron types present in both A and B; and c) the degree of similarity between objects represented by neuronal packets A and B is determined by functional distance between A and B.
-
Specification