ADAPTIVE VOTING EXPERTS FOR INCREMENTAL SEGMENTATION OF SEQUENCES WITH PREDICTION IN A VIDEO SURVEILLANCE SYSTEM
First Claim
1. A computer-implemented method for analyzing a scene depicted in an input stream of video frames captured by a video camera, the method comprising:
- receiving a plurality of sequences, wherein each sequence stores an ordered string of labels assigned to clusters in an adaptive resonance theory (ART) network, wherein the ART network clusters nodes of a self-organizing map (SOM), and wherein the SOM is generated by mapping, to nodes of the SOM, kinematic data vectors generated for foreground objects detected in the input stream of video frames;
generating, from the plurality sequences, an ngram trie to a specified depth, wherein the ngram trie includes a node for each subsequence present in the plurality of sequences, up to the specified depth;
determining, for each node in the ngram trie, an entropy measure based on a count of how many times the subsequence represented by the node appears in the plurality of sequences;
receiving a first sequence; and
determining, in the first sequence, one or more segments, based on the determined entropies.
6 Assignments
0 Petitions
Accused Products
Abstract
A sequence layer in a machine-learning engine configured to learn from the observations of a computer vision engine. In one embodiment, the machine-learning engine uses the voting experts to segment adaptive resonance theory (ART) network label sequences for different objects observed in a scene. The sequence layer may be configured to observe the ART label sequences and incrementally build, update, and trim, and reorganize an ngram trie for those label sequences. The sequence layer computes the entropies for the nodes in the ngram trie and determines a sliding window length and vote count parameters. Once determined, the sequence layer may segment newly observed sequences to estimate the primitive events observed in the scene as well as issue alerts for inter-sequence and intra-sequence anomalies.
-
Citations
25 Claims
-
1. A computer-implemented method for analyzing a scene depicted in an input stream of video frames captured by a video camera, the method comprising:
-
receiving a plurality of sequences, wherein each sequence stores an ordered string of labels assigned to clusters in an adaptive resonance theory (ART) network, wherein the ART network clusters nodes of a self-organizing map (SOM), and wherein the SOM is generated by mapping, to nodes of the SOM, kinematic data vectors generated for foreground objects detected in the input stream of video frames; generating, from the plurality sequences, an ngram trie to a specified depth, wherein the ngram trie includes a node for each subsequence present in the plurality of sequences, up to the specified depth; determining, for each node in the ngram trie, an entropy measure based on a count of how many times the subsequence represented by the node appears in the plurality of sequences; receiving a first sequence; and determining, in the first sequence, one or more segments, based on the determined entropies. - View Dependent Claims (2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12)
-
-
13. A computer-readable storage medium containing a program, which when executed on a processor, performs an operation for analyzing a scene depicted in an input stream of video frames captured by a video camera, the operation comprising:
-
receiving a plurality of sequences, wherein each sequence stores an ordered string of labels assigned to clusters in an adaptive resonance theory (ART) network, wherein the ART network clusters nodes of a self-organizing map (SOM), and wherein the SOM is generated by mapping, to nodes of the SOM, kinematic data vectors generated for foreground objects detected in the input stream of video frames; generating, from the plurality sequences, an ngram trie to a specified depth, wherein the ngram trie includes a node for each subsequence present in the plurality of sequences, up to the specified depth; determining, for each node in the ngram trie, an entropy measure based on a count of how many times the subsequence represented by the node appears in the plurality of sequences; receiving a first sequence; and determining, in the first sequence, one or more segments, based on the determined entropies. - View Dependent Claims (14, 15, 16, 17, 18, 19, 20)
-
-
21. A system, comprising:
-
a video input source configured to provide an input stream of video frames captured by a video camera, each depicting a scene; a processor; and a memory containing a program, which, when executed on the processor is configured to perform an operation for analyzing the scene depicted in the input stream of video frames, the operation comprising; receiving a plurality of sequences, wherein each sequence stores an ordered string of labels assigned to clusters in an adaptive resonance theory (ART) network, wherein the ART network clusters nodes of a self-organizing map (SOM), and wherein the SOM is generated by mapping, to nodes of the SOM, kinematic data vectors generated for foreground objects detected in the input stream of video frames, generating, from the plurality sequences, an ngram trie to a specified depth, wherein the ngram trie includes a node for each subsequence present in the plurality of sequences, up to the specified depth, determining, for each node in the ngram trie, an entropy measure based on a count of how many times the subsequence represented by the node appears in the plurality of sequences, receiving a first sequence, and determining, in the first sequence, one or more segments, based on the determined entropies. - View Dependent Claims (22, 23, 24, 25)
-
Specification