Multi-source multi-modal activity recognition in aerial video surveillance
First Claim
Patent Images
1. A system for multi-source multi-modal activity recognition in conducting aerial video surveillance comprising:
- from a moving platform, detecting and tracking, with a video imager, multiple dynamic targets, wherein the detecting and tracking comprisesdifferencing registered frames;
using high pixel difference point features to establish correspondences between other points in a previous frame; and
clustering point-velocity pairs into motion regions assumed to be individual targets;
recording analyst call outs or chats, and appending said analyst call outs or chats to a file;
representing full motion video (FMV) target tracks and chat-messages as graphs of attributes;
associating said FMV tracks and said chat-messages using a probabilistic graph based mapping approach;
detecting spatial-temporal activity boundaries;
categorizing activity of said detected multiple dynamic targets; and
on a display, presenting said activity.
2 Assignments
0 Petitions
Accused Products
Abstract
Multi-source multi-modal activity recognition for conducting aerial video surveillance comprising detecting and tracking multiple dynamic targets from a moving platform, representing FMV target tracks and chat-messages as graphs of attributes, associating FMV tracks and chat-messages using a probabilistic graph based mapping approach; and detecting spatial-temporal activity boundaries.
-
Citations
19 Claims
-
1. A system for multi-source multi-modal activity recognition in conducting aerial video surveillance comprising:
-
from a moving platform, detecting and tracking, with a video imager, multiple dynamic targets, wherein the detecting and tracking comprises differencing registered frames; using high pixel difference point features to establish correspondences between other points in a previous frame; and clustering point-velocity pairs into motion regions assumed to be individual targets; recording analyst call outs or chats, and appending said analyst call outs or chats to a file; representing full motion video (FMV) target tracks and chat-messages as graphs of attributes; associating said FMV tracks and said chat-messages using a probabilistic graph based mapping approach; detecting spatial-temporal activity boundaries; categorizing activity of said detected multiple dynamic targets; and on a display, presenting said activity. - View Dependent Claims (2, 3, 4, 5, 6, 7, 8, 9, 10)
-
-
11. A method for multi-source multi-modal activity recognition in conducting aerial video surveillance comprising:
-
tracking a target using a video device on an airborne platform; mapping tracks to graphs comprising multi-graph representation of a single full motion video (FMV) track; parsing and graph representation of chats; associating multi-source graphs and assigning activity classes; learning activity patterns from multi-source associated data; and visualizing event/activity reports on a display and querying by activity type and geo-location, wherein the reports comprise a video summary of activities-of-interest/targets-of-interest AOIs/TOIs allowing non-linear browsing of video content, annotated text-over-video media where only TOIs are highlighted with bounding boxes and synchronized with chat-messages; grouping activities of a same type into an activities index, and allowing adaptive data play-back for user-selected filtering by geographic location. - View Dependent Claims (12, 13, 14, 15, 16, 17, 18)
-
-
19. A system for a multi-source multi-modal probabilistic graph-based association framework for aerial video surveillance comprising:
-
reviewing by a reviewer at least some of the aerial video surveillance providing reviewed FMV data with a resulting set of non-reviewed FMV data; identifying targets-of-interest corresponding to chat-messages, wherein said chat-messages are the only source to describe a true activity of a target of interest (TOI); extracting objects from a full motion video (FMV) of the aerial video surveillance; detecting activity boundaries comprising segmenting full motion video (FMV) tracks from said aerial video surveillance into semantic sub-tracks/segments; learning activity patterns in low-level feature spaces using the reviewed FMV data; indexing non-reviewed FMV data; and providing to FMV analysts a user interface display for querying and non-linear browsing of multi-source data.
-
Specification