Detection and Tracking of Moving Objects

• US 20120154579A1
• Filed: 12/20/2010
• Published: 06/21/2012
• Est. Priority Date: 12/20/2010
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Hampapur, Arun, Otto, Charles A., Li, Jun, Pankanti, Sharathchandra
• Granted Patent: US 9,147,260 B2
• Time in Patent Office: Days
• US Class Current: 348/143
• CPC Class Codes: G03B 15/16 for photographing the track of moving objects high-speed photography G03B39/00
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/20076 Probabilistic image processing
• CPC Class Codes: G06T 2207/30232 Surveillance
• CPC Class Codes: G06T 2207/30241 Trajectory
• CPC Class Codes: G06T 7/207 for motion estimation over a hierarchy of resolutions multi-resolution motion estimation or hierarchical motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/53
• CPC Class Codes: G06T 7/215 Motion-based segmentation
• CPC Class Codes: G06T 7/223 using block-matching
• CPC Class Codes: G06T 7/246 using feature-based methods, e.g. the tracking of corners or segments
• CPC Class Codes: G06T 7/248 involving reference images or patches
• CPC Class Codes: G06T 7/254 involving subtraction of images
• CPC Class Codes: G06T 7/277 involving stochastic approaches, e.g. using Kalman filters

Abstract

Techniques for performing visual surveillance of one or more moving objects are provided. The techniques include registering one or more images captured by one or more cameras, wherein registering the one or more images comprises region-based registration of the one or more images in two or more adjacent frames, performing motion segmentation of the one or more images to detect one or more moving objects and one or more background regions in the one or more images, and tracking the one or more moving objects to facilitate visual surveillance of the one or more moving objects.

Claims

1. A method for performing visual surveillance of one or more moving objects, wherein the method comprises:

registering one or more images captured by one or more cameras, wherein registering the one or more images comprises region-based registration of the one or more images in two or more adjacent frames;

performing motion segmentation of the one or more images to detect one or more moving objects and one or more background regions in the one or more images; and

tracking the one or more moving objects to facilitate visual surveillance of the one or more moving objects.

2. The method of claim 1, wherein registering one or more images comprises recursive global and local geometric registration of the one or more images.

3. The method of claim 1, wherein registering one or more images comprises using one or more sub-pixel image matching techniques.

4. The method of claim 1, wherein performing motion segmentation of the one or more images comprises forward and backward frame differencing.

5. The method of claim 4, wherein forward and backward frame differences comprises automatic dynamic threshold estimation based on at least one of temporary filtering and spatial filtering.

6. The method of claim 4, wherein forward and backward frame differences comprises removing one or more false moving pixels based on independent motions of one or more image features.

7. The method of claim 4, wherein forward and backward frame differences comprises performing a morphological operation and generating one or more motion pixels.

8. The method of claim 1, wherein tracking the one or more moving objects comprises performing hybrid target tracking, wherein hybrid target tracking comprises using a Kanade-Lucas-Tomasi feature tracker and meanshift, using auto kernel scale estimation and updating, and using one or more feature trajectories.

9. The method of claim 1, wherein tracking the one or more moving objects comprises using one or more multi-target tracking algorithms based on feature matching and distance matrices for one or more targets.

10. The method of claim 1, wherein tracking the one or more moving objects comprises:

generating a motion map;

identifying one or more moving objects;

performing object initialization and object checking;

identifying one or more object regions in the motion map;

extracting one or more features;

setting a search region in the motion map;

identifying one or more candidate regions in the motion map;

meanshift tracking;

identifying one or more moving objects in the one or more candidate regions;

performing Kanade-Lucas-Tomasi feature matching;

performing an affine transform;

making a final regions determination via the Bhattacharyya coefficient; and

updating a target model and trajectory information.

11. The method of claim 1, wherein tracking the one or more moving objects comprises reference plane-based registration and tracking.

12. The method of claim 1, further comprising relating each camera view with one or more other camera views.

13. The method of claim 1, further comprising forming a panoramic view from the one or more images captured by one or more cameras.

14. The method of claim 13, further comprising estimating motion of each camera based on video information of one or more static objects in the panoramic view.

15. The method of claim 13, further comprising estimating one or more background structures in the panoramic view based on linear structure detection and statistical analysis of the one or more moving objects over a period of time.

16. The method of claim 1, further comprising automatic feature extraction, wherein automatic feature extraction comprises:

framing an image;

performing a Gaussian smoothing operation;

using a canny detector to extract one or more feature edges;

implementing a hough transformation for feature analysis;

determining a maximum response finding for reducing an influence of multiple peaks in a transform space;

determining if a length of a feature is greater than a certain threshold, and if the length of the feature is greater than the threshold, performing feature extraction and pixel removal.

17. The method of claim 16, wherein automatic feature extraction further comprises performing frame differencing and verification via motion history images.

18. The method of claim 1, further comprising performing outlier removal to remove one or more incorrect moving object matches.

19. The method of claim 1, further comprising false blob filtering, wherein false blob filtering comprises:

generating a motion map;

applying a connected component process to link each blob data;

creating a motion blob table;

extracting one or more features for each blob in a previously registered frame; and

applying a Kanade-Lucas-Tomasi method to estimate motion of each blob, and, if no motion occurs for a blob, deleting the blob from the blob table.

20. The method of claim 1, further comprising updating a target model on at least one of a temporal domain and a spatial domain.

21. The method of claim 1, further comprising creating an index of object appearances and object tracks in a panoramic view.

22. The method of claim 21, further comprising determining a similarity metric between a query and an entry in the index.

23. The method of claim 1, further comprising providing a system, wherein the system comprises one or more distinct software modules, each of the one or more distinct software modules being embodied on a tangible computer-readable recordable storage medium, and wherein the one or more distinct software modules comprise a geometric registration module, a motion extraction module and an object tracking module executing on a hardware processor.

24. A computer program product comprising a tangible computer readable recordable storage medium including computer useable program code for performing visual surveillance of one or more moving objects, the computer program product including:

computer useable program code for registering one or more images captured by one or more cameras, wherein registering the one or more images comprises region-based registration of the one or more images in two or more adjacent frames;

computer useable program code for performing motion segmentation of the one or more images to detect one or more moving objects and one or more background regions in the one or more images; and

computer useable program code for tracking the one or more moving objects to facilitate visual surveillance of the one or more moving objects.

25. A system for performing visual surveillance of one or more moving objects, comprising:

a memory; and

at least one processor coupled to the memory and operative to;

register one or more images captured by one or more cameras, wherein registering the one or more images comprises region-based registration of the one or more images in two or more adjacent frames;

perform motion segmentation of the one or more images to detect one or more moving objects and one or more background regions in the one or more images; and

track the one or more moving objects to facilitate visual surveillance of the one or more moving objects.