Object tracking with integrated motion-based object detection (MogS) and enhanced kalman-type filtering

• US 9,552,648 B1
• Filed: 10/29/2013
• Issued: 01/24/2017
• Est. Priority Date: 01/23/2012
• Status: Active Grant

• Current Assignee: HRL Laboratories LLC (The Boeing Co.)
• Original Assignee: HRL Laboratories LLC (The Boeing Co.)
• Inventors: Zhang, Lei, Khosla, Deepak, Chen, Yang
• Primary Examiner(s): HAUSMANN, MICHELLE M
• Assistant Examiner(s): Entezari, Michelle
• Application Number:
  US14/066,600
• Time in Patent Office: 1,183 Days
• Field of Search: None
• US Class Current: 1/1
• CPC Class Codes: G06T 7/187 involving region growing; involving region merging; involving connected component labelling
• CPC Class Codes: G06T 7/194 involving foreground-background segmentation
• CPC Class Codes: G06T 7/215 Motion-based segmentation
• CPC Class Codes: G06T 7/246 using feature-based methods, e.g. the tracking of corners or segments
• CPC Class Codes: G06T 7/254 involving subtraction of images
• CPC Class Codes: G06T 7/277 involving stochastic approaches, e.g. using Kalman filters
• CPC Class Codes: G06V 10/255 Detecting or recognising potential candidate objects based on visual cues, e.g. shapes
• CPC Class Codes: G06V 20/52 Surveillance or monitoring of activities, e.g. for recognising suspicious objects recognising microscopic objects G06V20/69

Abstract

Described is a system for object tracking with integrated motion-based object detection and enhanced Kalman-type filtering. The system detects a location of a moving object in an image frame using an object detection MogS module, thereby generating an object detection. For each image frame in a sequence of image frames, the system predicts the location of the moving object in the next image frame using a Kalman filter prediction module to generate a predicted object location. The predicted object location is refined using a Kalman filter updating module, and the Kalman filter updating module is controlled by a controller module that monitors a similarity between the predicted object location and the moving object'"'"'s location in a previous image frame. Finally, a set of detected moving object locations in the sequence of image frames is output.

Claims

1. A system for object tracking, the system comprising:

one or more processors and a non-transitory memory having instructions encoded thereon such that when the instructions are executed, the one or more processors perform operations of;

detecting a location of a moving object in an image frame of a sequence of image frames using a motion-based object detection MogS module, thereby generating an object detection;

for each image frame in the sequence of image frames, predicting the location of the moving object in the next image frame through object tracking using a Kalman filter prediction module, thereby generating a predicted object location;

monitoring object tracking quality using a controller module;

refining the predicted object location using a Kalman filter updating module;

adjusting, by the controller module, the Kalman filter updating module according to a dissimilarity measure based on a spatial distance constraint and an appearance-based matching score, thereby improving object tracking quality and ameliorating gradual drifting of object tracking; and

outputting a set of detected moving object locations in the sequence of image frames.

2. The system as set forth in claim 1, wherein the one or more processors further perform an operation of determining a spatial distance d between each object detection in a current image frame and the object detection in the previous image frame with the controller module.

3. The system as set forth in claim 2, wherein the one or more processors further perform an operation of determining a dissimilarity metric diff based on the appearance difference between a detected image at the predicted object location and a target image template stored in a historical template database.

4. The system as set forth in claim 3, wherein the one or more processors further perform an operation of merging the spatial distance d with the dissimilarity metric diff to generate a combined dissimilarity measure D that quantifies the dissimilarity of the predicted object location with the moving object'"'"'s location in the previous image frame.

5. The system as set forth in claim 4, wherein the one or more processors further perform an operation of generating the combined dissimilarity measure D according to the following:

6. The system as set forth in claim 5, wherein the one or more processors further perform operations of:

updating a background model comprising a plurality of pixels belonging to a background of an image frame;

in each image frame, detecting at least one moving pixel by subtracting the current image frame from the background model, generating detected moving pixels;

processing the detected moving pixels to reduce noise; and

detecting blobs of foreground pixels through morphological operations and blob detection, resulting in an object detection.

7. The system as set forth in claim 1, wherein the one or more processors further perform operations of:

updating a background model comprising a plurality of pixels belonging to a background of an image frame;

in each image frame, detecting at least one moving pixel by subtracting the current image frame from the background model, generating detected moving pixels;

processing the detected moving pixels to reduce noise; and

detecting blobs of foreground pixels through morphological operations and blob detection, resulting in an object detection.

8. A computer-implemented method for object tracking, comprising:

an act of causing a data processor to execute instructions stored on a non-transitory memory such that upon execution, the data processor performs operations of;

detecting a location of a moving object in an image frame of a sequence of image detecting a location of a moving object in an image frame of a sequence of image frames using a motion-based object detection MogS module, thereby generating an object detection;

for each image frame in the sequence of image frames, predicting the location of the moving object in the next image frame through object tracking using a Kalman filter prediction module, thereby generating a predicted object location;

monitoring object tracking quality using a controller module;

refining the predicted object location using a Kalman filter updating module;

adjusting, by the controller module, the Kalman filter updating module according to a dissimilarity measure based on a spatial distance constraint and an appearance-based matching score, thereby improving object tracking quality and ameliorating gradual drifting of object tracking; and

outputting a set of detected moving object locations in the sequence of image frames.

9. The method as set forth in claim 8, wherein the data processor further performs an operation of determining a spatial distance d between each object detection in a current image frame and the object detection in the previous image frame with the controller module.

10. The method as set forth in claim 9, wherein the data processor further perform an operation of determining a dissimilarity metric diff based on the appearance difference between a detected image at the predicted object location and a target image template stored in a historical template database.

11. The method as set forth in claim 10, wherein the data processor further performs an operation of merging the spatial distance d with the dissimilarity metric diff to generate a combined dissimilarity measure D that quantifies the dissimilarity of the predicted object location with the moving object'"'"'s location in the previous image frame.

12. The method as set forth in claim 11, wherein the data processor further performs an operation of generating the combined dissimilarity measure D according to the following:

13. The method as set forth in claim 12, wherein the data processor further performs operations of:

updating a background model comprising a plurality of pixels belonging to a background of an image frame;

in each image frame, detecting at least one moving pixel by subtracting the current image frame from the background model, generating detected moving pixels;

processing the detected moving pixels to reduce noise; and

detecting blobs of foreground pixels through morphological operations and blob detection, resulting in an object detection.

14. The method as set forth in claim 8, wherein the data processor further performs operations of:

updating a background model comprising a plurality of pixels belonging to a background of an image frame;

in each image frame, detecting at least one moving pixel by subtracting the current image frame from the background model, generating detected moving pixels;

processing the detected moving pixels to reduce noise; and

detecting blobs of foreground pixels through morphological operations and blob detection, resulting in an object detection.

15. A computer program product for object tracking, the computer program product comprising computer-readable instructions stored on a non-transitory computer-readable medium that are executable by a computer having a processor for causing the processor to perform operations of:

detecting a location of a moving object in an image frame of a sequence of image frames using a motion-based object detection MogS module, thereby generating an object detection;

for each image frame in the sequence of image frames, predicting the location of the moving object in the next image frame through object tracking using a Kalman filter prediction module, thereby generating a predicted object location;

monitoring object tracking quality using a controller module;

refining the predicted object location using a Kalman filter updating module;

adjusting, by the controller module, the Kalman filter updating module according to a dissimilarity measure based on a spatial distance constraint and an appearance-based matching score, thereby improving object tracking quality and ameliorating gradual drifting of object tracking; and

outputting a set of detected moving object locations in the sequence of image frames.

16. The computer program product as set forth in claim 15, further comprising instructions for causing the processor to perform an operation of determining a spatial distance d between each object detection in a current image frame and the object detection in the previous image frame with the controller module.

17. The computer program product as set forth in claim 16, further comprising instructions for causing the processor to perform an operation of determining a dissimilarity metric diff based on the appearance difference between a detected image at the predicted object location and a target image template stored in a historical template database.

18. The computer program product as set forth in claim 17, further comprising instructions for causing the processor to perform an operation of merging the spatial distance d with the dissimilarity metric diff to generate a combined dissimilarity measure D that quantifies the dissimilarity of the predicted object location with the moving object'"'"'s location in the previous image frame.

19. The computer program product as set forth in claim 18, further comprising instructions for causing the processor to perform an operation of generating the combined dissimilarity measure D according to the following:

20. The computer program product as set forth in claim 19, further comprising instructions for causing the processor to perform operations of:

updating a background model comprising a plurality of pixels belonging to a background of an image frame;

in each image frame, detecting at least one moving pixel by subtracting the current image frame from the background model, generating detected moving pixels;

processing the detected moving pixels to reduce noise; and

detecting blobs of foreground pixels through morphological operations and blob detection, resulting in an object detection.

21. The computer program product as set forth in claim 15, further comprising instructions for causing the processor to perform operations of:

updating a background model comprising a plurality of pixels belonging to a background of an image frame;

in each image frame, detecting at least one moving pixel by subtracting the current image frame from the background model, generating detected moving pixels;

processing the detected moving pixels to reduce noise; and

detecting blobs of foreground pixels through morphological operations and blob detection, resulting in an object detection.