Autonomous lock-on target tracking with geospatial-aware PTZ cameras

• US 8,929,603 B1
• Filed: 03/17/2014
• Issued: 01/06/2015
• Est. Priority Date: 03/15/2013
• Status: Active Grant

• Current Assignee: PureTech Systems, Inc. (Teledyne Technologies Incorporated)
• Original Assignee: PureTech Systems, Inc. (Teledyne Technologies Incorporated)
• Inventors: Maali, Fereydoun, Swaminathan, Dilip, Barnes, Marvin Wade
• Primary Examiner(s): Vu, Kim
• Assistant Examiner(s): HU, FRED H.
• Application Number:
  US14/215,475
• Time in Patent Office: 295 Days
• Field of Search: None
• US Class Current: 382/103
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/20036 Morphological image processing
• CPC Class Codes: G06T 2207/30232 Surveillance
• CPC Class Codes: G06T 7/136 involving thresholding
• CPC Class Codes: G06T 7/194 involving foreground-background segmentation
• CPC Class Codes: G06T 7/254 involving subtraction of images
• CPC Class Codes: G06V 10/751 Comparing pixel values or logical combinations thereof, or feature values having positional relevance, e.g. template matching
• CPC Class Codes: G06V 20/52 Surveillance or monitoring of activities, e.g. for recognising suspicious objects recognising microscopic objects G06V20/69

Abstract

An autonomous lock-on target tracking system and method with geospatial-aware PTZ cameras includes a camera imaging a terrain space. The camera acquires images, and first and second images are aligned. A frame-differencing operation produces a resultant image including blobs corresponding to elements in the terrain space. One of the blobs is classified as an object and tracked as a target. The target is tracked by determining the distance between a centroid of the target and a center of a field of view of the camera, and instructing the camera to move through the distance. The distance is continually updated as the camera and the target move.

Claims

1. A video camera surveillance method comprising the steps of:

providing a camera imaging a terrain space in a field of view of the camera, the field of view having a center;

acquiring images in a video stream from the camera;

forming an image pair of first and second images between two of the images;

aligning the first and second images with respect to each other;

performing a frame differencing operation on the image pair to form a binary resultant image including blobs corresponding to elements in the terrain space, the frame differencing operation comprising;

determining an overlapping region of the first and second images;

determining a high illumination threshold and a low illumination threshold; and

for each pixel in the overlapping region, identifying the respective pixel as a foreground pixel if (1) the respective pixel in the first image deviates from a corresponding pixel in the second image by at least the high illumination threshold, (2) the respective pixel in the first image deviates from pixels adjacent to the corresponding pixel in the second image by at least the high illumination threshold, and (3) the corresponding pixel in the second image deviates from pixels adjacent to the respective pixel in the first image by at least the low illumination threshold;

classifying one of the blobs as an object;

selecting the object as a target, the target having a centroid;

tracking the target by moving the camera toward the centroid by an amount corresponding to the distance between the centroid and the center of the field of view; and

continually repeating the tracking step as the target moves in the terrain space.

2. The method of claim 1, wherein the step of performing a frame differencing operation further comprises identifying as blobs pixels in the resultant image which correspond to clustered foreground pixels in the overlapping region of the first and second images.

3. The method of claim 1, wherein the step of classifying one of the blobs as an object comprises:

each of the blobs having a respective centroid;

determining distances between the center of the field of view and each of the centroids of the blobs;

identifying a close blob from the distances between the center of the field of view and each of the centroids of the blobs; and

classifying the close blob as the object.

4. The method of claim 1, wherein the step of classifying one of the blobs as an object comprises:

identifying a first blob and a corresponding second blob in the resultant image;

determining a first set of pixels in the second image which correspond to the first blob in the resultant image;

determining a second set of pixels in the second image which correspond to the second blob in the resultant image;

dilating both the first and second set of pixels in the second image to form a dilated first set of pixels and a dilated second set of pixels in the second image;

measuring a first divergence between the first set of pixels and the dilated first set of pixels;

measuring a second divergence between the second set of pixels and the dilated second set of pixels;

classifying the first blob as the object and the second blob as a wake if the first divergence is greater than the second divergence; and

classifying the first blob as the wake and the second blob as the object if the second divergence is greater than the first divergence.

5. The method of claim 1, wherein the step of tracking the target further comprises zooming the camera to a discrete zoom level.

6. A video camera surveillance method comprising the steps of:

providing a camera imaging a terrain space in a field of view of the camera, the field of view having a center;

acquiring images in a video stream from the camera;

forming an image pair of first and second images between two of the images;

performing a frame differencing operation on the image pair to form a resultant image including blobs corresponding to elements in the terrain space;

detecting and discarding portions of the blobs which correspond to shadows by;

selecting one of the blobs and defining a bounding box around the one of the blobs;

defining a first half of the bounding box, an opposed second half of the bounding box, a first quadrant of the bounding box in the first half, and a second quadrant of the bounding box in the second half;

computing a first column-sum ratio for the first quadrant;

computing a second column-sum ratio for the second quadrant; and

either discarding a portion of the one of the blobs in the second half of the bounding box if the first column-sum ratio is greater than a user-defined threshold, or discarding another portion of the one of the blobs in the first half of the bounding box if the second column-sum ratio is greater than the user-defined threshold;

selecting a target from the blobs, the target having a centroid;

tracking the target by moving the camera toward the centroid by an amount corresponding to the distance between the centroid and the center of the field of view; and

continually repeating the tracking step as the target moves in the terrain space.

7. The method of claim 6, wherein:

pixels in the first and second images of the image pair have grayscale values; and

the resultant image is a binary image.

8. The method of claim 7, wherein the step of performing a frame differencing operation comprises:

determining an overlapping region of the first and second images;

determining a high illumination threshold and a low illumination threshold; and

for each pixel in the overlapping region, identifying the respective pixel as a foreground pixel if (1) the respective pixel in the first image deviates from a corresponding pixel in the second image by at least the high illumination threshold, (2) the respective pixel in the first image deviates from pixels adjacent to the corresponding pixel in the second image by at least the high illumination threshold, and (3) the corresponding pixel in the second image deviates from pixels adjacent to the respective pixel in the first image by at least the low illumination threshold.

9. The method of claim 8, further comprising identifying as blobs pixels in the resultant image which correspond to clustered foreground pixels in the overlapping region of the first and second images.

10. The method of claim 6, wherein the step of selecting one of the blobs as a target comprises:

each of the blobs having a respective centroid;

determining distances between the center of the field of view and each of the centroids of the blobs;

identifying a close blob based on the distances between the center of the field of view and each of the centroids of the blobs; and

selecting the close blob as the target.

11. The method of claim 6, wherein the step of selecting a target from the blobs comprises:

identifying a first blob and a corresponding second blob in the resultant image;

determining a first set of pixels in the second image which correspond to the first blob in the resultant image;

determining a second set of pixels in the second image which correspond to the second blob in the resultant image;

dilating both the first and second set of pixels in the second image to form a dilated first set of pixels and a dilated second set of pixels in the second image;

measuring a first divergence between the first set of pixels and the dilated first set of pixels;

measuring a second divergence between the second set of pixels and the dilated second set of pixels;

classifying the first blob as the object and the second blob as a wake if the first divergence is greater than the second divergence; and

classifying the first blob as the wake and the second blob as the object if the second divergence is greater than the first divergence.

12. The method of claim 6, wherein the step of tracking the target further comprises zooming the camera to a discrete zoom level.

13. A video camera surveillance method comprising the steps of:

providing a camera imaging a terrain space in a field of view of the camera, the field of view having a center;

acquiring images in a video stream from the camera;

forming an image pair of first and second images between two of the images;

aligning the first and second images with respect to each other;

performing a frame differencing operation on the image pair to form a resultant image including blobs corresponding to elements in terrain space;

selecting one of the blobs as a target, the target having a centroid, wherein the step of selecting one of the blobs as the target includes;

identifying a first blob and a corresponding second blob in the resultant image;

determining a first set of pixels in the second image which correspond to the first blob in the resultant image;

determining a second set of pixels in the second image which correspond to the second blob in the resultant image;

dilating both the first and second set of pixels in the second image to form a dilated first set of pixels and a dilated second set of pixels in the second image;

measuring a first divergence between the first set of pixels and the dilated first set of pixels;

measuring a second divergence between the second set of pixels and the dilated second set of pixels;

classifying the first blob as the target if the first divergence is greater than the second divergence; and

classifying the second blob as the target if the second divergence is greater than the first divergence;

tracking the target by moving the camera toward the centroid by an amount corresponding to the distance between the centroid and the center of the field of view; and

continually repeating the tracking step as the target moves in the terrain space.

14. The method of claim 13, wherein the step of performing a frame differencing operation comprises:

determining an overlapping region of the first and second images;

determining a high illumination threshold and a low illumination threshold;

for each pixel in the overlapping region, identifying the respective pixel as a foreground pixel if (1) the respective pixel in the first image deviates from a corresponding pixel in the second image by at least the high illumination threshold, (2) the respective pixel in the first image deviates from pixels adjacent to the corresponding pixel in the second image by at least the high illumination threshold, and (3) the corresponding pixel in the second image deviates from pixels adjacent to the respective pixel in the first image by at least the low illumination threshold; and

identifying as blobs pixels in the resultant image which correspond to clustered foreground pixels in the overlapping region of the first and second images.

15. The method of claim 13, wherein the step of tracking the target further comprises zooming the camera to a discrete zoom level.

16. The method of claim 13, wherein the step of detecting and discarding portions of the blobs which correspond to shadows comprises:

selecting one of the blobs;

defining a bounding box around the one of the blobs;

defining a first half of the bounding box, an opposed second half of the bounding box, a first quadrant of the bounding box in the first half, and a second quadrant of the bounding box in the second half;

computing a first column-sum ratio for the first quadrant;

computing a second column-sum ratio for the second quadrant; and

either discarding a portion of the one of the blobs in the second half of the bounding box if the first column-sum ratio is greater than a user-defined threshold, or discarding another portion of the one of the blobs in the first half of the bounding box if the second column-sum ratio is greater than the user-defined threshold.