Spurious object detection in a video surveillance system

• US 7,391,907 B1
• Filed: 10/01/2004
• Issued: 06/24/2008
• Est. Priority Date: 10/01/2004
• Status: Active Grant

• Current Assignee: Motorola Solutions, Inc.
• Original Assignee: ObjectVideo Incorporated (Alarm.com Holdings, Inc.)
• Inventors: Brewer, Paul C., Lipton, Alan J., Liu, Haiying, Clark, John I. W., Venetianer, Péter L.
• Primary Examiner(s): Wu; Jingge
• Assistant Examiner(s): Lee; John W
• Application Number:
  US10/954,309
• Time in Patent Office: 1,362 Days
• Field of Search: 382/133, 382/107, 382/224, 348/143, 348/155, 348/669
• US Class Current: 382/224
• CPC Class Codes: G06V 10/62 relating to a temporal dimension, e.g. time-based feature extraction; Pattern tracking
• CPC Class Codes: G06V 20/52 Surveillance or monitoring of activities, e.g. for recognising suspicious objects recognising microscopic objects G06V20/69

Abstract

A system detects an object in frames of a video sequence to obtain a detected object, tracks the detected object in the frames of the video sequence to obtain a tracked object, and classifies the tracked object as a real object or a spurious object based on spatial and/or temporal properties of the tracked object.

Claims

1. A computer-readable medium comprising software to detect spurious objects, which when executed by a computer system, cause the computer system to perform operations comprising a method of:

detecting an object in frames of a video sequence to obtain a detected object;

tracking said detected object in said frames of said video sequence to obtain a tracked object;

classifying said tracked object as a real object or a spurious object based on at least one of;

a shape consistency metric, including;

extracting an enlarged view (blob) of said object for each of two frames;

creating a mask for each blob;

computing a centroid for each blob;

aligning said masks by said centroids, whereby an aligned mask is created; and

computing an overlap ratio from said aligned mask from a number of overlapping pixels in the aligned mask and a total number of pixels in said aligned mask,

a size consistency metric, including;

measuring a number of pixels in said detected object in each of two frames; and

computing the ratio of said number of pixels in each of said two frames,

a texture consistency metric,

a color consistency metric, including;

computing a color histogram of said object in a plurality of frames, wherein said color histogram for n previous frames is a vector with m elements, where each element of said vector represents a value of said color histogram;

computing a color consistency vector using color histogram vectors for two frames; and

computing a color consistency metric based on said color consistency vector and m, an intensity consistency metric,

a speed consistency metric, including;

computing an instantaneous speed of said object in two frames; and

computing a speed consistency metric based on a change in said instantaneous speed in said two frames,

a direction of motion consistency metric, including;

computing an instantaneous direction of motion of said object in two frames; and

computing a direction of motion consistency metric based on a change in said instantaneous direction of motion in said two frames,

a salient motion metric,

an absolute motion metric, or

a persistent motion metric of said tracked object.

2. A computer-readable medium as in claim 1, further comprising classifying said object based on a size metric including:

determining a current size of said object at a position in a frame;

determining a mean size at said position;

computing a standard deviation of a size at said position; and

computing a size metric from said current size, said mean size and said standard deviation.

3. A computer-readable medium as in claim 1, wherein classifying said object based on a texture consistency metric comprises:

computing an object texture for said object in two frames; and

computing a texture consistency metric based on a change in said computed object textures in said two frames.

4. A computer-readable medium as in claim 1, wherein classifying said object based on a intensity consistency metric comprises:

computing an average intensity metric of said object in two frames; and

computing an intensity consistency metric based on a change in said average intensity metric in said two frames.

5. A computer-readable medium as in claim 1, wherein classifying said object based on a salient motion metric comprises:

determining a position of said object over a window of interest of n frames;

computing a salient motion metric based on a distance between said object position at the start and end times of said window of interest, and a total distance traveled by the object.

6. A computer-readable medium as in claim 1, wherein classifying said object based on an absolute motion metric comprises:

computing a distance between a position of said object at a start and an end of a time window of n frames.

7. A computer-readable medium as in claim 1, wherein classifying said object based on a persistent motion metric comprises:

computing a first number of frames in which said object appears over a window of interest having a second number of frames; and

computing a persistent motion metric based on a ratio of said first number of frames to said second number of frames.

8. A computer-readable medium as in claim 1, wherein said object is classified based on at least two of said classification metrics.

9. A computer-readable medium as in claim 8, wherein said object is classified based on at least three of said classification metrics.

10. A computer-readable medium as in claim 9, wherein said object is classified based on at least four of said classification metrics.

11. A computer-readable medium as in claim 1, wherein said object is classified based on at least one time-averaged classification metric.

12. A computer-readable medium as in claim 1, wherein said object is classified based on a combination of at least two classification metrics, each said classification metric quantifying a spatial property and/or a temporal property of said tracked object.

13. A computer-readable medium as in claim 12, wherein said classification metrics are combined using neural networks.

14. A computer-readable medium as in claim 12, wherein said classification metrics are combined using linear discriminant analysis.

15. A computer-readable medium as in claim 12, wherein said classification metrics are combined using non-linear weighting.

16. A computer-readable medium as in claim 12, wherein said classification metrics are time-average weighted over at least one time window.

17. A computer-readable medium as in claim 1, wherein said method further comprising, if said object is classified as said real object, further classifying said real object as to a type of real object.

18. A computer-readable medium as in claim 1, wherein said method further comprising if said object is classified as said real object, initiating a triggered response.

19. A computer-readable medium as in claim 1, wherein said frames are consecutive.

20. A computer-readable medium as in claim 1, wherein said frames are non-consecutive.

33. A computer-readable medium as in claim 1, wherein classifying said object based on an intensity consistency metric comprises:

computing an intensity histogram of said object in a plurality of frames, wherein said intensity histogram for n previous frames is a vector with m elements, where each element represents a value of said intensity histogram;

computing an intensity consistency vector using intensity histogram vectors for two frames; and

computing an intensity consistency metric based on said intensity consistency vector and m.

21. An apparatus to perform a method to detect spurious objects, said method comprising:

detecting an object in frames of a video sequence to obtain a detected object;

tracking said detected object in said frames of said video sequence to obtain a tracked object;

classifying said tracked object as a real object or a spurious object based on at least one of;

a shape consistency metric, including;

extracting an enlarged view (blob) of said object for each of two frames;

creating a mask for each blob;

computing a centroid for each blob;

aligning said masks by said centroids, whereby an aligned mask is created; and

computing an overlap ratio from said aligned mask from a number of overlapping pixels in the aligned mask and a total number of pixels in said aligned mask,

a size consistency metric, including;

measuring a number of pixels in said detected object in each of two frames; and

computing the ratio of said number of pixels in each of said two frames,

a texture consistency metric,

a color consistency metric, including;

computing a color histogram of said object in a plurality of frames, wherein said color histogram for n previous frames is a vector with m elements, where each element of said vector represents a value of said color histogram;

computing a color consistency vector using color histogram vectors for two frames; and

computing a color consistency metric based on said color consistency vector and m,

an intensity consistency metric,

a speed consistency metric, including;

computing an instantaneous speed of said object in two frames; and

computing a speed consistency metric based on a change in said instantaneous speed in said two frames,

a direction of motion consistency metric, including;

computing an instantaneous direction of motion of said object in two frames; and

computing a direction of motion consistency metric based on a change in said instantaneous direction of motion in said two frames,

a salient motion metric,

an absolute motion metric, or

a persistent motion metric of said tracked object.

22. An apparatus as in claim 21, wherein said object is classified based on at least two of said classification metrics.

23. An apparatus as in claim 22, wherein said object is classified based on at least three of said classification metrics.

24. An apparatus as in claim 23, wherein said object is classified based on at least four of said classification metrics.

25. A method to detect spurious objects, comprising:

detecting an object in frames of a video sequence to obtain a detected object;

tracking said detected object in said frames of said video sequence to obtain a tracked object;

classifying said tracked object as a real object or a spurious object based on at least one of the following classification metrics;

a shape consistency metric, including;

extracting an enlarged view (blob) of said object for each of two frames;

creating a mask for each blob;

computing a centroid for each blob;

aligning said masks by said centroids, whereby an aligned mask is created; and

computing an overlap ratio from said aligned mask from a number of overlapping pixels in the aligned mask and a total number of pixels in said aligned mask,

a size consistency metric, including;

measuring a number of pixels in said detected object in each of two frames; and

computing the ratio of said number of pixels in each of said two frames,

a texture consistency metric,

a color consistency metric, including;

computing a color histogram of said object in a plurality of frames, wherein said color histogram for n previous frames is a vector with m elements, where each element of said vector represents a value of said color histogram;

computing a color consistency vector using color histogram vectors for two frames; and

computing a color consistency metric based on said color consistency vector and m,

an intensity consistency metric,

a speed consistency metric, including;

computing an instantaneous speed of said object in two frames; and

computing a speed consistency metric based on a change in said instantaneous speed in said two frames,

a direction of motion consistency metric, including;

computing an instantaneous direction of motion of said object in two frames, and

computing a direction of motion consistency metric based on a change in said instantaneous direction of motion in said two frames,

a salient motion metric,

an absolute motion metric, or

a persistent motion metric of said tracked object.

26. A method as in claim 25, wherein said object is classified based on at least two of said classification metrics.

27. A method as in claim 26, wherein said object is classified based on at least three of said classification metrics.

28. A method as in claim 27, wherein said object is classified based on at least four of said classification metrics.

29. A system to detect spurious objects, comprising:

means for detecting an object in frames of a video sequence to obtain a detected object;

means for tracking said detected object in said frames of said video sequence to obtain a tracked object;

means for classifying said tracked object as a real object or a spurious object based on at least one of the following classification metrics;

a shape consistency metric, including;

means for extracting an enlarged view (blob) of said object for each of two frames,

means for creating a mask for each blob,

means for computing a centroid for each blob,

means for aligning said masks by said centroids, whereby an aligned mask is created; and

means for computing an overlap ratio from said aligned mask from a number of overlapping pixels in the aligned mask and a total number of pixels in said aligned mask,

a size consistency metric, including;

means for measuring a number of pixels in said detected object in each of two frames; and

means for computing the ratio of said number of pixels in each of said two frames,

a texture consistency metric,

a color consistency metric, including;

means for computing an instantaneous speed of said object in two frames; and

computing a speed consistency metric based on a change in said instantaneous speed in said two frames,

an intensity consistency metric,

a speed consistency metric,

a direction of motion consistency metric, including;

means for computing an instantaneous direction of motion of said object in two frames and

means for computing a direction of motion consistency metric based on a change in said instantaneous direction of motion in said two frames,

a salient motion metric,

an absolute motion metric, or

a persistent motion metric of said tracked object.

30. A system as in claim 29, wherein said object is classified based on at least two of said classification metrics.

31. A system as in claim 30, wherein said object is classified based on at least three of said classification metrics.

32. A system as in claim 31, wherein said object is classified based on at least four of said classification metrics.