METHOD AND SYSTEM FOR SELECTING A VIDEO ANALYSIS METHOD BASED ON AVAILABLE VIDEO REPRESENTATION FEATURES

• US 20120288140A1
• Filed: 05/13/2011
• Published: 11/15/2012
• Est. Priority Date: 05/13/2011
• Status: Abandoned Application

• Current Assignee: Carnegie Mellon University, Motorola Solutions, Inc.
• Original Assignee: Carnegie Mellon University, Motorola Solutions, Inc.
• Inventors: SUPER, BOAZ J., HAUPTMANN, ALEXANDER
• Application Number:
  US13/107,427
• Publication Number:
  US 20120288140A1
• Time in Patent Office: Days
• US Class Current: 382/103
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/30196 Human being; Person
• CPC Class Codes: G06T 2207/30232 Surveillance
• CPC Class Codes: G06T 2207/30241 Trajectory
• CPC Class Codes: G06T 7/292 Multi-camera tracking
• CPC Class Codes: G06V 20/52 Surveillance or monitoring of activities, e.g. for recognising suspicious objects recognising microscopic objects G06V20/69

Abstract

A method is performed for selecting a video analysis method based on available video representation features. The method includes: determining a plurality of available video representation features for a first video output from a first video source and for a second video output from a second video source; and analyzing the plurality of video representation features as compared to at least one threshold to select one of a plurality of video analysis methods to track an object between the first and the second videos.

Claims

1. A method for selecting a video analysis method based on available video representation features, the method comprising:

determining a plurality of available video representation features for a first video output from a first video source and for a second video output from a second video source;

analyzing the plurality of video representation features as compared to at least one threshold to select one of a plurality of video analysis methods to track an object between the first and the second videos.

2. The method of claim 1, wherein the plurality of video analysis methods comprises a spatio-temporal feature matching method and a spatial feature matching method.

3. The method of claim 2, wherein the plurality of video analysis methods further comprises at least one alternative matching method to the spatio-temporal feature matching method and the spatial feature matching method.

4. The method of claim 3, wherein the at least one alternative matching method comprises at least one of a hybrid spatio-temporal and spatial feature matching method or an appearance matching method.

5. The method of claim 4, wherein the determined plurality of available video representation features is of a type that includes at least one of:

a set of spatio-temporal features for the first video, a set of spatial features for the first video, a set of appearance features for the first video, a set of spatio-temporal features for the second video, or a set of spatial features for the second video, or a set of appearance features for the second video.

6. The method of claim 5 further comprising:

determining an angle between the two video sources and comparing the angle to a first angle threshold;

selecting the appearance matching method to track the object when the first angle is larger than the first angle threshold;

when the angle is less than the first angle threshold, the method further comprises;

determining the type of the plurality of available video representation features;

when the type of the plurality of available video representation features comprises the sets of spatio-temporal features for the first and second videos, the method further comprises;

determining from the sets of spatio-temporal features for the first and second videos a number of corresponding spatio-temporal feature pairs;

comparing the number of corresponding spatio-temporal feature pairs to a second threshold; and

when the number of corresponding spatio-temporal feature pairs exceeds the second threshold, selecting the spatio-temporal feature matching method to track the object;

when the type of the plurality of available video representation features comprises the sets of spatial features for the first and second video and the set of spatio-temporal features for the first or second videos, the method further comprises;

determining from the sets of spatial features for the first and second videos a number of corresponding spatial feature pairs and comparing the number of corresponding spatial feature pairs to a third threshold;

determining a first number of spatio-temporal features near the set of spatial features for the first video or a second number of spatio-temporal features near the set of spatial features for the second video, and comparing the first or second numbers of spatio-temporal features to a fourth threshold; and

when the number of corresponding spatial feature pairs exceeds the third threshold and the first or the second numbers of spatio-temporal features exceeds the fourth threshold, selecting the hybrid spatio-temporal and spatial feature matching method to track the object;

otherwise, selecting the spatial feature matching method to track the object.

7. The method of claim 4, wherein the hybrid spatio-temporal and spatial feature matching method comprises a motion-scale-invariant feature transform (motion-SIFT) matching method and a scale-invariant feature transform (SIFT) matching method.

8. The method of claim 2, wherein the spatio-temporal feature matching method comprises one of a motion-SIFT matching method or a Spatio-Temporal Invariant Point matching method.

9. The method of claim 2, wherein the spatial feature matching method comprises one of a scale-invariant feature transform matching method, a HoG matching method, a Maximally Stable Extremal Region matching method, or an affine-invariant patch matching method.

10. The method for claim 1, wherein the determining and analyzing of the plurality of video representation features is performed on a frame by frame basis for the first and second videos.

11. The method of claim 1 further comprising determining at least one of a spatial transform or a temporal transform between the first and second video sources.

12. The method of claim 11, wherein determining the at least one of the spatial transform or the temporal transform comprises:

determining a type of the plurality of available video representation features;

when the type of the plurality of available video representation features comprises both spatio-temporal features and spatial features, the method further comprising;

determining a spatial transformation using correspondences between stable spatial features; and

determining a temporal transformation between the first and second video by finding correspondences between the spatio-temporal features or non-stable spatial features for the first and second videos.

when the type of the plurality of available video representation features comprises spatio-temporal features but not spatial features, the method further comprising;

determining a Bag of Features (BoF) representation from the spatio-temporal features;

determining a temporal transformation by an optimizing match of the BoF representations; and

determining a spatial transformation between the spatio-temporal features on temporally registered video.

13. A non-transitory computer-readable storage element having computer readable code stored thereon for programming a computer to perform a method for selecting a video analysis method based on available video representation features, the method comprising:

determining a plurality of available video representation features for a first video output from a first video source and for a second video output from a second video source;

analyzing the plurality of video representation features as compared to at least one threshold to select one of a plurality of video analysis methods to track an object between the first and the second videos.