Trajectory matching across disjointed video views

• US 10,095,954 B1
• Filed: 01/17/2013
• Issued: 10/09/2018
• Est. Priority Date: 01/17/2012
• Status: Active Grant

• Current Assignee: Cognyte Technologies Israel Ltd. (Cognyte Software Ltd.)
• Original Assignee: Verint Systems Limited (Verint Systems Incorporated)
• Inventors: Daliyot, Shahar, Zimerman, Gadi, Kiro, Shmuel, Golan, Oren
• Primary Examiner(s): CESE, KENNY A
• Application Number:
  US13/743,635
• Time in Patent Office: 2,091 Days
• Field of Search: 382103, 382107, 382180, 382224, 382225
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/30196 Human being; Person
• CPC Class Codes: G06T 2207/30232 Surveillance
• CPC Class Codes: G06T 2207/30241 Trajectory
• CPC Class Codes: G06T 7/246 using feature-based methods, e.g. the tracking of corners or segments
• CPC Class Codes: G06T 7/292 Multi-camera tracking
• CPC Class Codes: G06V 10/62 relating to a temporal dimension, e.g. time-based feature extraction; Pattern tracking
• CPC Class Codes: G06V 20/52 Surveillance or monitoring of activities, e.g. for recognising suspicious objects recognising microscopic objects G06V20/69

Abstract

Embodiments disclosed herein provide systems and methods for matching trajectories across disjointed video views. In a particular embodiment, a method provides receiving a plurality of tagged trajectories that are tagged with an indicator of which trajectory pairs of the plurality of tagged trajectories are matching trajectory pairs and processing each of the trajectory pairs using each of a plurality of trajectory matching algorithms. The method further provides creating a model for matching trajectories based on the output of each of the plurality of trajectory matching algorithms for each of the pairs. The method further provides receiving a pair of trajectories and processing the pair of trajectories using each of the plurality of the trajectory matching algorithms. The method further provides processing the output of each of the plurality of the trajectory matching algorithms for the pair of trajectories using the model to determine whether the pair of trajectories is matching.

Claims

1. A method for tracking the trajectory of an object moving in regions covered by a video surveillance system, the method comprising:

locating objects in video captured by a first camera covering a region;

generating a signature for each object in the region, wherein the signature for each object comprises views of the object obtained from a subset of frames selected from the video of the region, and wherein each view of the object is a portion of a video frame that shows the object from a particular view point, and wherein the frames in the subset of frames are spaced apart in time to decrease the likelihood of redundant views in the signature;

computing a trajectory for each object in the region from the captured video;

determining that a particular object that traced a trajectory in the region has also traced a trajectory in another region, wherein the determining comprises;

locating objects in video captured by a second camera covering the other region,

generating a signature for each object in the other region, wherein the signature for each object in the other region comprises views of the object obtained from a subset of frames selected from the video captured by the second camera covering the other region, wherein the frames in the subset of frames are spaced apart in time to decrease the likelihood of redundant views in the signature,

comparing the signatures from the region to the signatures from the other region to find a match, the match determined by a number of matching views exceeding a threshold, and

updating the signature for the particular object by adding views of the particular object in the other region, wherein only views of the particular object in the other region that are different from the views already contained in the signature of the particular object are added;

creating a composite trajectory for the particular object that includes the trajectory in each region and an estimation of the trajectory between the region and the other region not covered by the first camera or the second camera;

repeating the determining and creating for other regions using the updated signature for the particular object so that as the particular object is tracked its signature is expanded to include different views of the particular object; and

displaying a map of the region and the other region with the composite trajectory of the object superimposed over the map.

2. The method according to claim 1, wherein comparing comprises image recognition.

3. The method according to claim 1, wherein the object is a person and the views are of the person'"'"'s head.

7. The method according to claim 1, wherein the frames in the subset of frames are spaced apart in time according to a rate of motion of the object relative to a camera frame rate.

4. A non-transitory computer readable medium having instructions stored thereon for operating a video processing system, wherein the instructions, when executed by the video processing system, direct the video processing system to:

locate objects in video captured by a first camera covering a region;

generate a signature for each object in the region, wherein the signature for each object comprises views of each object obtained from a subset of frames selected from the video of the region, and wherein each view of each object comprises a portion of a video frame that shows the object from a particular view point, and wherein the frames in the subset of frames are spaced apart in time to decrease the likelihood of redundant views in the signature;

compute a trajectory for each object in the region from the captured video;

determine that a particular object that traced a trajectory in the region has also traced a trajectory in another region, wherein the determining directs the video processing system to;

locate objects in video captured by a second camera covering the other region,

generate a signature for each object in the other region, wherein the signature for each object in the other region comprises views of the object obtained from a subset of frames selected from the video captured by the second camera covering the other region, wherein the frames in the subset of frames are spaced apart in time to decrease the likelihood of redundant views in the signature,

compare the signatures from the region to the signatures from the other region to find a match, the match determined by a number of matching views exceeding a threshold, and

update the signature for the particular object by adding views of the particular object in the other region, wherein only views of the particular object in the other region that are different from the views already contained in the signature of the particular object are added;

create a composite trajectory for the particular object that includes the trajectory in each region and an estimation of the trajectory between the region and the other region not covered by the first camera or the second camera;

repeat the determining and creating for other regions using the updated signature for the particular object so that as the particular object is tracked its signature is expanded to included different views of the particular object; and

display a map that includes the region and the other region with the composite trajectory of the object superimposed over the map.

5. The non-transitory computer readable medium according to claim 4, wherein comparing comprises image recognition.

6. The non-transitory computer readable medium according to claim 4, wherein the object is a person and the views are of the person'"'"'s head.

8. The non-transitory computer readable medium according to claim 4, wherein the frames in the subset of frames are spaced apart in time according to a rate of motion of the object relative to a camera frame rate.