Multi-state target tracking

• US 7,825,954 B2
• Filed: 05/31/2005
• Issued: 11/02/2010
• Est. Priority Date: 05/31/2005
• Status: Active Grant

• Current Assignee: Motorola Solutions, Inc.
• Original Assignee: ObjectVideo Incorporated (Alarm.com Holdings, Inc.)
• Inventors: Rasheed, Zeeshan, Chosak, Andrew J., Haering, Niels, Zhang, Zhong, Brewer, Paul C., Venetianer, Peter L., Lipton, Alan J., Yin, Weihong, Liu, Haiying
• Primary Examiner(s): Philippe; Gims S
• Application Number:
  US11/139,600
• Publication Number:
  US 20060268111A1
• Time in Patent Office: 1,981 Days
• Field of Search: 348/169, 348/143, 348/171, 382/103, 706/14, 706/16
• US Class Current: 348/169
• CPC Class Codes: G01S 3/7864 T.V. type tracking systems

Abstract

A method of video analysis may comprise tracking a state of each target in a video through multiple frames of said video, each state indicating a visibility condition of a target.

Claims

1. A method for target tracking, comprising:

tracking a state of each target in a video through multiple frames of said video, each state indicating a visibility condition of a target;

maintaining connection data for each target, said connection data indicating one or more relationships between said target and at least one other target, said connection data being based at least on a state of each target; and

tracking at least one target in said video based on said tracking a state of each target and on said maintaining connection data for each target;

wherein said tracking at least one target includes managing a visible target;

wherein said managing a visible target comprises;

predicting a location of at least one blob;

matching at least one existing target, if one exists, to said at least one blob, to thus generate at least one matched target;

updating said at least one matched target, if one exists; and

creating a new target corresponding to a remaining blob for which there is no corresponding matched target; and

wherein a computer performs said tracking a state of each target, said maintaining connection data for each target, and said tracking at least one target.

2. The method of claim 1, wherein said matching is based on at least one of the properties selected from the group consisting of:

location overlap and appearance.

3. The method of claim 1, wherein said updating comprises:

estimating a target tracking stability status.

4. The method of claim 1, further comprising:

changing a state of a visible target for which there is no matching blob to a disappeared state.

5. The method of claim 1, wherein said tracking at least one target further includes managing an occluded target, and wherein said managing an occluded target comprises:

updating a target connection;

detecting an occluded target, if any; and

verifying that a detected occluded target, if any, is an occluded target.

6. The method of claim 5, wherein said verifying comprises:

finding potential equal targets;

computing one or more matching measures between said potential equal targets; and

making a verification decision based on a global matching criterion.

7. The method of claim 6, wherein said global matching criterion comprises a best satisfaction metric.

8. The method of claim 1, wherein said tracking at least one target further includes managing a stationary target, and wherein said managing a stationary target comprises:

detecting a stationary target; and

monitoring said stationary target.

9. The method of claim 8, wherein said detecting a stationary target comprises:

detecting if a target is a stopped target;

detecting if said stopped target is a passive target; and

detecting if said stopped target is a ghost target.

10. The method of claim 9, wherein said detecting if a target is a stopped target comprises:

comparing a current target slice to at least one previous target slice corresponding to said target to determine if there is any change;

computing a stopped time between a previous target slice and said current target slice; and

comparing said stopped time to a user-selected threshold to verify that said target is a stopped target.

11. The method of claim 9, wherein said detecting if said stopped target is a passive target comprises:

checking a current mobility state of said stopped target;

determining if said stopped target is a drop target;

determining if said stopped target has a parent target; and

performing a static passive test on said stopped target.

12. The method of claim 11, wherein said determining if said stopped target is a drop target comprises:

measuring a velocity change of said stopped target prior to the stopped target having stopped.

13. The method of claim 9, wherein said detecting if said stopped target is a ghost target comprises:

checking connection information of said stopped target to determine if said stopped target can be a ghost target;

performing foreground/background texture analysis to detect a ghost target;

classifying said ghost target; and

processing said ghost target.

14. The method of claim 13, wherein said performing foreground/background texture analysis comprises:

analyzing a boundary gradient intensity of said stopped target; and

analyzing a foreground/background histogram similarity.

15. The method of claim 13, wherein said classifying said ghost target comprises:

determining if said ghost target is an active ghost target or a passive ghost target.

16. The method of claim 15, wherein said processing said ghost target comprises:

aligning said ghost target with its corresponding real target if it is determined to be an active ghost target.

17. The method of claim 15, wherein said determining if said ghost target is an active ghost target or a passive ghost target comprises:

testing whether said ghost target has a single parent target and a single sibling target;

testing whether said parent target spatially overlaps with said ghost target at a point in time when said parent target is first stable and whether said parent target was visible when said ghost target was first created, if said ghost target has a single parent target; and

testing to determine if said ghost target has a size approximately the same as said sibling target and if said ghost target and said sibling target have approximately matching foreground histograms, if said parent target spatially overlaps with said ghost target at a point in time when said parent target is first stable and if said parent target was visible when said ghost target was first created.

18. The method of claim 9, further comprising:

detecting if a target is an active target if it is not a stopped target.

19. The method of claim 18, wherein said detecting if a target is an active target comprises:

determining, based on size, if said target has a potential parent target;

measuring a duration of motion of said target if said target has a potential parent target; and

comparing said duration of motion to a user-controllable threshold.

20. The method of claim 18, wherein said detecting if a target is an active target further comprises:

determining if said target is a particular type of active target if said duration of motion is not sufficient in comparison with said threshold to determine that said target is active.

21. The method of claim 8, wherein said monitoring said stationary target comprises:

determining if there is any visible target that spatially overlaps said stationary target;

checking connection data to determine if there is any connection between a visible target that spatially overlaps said stationary target and said stationary target, if there is a visible target that spatially overlaps said stationary target; and

determining if said stationary target begins to move.

22. The method of claim 21, wherein said monitoring said stationary target further comprises:

verifying that said stationary target is a stationary target if there is no visible target that spatially overlaps said stationary target; and

determining if said stationary target has disappeared.

23. The method of claim 22, wherein said verifying that said stationary target is a stationary target comprises:

computing a foreground same ratio; and

computing a background reappear ratio.

24. The method of claim 21, wherein said checking connection data comprises:

determining if said stationary target has begun to move; and

determining if said visible target is the same as said stationary target if said stationary target has begun to move.

25. The method of claim 24, wherein said checking connection data further comprises:

determining if said visible target is a ghost target if said stationary target has not begun to move; and

determining if said visible target has a sibling target that matches said stationary target if said visible target is a ghost target.

26. The method of claim 24, wherein said determining if said stationary target has begun to move comprises:

defining a target boundary region; and

comparing said stationary target to said target boundary region.

27. The method of claim 8, wherein said monitoring said stationary target comprises:

tracking both foreground and background images of said stationary target; and

monitoring both foreground and background images of said stationary target.

28. The method of claim 1, wherein said tracking at least one target further includes managing a disappeared target, and wherein said managing a disappeared target comprises:

detecting a disappeared target, if any;

verifying a previous disappeared target, if any, to check if it is still disappeared; and

performing disappeared target cleanup.

29. The method of claim 28, wherein said detecting a disappeared target comprises:

if there are no visible targets, changing a state of any previously occluded target to a disappeared state; and

if a visible target has become a disappeared target, changing occluded targets connected to said visible target to disappeared targets.

30. The method of claim 28, wherein said verifying a previous disappeared target comprises:

checking if said disappeared target has become visible.

31. The method of claim 1, further comprising:

maintaining one or more target slices for each target, each target slice corresponding to a frame of said video, wherein said tracking a state of each target in a video is performed based on said one or more target slices.

32. A method of analyzing video comprising:

detecting at least one property of said video, said at least one property selected from the group consisting of change and motion;

extracting foreground blobs from said video, based on results of said detecting;

tracking at least one target based on said foreground blobs, said tracking comprising the method according to claim 1.

33. A method of video surveillance comprising the method according to claim 32, and further comprising:

detecting a predetermined event in said video based on said tracking of at least one target.

34. A non-transitory computer-readable medium containing executable code that when executed on a computer causes the computer to perform the method of claim 1.

35. A video processing system comprising:

a video source providing video;

a computer coupled to said video source to receive said video; and

the non-transitory computer-readable medium according to claim 34, wherein said computer-readable medium is coupled to said computer to permit said computer to execute said computer-executable code.

36. Application-specific hardware adapted to implement the method according to claim 1.

37. A video analysis system comprising:

a video source providing video;

a computer coupled to said video source to receive said video;

a non-transitory computer-readable medium coupled to said computer;

a motion/change detection module;

a foreground blob extraction module coupled to receive output of said motion/change detection module; and

a target tracking module coupled to receive output of said foreground blob extraction module and to perform target tracking based on visibility state information of a target;

wherein said target tracking includes managing a stationary target;

wherein said managing a stationary target comprises;

detecting a stationary target; and

monitoring said stationary target;

wherein said monitoring said stationary target comprises;

determining if there is any visible target that spatially overlaps said stationary target;

checking connection data to determine if there is any connection between a visible target that spatially overlaps said stationary target and said stationary target, if there is a visible target that spatially overlaps said stationary target; and

determining if said stationary target begins to move; and

wherein said motion/change detection module, said foreground blob extraction module, and said target tracking module are stored on the non-transitory computer readable medium.

38. The video analysis system of claim 37, wherein said target tracking module is further adapted to perform target tracking based on target connection data.

39. The video analysis system of claim 37, further comprising:

an event detection module coupled to said target tracking module and adapted to detect at least one predetermined event based on an output of said target tracking module.

40. A method for target tracking, comprising:

tracking a state of each target in a video through multiple frames of said video, each state indicating a visibility condition of a target;

maintaining connection data for each target, said connection data indicating one or more relationships between said target and at least one other target, said connection data being based at least on a state of each target; and

tracking at least one target in said video based on said tracking a state of each target and on said maintaining connection data for each target;

wherein said tracking at least one target includes said managing a disappeared target; and

wherein managing a disappeared target comprises;

detecting a disappeared target, if any;

verifying a previous disappeared target, if any, to check if it is still disappeared; and

performing disappeared target cleanup.

41. The method of claim 40, wherein said detecting a disappeared target comprises:

if there are no visible targets, changing a state of any previously occluded target to a disappeared state; and

if a visible target has become a disappeared target, changing occluded targets connected to said visible target to disappeared targets.

42. The method of claim 40, wherein said verifying a previous disappeared target comprises:

checking if said disappeared target has become visible.