Robust object tracking system

• US 20070291984A1
• Filed: 06/15/2006
• Published: 12/20/2007
• Est. Priority Date: 06/15/2006
• Status: Active Grant

• Current Assignee: Northwestern University, Omron Corporation
• Original Assignee: Northwestern University, Omron Corporation
• Inventors: Lao, Shihong, Wu, Ying, Yang, Ming, Dai, Shengyang
• Granted Patent: US 8,050,453 B2
• Time in Patent Office: Days
• US Class Current: 382/103
• CPC Class Codes: G06T 7/20 Analysis of motion motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51

Abstract

A method for tracking objects includes identifying a target, identifying a plurality of auxiliary objects related to the target, and tracking the target using the plurality of auxiliary objects.

Claims

1. A method for tracking objects, comprising:

identifying a target;

identifying a plurality of auxiliary objects related to the target; and

tracking the target using the plurality of auxiliary objects.

2. The method of claim 1, further comprising:

verifying consistency of the tracking using a robust fusion technique.

3. The method of claim 2, wherein verifying consistency further comprises:

temporarily stopping identification of the plurality of auxiliary objects if the target is an outlier.

4. The method of claim 2, wherein verifying consistency further comprises:

excluding an auxiliary object if the auxiliary object is an outlier.

5. The method of claim 1, wherein identifying the target further comprises using a face detection algorithm.

6. The method of claim 1, wherein identifying the target comprises specifying the target manually.

7. The method of claim 1, wherein tracking the target comprises using an elliptical tracker to track the target.

8. The method of claim 7, wherein the elliptical tracker uses a frame difference image to verify the location of the target.

9. The method of claim 1, wherein a data mining technique is used to identify the plurality of auxiliary objects related to the target.

10. The method of claim 9, the data mining technique further comprising:

using a frequent pattern growth (FP growth) technique to select a plurality of candidate auxiliary objects with high co-occurrent frequency with the target; and

selecting the plurality of auxiliary objects from the plurality of candidate auxiliary objects,

wherein each of the plurality of auxiliary objects maintains a high motion correlation with the target.

11. The method of claim 10, further comprising using a mean shift technique to:

track the plurality of candidate auxiliary objects; and

determine motion correlations between the plurality of candidate auxiliary objects and the target.

12. The method of claim 11, wherein using the frequent pattern growth technique further comprises removing a candidate auxiliary object from the plurality of candidate auxiliary objects if the candidate auxiliary object cannot be tracked for a number of frames.

13. The method of claim 12, wherein the number of frames is four.

14. The method of claim 1, wherein a belief propagation technique is used to track the target using a result of tracking one of the plurality of auxiliary objects as input.

15. The method of claim 1, wherein each of the plurality of auxiliary objects is associated with:

a first value quantifying co-occurrence with the target; and

a second value quantifying consistent motion correlation with the target.

16. The method of claim 15, wherein the first value and second value are concurrently maintained above a first threshold and a second threshold within a set time interval.

17. The method of claim 1, wherein auxiliary objects comprise image regions.

18. The method of claim 17, wherein image regions are determined using a split-merge quad-tree color segmentation.

19. The method of claim 1, wherein the target and the plurality of auxiliary objects are modeled using a star topology Markov random field.

20. The method of claim 19, wherein the target corresponds to a hub of the star topology Markov random field.

21. The method of claim 1, wherein the plurality of auxiliary objects are discovered dynamically.

22. A camera configured to track a target using the method of claim 1.

23. The camera of claim 22, wherein the camera is a digital camera.

24. The camera of claim 22, wherein the camera is a video camera.

25. The method of claim 24, wherein the video camera is used for surveillance purposes.

26. A tracking system configured to:

receive an image sequence;

identify a target in the image sequence;

identify a plurality of auxiliary objects related to the target; and

track the target using the plurality of auxiliary objects.

27. The tracking system of claim 26, further configured to:

verify consistency of the tracking using a robust fusion technique.

28. The tracking system of claim 27, wherein verifying consistency further comprises:

temporarily stopping identification of the plurality of auxiliary objects if the target is an outlier.

29. The tracking system of claim 27, wherein verifying consistency further comprises:

excluding an auxiliary object if the auxiliary object is an outlier.

30. The tracking system of claim 26, wherein identifying the target further comprises using a face detection algorithm.

31. The tracking system of claim 26, wherein identifying the target comprises specifying the target manually.

32. The tracking system of claim 26, wherein tracking the target comprises using an elliptical tracker to track the target.

33. The tracking system of claim 32, wherein the elliptical tracker uses a frame difference image to verify the location of the target.

34. The tracking system of claim 26, wherein a data mining technique is used to identify the plurality of auxiliary objects related to the target.

35. The tracking system of claim 34, the data mining technique further comprising:

using a frequent pattern growth (FP growth) technique to select a plurality of candidate auxiliary objects with high co-occurrent frequency with the target; and

selecting the plurality of auxiliary objects from the plurality of candidate auxiliary objects,

wherein each of the plurality of auxiliary objects maintains a high motion correlation with the target.

36. The tracking system of claim 35, further comprising using a mean shift technique to:

track the plurality of candidate auxiliary objects; and

determine motion correlations between the plurality of candidate auxiliary objects and the target.

37. The tracking system of claim 36, wherein using the frequent pattern growth technique further comprises removing a candidate auxiliary object from the plurality of candidate auxiliary objects if the candidate auxiliary object cannot be tracked for a number of frames.

38. The tracking system of claim 36, wherein the number of frames is four.

39. The tracking system of claim 26, wherein a belief propagation technique is used to track the target using a result of tracking one of the plurality of auxiliary objects as input.

40. The tracking system of claim 26, wherein each of the plurality of auxiliary objects is associated with:

a first value quantifying co-occurrence with the target; and

a second value quantifying consistent motion correlation with the target.

41. The tracking system of claim 40, wherein the first value and second value are concurrently maintained above a first threshold and a second threshold within a set time interval.

42. The tracking system of claim 26, wherein auxiliary objects comprise image regions.

43. The tracking system of claim 41, wherein image regions are determined using a split-merge quad-tree color segmentation.

44. The tracking system of claim 26, wherein the target and the plurality of auxiliary objects are modeled using a star topology Markov random field.

45. The tracking system of claim 44, wherein the target corresponds to a hub of the star topology Markov random field.

46. The tracking system of claim 26, wherein the plurality of auxiliary objects are discovered dynamically.

47. The tracking system of claim 26, wherein a plurality of mean shift trackers are used to track the plurality of auxiliary objects.

48. A computer usable medium having computer readable program code embodied therein for causing a computer system to execute a method for tracking objects, comprising:

identifying a target;

identifying a plurality of auxiliary objects related to the target; and

tracking the target using the plurality of auxiliary objects.

49. A method for tracking a target head in an image sequence, comprising:

designating the target head in the image sequence;

tracking the target head using an elliptical tracker;

identifying a plurality of auxiliary objects in the image sequence related to the target head, wherein identifying comprises;

splitting an image in the image sequence using a split-merge quad tree color segmentation algorithm to form a plurality of image regions;

creating a plurality of item candidates by merging adjacent image regions of the plurality of image regions with similar appearances into item candidates;

pruning item candidates inappropriate for tracking from the plurality of item candidates to obtain a reduced item candidate set;

creating a plurality of quantized item candidates by k-means clustering of the reduced item candidate set;

creating a plurality of transactions from the plurality of quantized item candidates by grouping neighboring quantized item candidates into transactions;

building a transaction database from the plurality of transactions;

removing at least one transaction from the plurality of transactions if the at least one transaction has a low co-occurrent frequency with the target head to obtain a remaining plurality of transactions;

creating a plurality of candidate auxiliary objects from the remaining plurality of transactions;

removing at least one candidate auxiliary object of the plurality of candidate auxiliary objects if the at least one candidate auxiliary object has a low motion correlation with the target head; and

creating a plurality of auxiliary objects from the remaining plurality of candidate auxiliary objects;

collaboratively tracking the target head using the plurality of auxiliary objects, wherein collaboratively tracking comprises;

tracking the target head and each of the plurality of auxiliary objects using a plurality of independent trackers; and

using a belief propagation technique to track the target head with the plurality of auxiliary objects, the belief propagation technique further comprising;

using a mean-shift technique to determine motion correlations between the target head and the plurality of auxiliary objects; and

applying the belief propagation technique to the target head and plurality of auxiliary objects using a star topology Markov random field,

wherein the target head comprises the hub of the star topology Markov random field;

verifying the consistency between the target head and plurality of auxiliary objects, wherein verifying comprises;

identifying a plurality of statistical outliers among the target head and plurality of auxiliary objects using a robust fusion technique;

excluding a result of an auxiliary object tracker if an auxiliary object of the plurality of auxiliary objects corresponding to the auxiliary object tracker is an outlier;

stopping identification of the plurality of auxiliary objects temporarily if the target head is an outlier; and

asserting a tracking failure if a majority of objects comprising the target head and the plurality of auxiliary objects are outliers.

50. A head tracking system configured to:

designate the target head in an image sequence;

track the target head using an elliptical tracker;

identify a plurality of auxiliary objects in the image sequence related to the target head, wherein identifying comprises;

splitting an image in the image sequence using a split-merge quad tree color segmentation algorithm to form a plurality of image regions;

creating a plurality of item candidates by merging adjacent image regions of the plurality of image regions with similar appearances into item candidates;

pruning item candidates inappropriate for tracking from the plurality of item candidates to obtain a reduced item candidate set;

creating a plurality of quantized item candidates by k-means clustering of the reduced item candidate set;

creating a plurality of transactions from the plurality of quantized item candidates by grouping neighboring quantized item candidates into transactions;

building a transaction database from the plurality of transactions;

removing at least one transaction from the plurality of transactions if the at least one transaction has a low co-occurrent frequency with the target head to obtain a remaining plurality of transactions;

creating a plurality of candidate auxiliary objects from the remaining plurality of transactions;

removing at least one candidate auxiliary object of the plurality of candidate auxiliary objects if the at least one candidate auxiliary object has a low motion correlation with the target head; and

creating a plurality of auxiliary objects from the remaining plurality of candidate auxiliary objects;

collaboratively track the target head using the plurality of auxiliary objects, wherein collaboratively tracking comprises;

tracking the target head and each of the plurality of auxiliary objects using a plurality of independent trackers; and

using a belief propagation techniques to track the target head with the plurality of auxiliary objects, the belief propagation technique further comprising;

using a mean-shift technique to determine motion correlations between the target head and the plurality of auxiliary objects; and

applying the belief propagation technique to the target head and plurality of auxiliary objects using a star topology Markov random field,

wherein the target head comprises the hub of the star topology Markov random field;

verify the consistency between the target head and plurality of auxiliary objects, wherein verifying comprises;

identifying a plurality of statistical outliers among the target head and plurality of auxiliary objects using a robust fusion technique;

excluding a result of an auxiliary object tracker if an auxiliary object of the plurality of auxiliary objects corresponding to the auxiliary object tracker is an outlier;

stopping identification of the plurality of auxiliary objects temporarily if the target head is an outlier; and

asserting a tracking failure if a majority of objects comprising the target head and the plurality of auxiliary objects are outliers.