Method for simultaneous visual tracking of multiple bodies in a closed structured environment

• US 6,950,123 B2
• Filed: 03/22/2002
• Issued: 09/27/2005
• Est. Priority Date: 03/22/2002
• Status: Expired due to Term

• Current Assignee: Aurora Operations Incorporated (Aurora Innovation, Inc.)
• Original Assignee: Intel Corporation
• Inventors: Martins, Fernando C. M.
• Primary Examiner(s): PHILIPPE, GIMS S
• Application Number:
  US10/104,823
• Publication Number:
  US 20030179294A1
• Time in Patent Office: 1,285 Days
• Field of Search: 348/157, 348/159, 348/135, 348/155, 348/143, 348/699, 362/173, 362/164, 362/103, 362/236
• US Class Current: 348/157
• CPC Class Codes: G01S 3/781 Details
• CPC Class Codes: G01S 3/7864 T.V. type tracking systems
• CPC Class Codes: G06T 2207/30221 Sports video; Sports image
• CPC Class Codes: G06T 2207/30241 Trajectory
• CPC Class Codes: G06T 7/292 Multi-camera tracking
• CPC Class Codes: G06V 10/24 Aligning, centring, orientation detection or correction of the image

Abstract

Simultaneous tracking of multiple objects in a sequence of video frames captured by multiple cameras may be accomplished by extracting a foreground elements from a background in a frame, segmenting objects from the foreground surface, tracking objects within the frame, globally tracking positions of objects over time across multiple frames, fusing track data of objects obtained from multiple cameras to infer object positions, and resolving conflicts to estimate most likely object positions over time. Embodiments of the present invention improve substantially over existing trackers by including a technique for extraction of the region of interest that corresponds to a playing field, a technique for segmenting players from the field under varying illuminations, a template matching criteria that does not rely on specific shapes or color coherency of objects but on connected component properties, and techniques for reasoning about occlusions and consolidating tracking data from multiple cameras.

Claims

1. A method of simultaneous tracking of multiple objects in a sequence of video frames captured by multiple cameras comprising:

extracting a foreground image from a background image in a frame;

segmenting objects from the foreground image;

tracking objects within the frame by identifying the largest connected components inside each objects region of interest;

globally tracking positions of objects over time across multiple frames;

fusing track data of objects obtained from multiple cameras to infer object positions; and

resolving conflicts to estimate most likely object positions over time.

2. The method of claim 1, wherein segmenting the objects from the foreground image comprises segmenting the objects from the foreground image by examining motion of the objects in the sequence of video frames.

3. The method of claim 2, wherein examining motion of the objects comprises using a two tier hierarchical median filter.

4. The method of claim 3, wherein segmenting objects from the foreground image comprises segmenting objects from the foreground image under varying illuminations of the foreground image and the objects.

5. The method of claim 3, wherein segmenting objects comprises computing a difference between a current frame and a field model to produce a difference image, applying a field template to clip a region of interest from the difference image to produce a clipped difference image, and enhancing the clipped difference image to produce an object image.

6. The method of claim 1, wherein globally tracking positions of objects over time comprises using a sequential Monte Carlo technique.

7. The method of claim 1, wherein fusing track data of objects comprises disambiguating disparities due to occlusion of objects.

8. The method of claim 1, wherein the foreground image comprises a playing field for a sporting event and the objects comprise players participating in the sporting event.

9. The method of claim 1, further comprising initializing object tracking processing by creating a foreground image template, calibrating the cameras, and identifying visible objects on a first frame.

10. An article comprising:

a storage medium having a plurality of machine accessible instructions, wherein when the instructions are executed by a processor, the instructions provide for simultaneous tracking of multiple objects in a sequence of video frames captured by multiple cameras by

extracting a foreground image from a background image in a frame;

segmenting objects from the foreground image;

tracking objects within the frame by identifying the largest connected components inside each object'"'"'s region of interest;

globally tracking positions of objects over time across multiple frames;

fusing track data of objects obtained from multiple cameras to infer object positions; and

resolving conflicts to estimate most likely object positions over time.

11. The article of claim 10, wherein instructions for segmenting the objects from the foreground image comprise instructions for segmenting the objects from the foreground image by examining motion of the objects in the sequence of video frames.

12. The article of claim 11, wherein instructions for examining motion of the objects comprise instructions for implementing a two tier hierarchical median filter.

13. The article of claim 12, wherein instructions for segmenting objects from the foreground image comprise instructions for segmenting objects from the foreground image under varying illuminations of the foreground image and the objects.

14. The article of claim 12, wherein instructions for segmenting objects comprise instructions for computing a difference between a current frame and a field model to produce a difference image, applying a field template to clip a region of interest from the difference image to produce a clipped difference image, and enhancing the clipped difference image to produce an object image.

15. The article of claim 10, wherein instructions for globally tracking positions of objects over time comprise instructions implementing a sequential Monte Carlo technique.

16. The article of claim 10, wherein instructions for fusing track data of objects comprise instructions for disambiguating disparities due to occlusion of objects.

17. The article of claim 10, wherein the foreground image comprises a playing field for a sporting event and the objects comprise players participating in the sporting event.

18. The article of claim 10, further comprising instructions for initializing object tracking processing by creating a foreground image template, calibrating the cameras, and identifying visible objects on a first frame.

19. A system for simultaneous tracking of multiple objects in a three dimensional physical scene by analysis of a sequence of video frames comprising:

a plurality of video cameras to generate a plurality of sequences of video frames representing the scene over time; and

a processing system having at least one processor to simultaneously track the multiple objects, the processing system comprising

a field model generator component to extract a foreground image from a background image in a frame;

a segmentation component to segment objects from the foreground image;

a tracking component to track objects within the frame by identifying the largest connected components inside each object'"'"'s region of interest and to globally track positions of objects over time across multiple frames; and

a data fusion component to fuse track data of objects obtained from the plurality of video cameras to infer object positions, and to resolve conflicts to estimate most likely object positions over time.

20. The system of claim 19, wherein the segmentation component segments the objects from the foreground image by examining motion of the objects in the sequence of video frames.

21. The system of claim 20, wherein the segmentation component examines motion of the objects using a two tier hierarchical median filter.

22. The system of claim 21, wherein the segmentation component segments objects from the foreground image under varying illuminations of the foreground image and the objects.

23. The system of claim 21, wherein the segmentation component computes a difference between a current frame and a field model to produce a difference image, applies a field template to clip a region of interest from the difference image to produce a clipped difference image, and enhances the clipped difference image to produce an object image.

24. The system of claim 19, wherein the tracking component globally tracks positions of objects over time using a sequential Monte Carlo technique.

25. The system of claim 19, wherein the data fusion component disambiguates disparities due to occlusion of objects.

26. The system of claim 19, further comprising an initialization component to initialize object tracking processing by creating a foreground image template, calibrating the cameras, and identifying visible objects on a first frame.