Method and apparatus for autonomous object tracking

• US 7,697,725 B2
• Filed: 04/03/2006
• Issued: 04/13/2010
• Est. Priority Date: 04/03/2006
• Status: Active Grant

• Current Assignee: SRI International, Inc.
• Original Assignee: SRI International, Inc.
• Inventors: Burns, John Brian, Eriksen, Michael, Shapiro, Carl, Heller, Aaron, Vincent, Regis
• Primary Examiner(s): Lu, Tom Y
• Application Number:
  US11/397,988
• Publication Number:
  US 20070230742A1
• Time in Patent Office: 1,471 Days
• Field of Search: 382/103, 382/104, 382/107
• US Class Current: 382/107
• CPC Class Codes: G06T 7/207 for motion estimation over a hierarchy of resolutions multi-resolution motion estimation or hierarchical motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/53

Abstract

In one embodiment, the present invention is a method and apparatus for autonomous object tracking. In one embodiment, a method for tracking a moving object across at least a portion of a video signal made up of a plurality of image frames includes stabilizing the video signal by processing selected portions of selected image frames, detecting at least one movement in the stabilized video signal, and computing a location of the detected movement(s).

Claims

1. A method for tracking a moving object across at least a portion of a video signal comprising a plurality of image frames, the method comprising:

using a processor to perform steps comprising;

stabilizing the video signal by processing only selected portions of selected frames from among the plurality of image frames to produce a stabilized video signal, wherein said selected portions of said selected frames cover less than a total area of said selected frames and have highest texture values with said selected frames;

detecting at least one movement in the stabilized video signal; and

computing a location of the at least one movement.

2. The method of claim 1, wherein the stabilizing comprises:

computing a Gaussian image pyramid for each of the selected frames, the Gaussian image pyramid comprising a plurality of levels of images;

for each of the selected frames, generating an initial motion estimate between the selected frame and a reference frame, the initial motion estimate predicting a predicted movement that is attributable to an image capturing device from which the video signal is received;

generating a coarse translation estimate for the predicted movement that is attributable to the image capturing device, to produce a refined motion estimate;

performing image alignment processing on selected levels of the Gaussian image pyramid, to produce a further refined motion estimate; and

computing one or more image warp parameters in accordance with the further refined motion estimate.

3. The method of claim 2, further comprising:

filtering selected images in the Gaussian image pyramid prior to generating the initial motion estimate.

4. The method of claim 2, wherein the initial motion estimate is generated in accordance with one or more movements observed in one or more image frames of the video signal preceding the reference frame.

5. The method of claim 2, wherein the coarse translation estimate is generated in accordance with correlation at a coarsest one of the plurality of levels of the Gaussian image pyramid.

6. The method of claim 2, wherein the image alignment processing is performed in accordance with one or more iterations as defined by an iteration schedule.

7. The method of claim 6, wherein the iteration schedule defines a number of iterations of the image alignment processing to be performed on the selected frames and the selected levels of the Gaussian image pyramid.

8. The method of claim 7, wherein the number of iterations varies according to a given one of the selected frames or a given one of the selected levels of the Gaussian image pyramid.

9. The method of claim 2, wherein the image alignment processing is performed in accordance with the Lucas-Kanade processing technique.

10. The method of claim 2, wherein only selected portions of each of the images of the Gaussian image pyramid are processed in accordance with the image alignment processing.

11. The method of claim 10, wherein the selected portions of each of the images are selected by:

dividing each of the images into an array of zones; and

selecting within each zone in the array of zones a portion having a highest texture value.

12. The method of claim 11, wherein the a texture value of a given portion is computed in accordance with an absolute value of a Laplacian averaged over a region approximately equal in size to a size of the portion.

13. The method of claim 2, wherein the generating the initial motion estimate further comprises:

pre-processing the reference frame to facilitate the image alignment processing.

14. The method of claim 13, wherein the pre-processing comprises:

computing motion templates used to fill a matrix part of a linearized minimization for the reference frame; and

computing, from each of the motion templates, one or more integral images.

15. The method of claim 1, wherein the computing a location of the at least one movement comprises:

computing a location of the at least one movement in one or more of the plurality of image frames.

16. The method of claim 15, wherein the computing a location of the at least one movement further comprises:

computing an absolute position of the at least one movement in accordance with positional information relating to an image capturing device from which the video signal is received.

17. A computer readable storage medium containing an executable program for tracking a moving object across at least a portion of a video signal comprising a plurality of image frames, where the program performs the steps of:

stabilizing the video signal by processing only selected portions of selected frames from among the plurality of image frames to produce a stabilized video signal, wherein said selected portions of said selected frames cover less than a total area of said selected frames and have highest texture values with said selected frames;

detecting at least one movement in the stabilized video signal; and

computing a location of the at least one movement.

18. The computer readable storage medium of claim 17, wherein the stabilizing comprises:

computing a Gaussian image pyramid for each of the selected frames, the Gaussian image pyramid comprising a plurality of levels of images;

for each of the selected frames, generating an initial motion estimate between the selected frame and a reference frame, the initial motion estimate predicting a predicted movement that is attributable to an image capturing device from which the video signal is received;

generating a coarse translation estimate for the predicted movement that is attributable to the image capturing device, to produce a refined motion estimate;

performing image alignment processing on selected levels of the Gaussian image pyramid, to produce a further refined motion estimate; and

computing one or more image warp parameters in accordance with the further refined motion estimate.

19. The computer readable storage medium of claim 18, further comprising:

filtering selected images in the Gaussian image pyramid prior to generating the initial motion estimate.

20. The computer readable storage medium of claim 18, wherein the initial motion estimate is generated in accordance with one or more movements observed in one or more image frames of the video signal preceding the reference frame.

21. The computer readable storage medium of claim 18, wherein the coarse translation estimate is generated in accordance with correlation at a coarsest one of the plurality of levels of the Gaussian image pyramid.

22. The computer readable storage medium of claim 18, wherein the image alignment processing is performed in accordance with one or more iterations as defined by an iteration schedule.

23. The computer readable storage medium of claim 22, wherein the iteration schedule defines a number of iterations of the image alignment processing to be performed on the selected frames and the selected levels of the Gaussian image pyramid.

24. The computer readable storage medium of claim 23, wherein the number of iterations varies according to a given one of the selected frames or a given one of the selected levels of the Gaussian image pyramid.

25. The computer readable storage medium of claim 18, wherein the image alignment processing is performed in accordance with the Lucas-Kanade processing technique.

26. The computer readable storage medium of claim 18, wherein only selected portions of each of the images of the Gaussian image pyramid are processed in accordance with the image alignment processing.

27. The computer readable storage medium of claim 26, wherein the selected portions of each of the images are selected by:

dividing each of the images into an array of zones; and

selecting within each zone in the array of zones a portion having a highest texture value.

28. The computer readable storage medium of claim 27, wherein a texture value of a given portion is computed in accordance with an absolute value of a Laplacian averaged over a region approximately equal in size to a size of the portion.

29. The computer readable storage medium of claim 18, wherein the generating the initial motion estimate further comprises:

pre-processing the reference frame to facilitate the image alignment processing.

30. The computer readable storage medium of claim 29, wherein the pre-processing comprises:

computing motion templates used to fill a matrix part of a linearized minimization for the reference frame; and

computing, from each of the motion templates, one or more integral images.

31. The computer readable storage medium of claim 17, wherein the computing a location of the at least one movement comprises:

computing a location of the at least one movement in one or more of the plurality of image frames.

32. The computer readable storage medium of claim 31, wherein the computing a location of the at least one movement further comprises:

computing an absolute position of the at least one movement in accordance with positional information relating to an image capturing device from which the video signal is received.

33. An apparatus for tracking a moving object across at least a portion of a video signal comprising a plurality of image frames, comprising:

means for stabilizing the video signal by processing only selected portions of selected frames from among the plurality of image frames to produce a stabilized video signal, wherein said selected portions of said selected frames cover less than a total area of said selected frames and have highest texture values with said selected frames;

means for detecting at least one movement in the stabilized video signal; and

means for computing a location of the at least one movement.