Tracking using multilevel representations

• US 9,519,837 B2
• Filed: 07/03/2014
• Issued: 12/13/2016
• Est. Priority Date: 07/03/2014
• Status: Active Grant

• Current Assignee: Toyota Jidosha Kabushiki Kaisha (Toyota Motor Corporation), University of Technology at Sydney
• Original Assignee: Toyota Motor Engineering & Manufacturing North America Incorporated (Toyota Motor Corporation), University of Technology at Sydney
• Inventors: Mei, Xue, Wang, Chaohui, Hong, Zhibin, Prokhorov, Danil V., Tao, Dacheng
• Primary Examiner(s): COUSO, YON JUNG
• Application Number:
  US14/323,360
• Publication Number:
  US 20160004909A1
• Time in Patent Office: 894 Days
• Field of Search: 382/103
• US Class Current: 1/1
• CPC Class Codes: G06T 7/207 for motion estimation over a hierarchy of resolutions multi-resolution motion estimation or hierarchical motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/53
• CPC Class Codes: G06V 10/255 Detecting or recognising potential candidate objects based on visual cues, e.g. shapes
• CPC Class Codes: G06V 10/62 relating to a temporal dimension, e.g. time-based feature extraction; Pattern tracking

Abstract

A method of tracking a target object in frames of video data includes receiving a first tracking position associated with the target object in a first frame of a video sequence; identifying, for a second frame of the video sequence, a plurality of representation levels and at least one node for each representation level; determining, by a processor, a second tracking position in the second frame by estimating motion of the target object in the second frame between the first frame and the second frame; determining, at each representation level by the processor, a value for each node based on a conditional property of the node in the second frame; and adjusting, by the processor, the second tracking position based on the values determined for each of the nodes and interactions between at least some of the nodes at different representation levels.

Claims

1. A method of tracking a target object in frames of video data, comprising:

receiving a first tracking position associated with the target object in a first frame of a video sequence;

identifying, for a second frame of the video sequence, a plurality of representation levels, wherein the plurality of representation levels includes a bounding box level;

determining, by a processor, at least one node for each representation level, wherein the at least one node for the bounding box level is included in a second tracking position associated with the target object in the second frame determined by estimating motion of the target object between the first frame and the second frame;

determining, at each of the plurality of representation levels by the processor, a value for each node based on a conditional property of the node; and

adjusting, by the processor, the second tracking position based on the values determined for each of the nodes and interactions between at least the at least one node for the bounding box level and the at least one node for a different representation level.

2. The method of claim 1, wherein the plurality of representation levels is hierarchical.

3. The method of claim 1, wherein the plurality of representation levels comprises a pixel level, a superpixel level, and the bounding box level.

4. The method of claim 1, wherein the first and second tracking positions each include a bounding box, with the bounding box included in the second tracking position being associated with the at least one node for the bounding box level.

5. The method of claim 1, wherein the conditional property is a probability of a node label.

6. The method of claim 1, wherein, at the bounding box level, the conditional property is a probability of a node pose.

7. The method of claim 1, wherein the value for each node is determined as an energy potential value for the corresponding representation level in a Conditional Random Field model.

8. The method of claim 1, wherein the interactions between the at least one node for the bounding box level and the at least one node for a different representation level are determined based on pairwise energy potential values therebetween in a Conditional Random Field model.

9. The method of claim 1, wherein the plurality of representation levels comprises a pixel level, a superpixel level, and the bounding box level, and wherein the at least one node for each representation level is associated with one of a pixel, a superpixel, or a bounding box based on the corresponding representation level.

10. The method of claim 9, further comprising:

determining, for training, a positive sample set and a negative sample set based on the first frame and the first tracking position;

determining a set of pixel level random fields and a set of superpixel level random fields based on training results from the positive sample set and the negative sample set; and

determining a set of superpixels by grouping pixels in the second frame based on the set of superpixel level random fields.

11. The method of claim 10, further comprising:

determining if an occlusion exists within the adjusted second tracking position; and

updating the positive sample set and the negative sample set based on the determination that no occlusion exists within the adjusted second tracking position.

12. An apparatus for tracking a target object in frames of video data, comprising:

one or more processors; and

a memory for storing data and program instructions executed by the one or more processors, wherein the one or more processors are configured to executed instructions stored in the memory to;

receive a first tracking position associated with the target object in a first frame of a video sequence;

identify, for a second frame of the video sequence, a plurality of representation levels, wherein the plurality of representation levels includes a bounding box level;

determine at least one node for each representation level, wherein the at least one node for the bounding box level is included in a second tracking position associated with the target object in the second frame determined by estimating motion of the target object between the first frame and the second frame;

determine, at each of the plurality of representation levels, a value for each node based on a conditional property of the node; and

adjust the second tracking position based on the values determined for each of the nodes and interactions between at least the at least one node for the bounding box level and the at least one node for a different representation level.

13. The apparatus of claim 12, wherein the plurality of representation levels is hierarchical.

14. The apparatus of claim 12, wherein the plurality of representation levels comprises a pixel level, a superpixel level, and the bounding box level.

15. The apparatus of claim 12, wherein the first and second tracking positions each include a bounding box, with the bounding box included in the second tracking position being associated with the at least one node for the bounding box level.

16. The apparatus of claim 12, wherein, at the bounding box level, the conditional property is a probability of a node pose.

17. The apparatus of claim 12, wherein the value for each node is determined as an energy potential value for the corresponding representation level in a Conditional Random Field model.

18. The apparatus of claim 12, wherein the interactions between the at least one node for the bounding box level and the at least one node for a different representation level are determined based on pairwise energy potential values therebetween in a Conditional Random Field model.

19. The apparatus of claim 12, wherein the second tracking position is adjusted based on interactions between at least some of the nodes at the same representation level.

20. The apparatus of claim 12, wherein the instructions further comprise instructions stored in the memory to:

determine, for training, a positive sample set and a negative sample set based on the first frame and the first tracking position;

determine a set of pixel level random fields and a set of superpixel level random fields based on training results from the positive sample set and the negative sample set; and

determine a set of superpixels by grouping pixels in the second frame based on the set of superpixel level random fields.