Robust feature fusion for multi-view object tracking

• US 8,989,442 B2
• Filed: 04/12/2013
• Issued: 03/24/2015
• Est. Priority Date: 04/12/2013
• Status: Expired due to Fees

• Current Assignee: Toyota Jidosha Kabushiki Kaisha (Toyota Motor Corporation)
• Original Assignee: Toyota Motor Engineering & Manufacturing North America Incorporated (Toyota Motor Corporation)
• Inventors: Mei, Xue, Prokhorov, Danil V.
• Primary Examiner(s): Lu, Tom Y
• Application Number:
  US13/861,632
• Publication Number:
  US 20140307917A1
• Time in Patent Office: 711 Days
• Field of Search: 382/103
• US Class Current: 382/103
• CPC Class Codes: G06V 20/52 Surveillance or monitoring of activities, e.g. for recognising suspicious objects recognising microscopic objects G06V20/69

Abstract

Multi-Task Multi-View Tracking (MTMVT) is used to visually identify and track an object. The MTMVT employs visual cues such as color, edge, and texture as complementary features to intensity in the target appearance representation, and combines a multi-view representation with a robust multi-task learning to solve feature fusion tracking problems. To reduce computational demands, feature matrices are sparsely represented in a single matrix and then decomposed into a pair of matrices to improve robustness to outliers. Views and particles are further combined based on interdependency and commonality single computational task. Probabilities are computed for each particle across all features and the particle with the greatest probability is selected as the target tracking result.

Claims

1. A method for tracking an object using a sensor network comprising:

selecting one or multiple reference frames from a plurality of data frames captured from the sensor network;

identifying the object to track and obtaining a plurality of tracking target templates from the reference frames;

extracting a set of multiple views from each of the tracking target templates;

sampling a plurality of image patches proximate the location of the object in subsequent frames relative to the reference frame;

extracting the set of multiple views from each image patch;

solving a minimization problem in a robust multi-view multi-task framework for each image patch to calculate a probability for each of the multiple views;

calculating an entropy of each of the multiple views using the probabilities of all the image patches; and

determining a tracking result using the image patch with the highest probability using a multi-view weighting for the purpose of tracking and identifying the object.

2. The method of claim 1, further comprising:

calculating a minimum reconstruction error for each of multiple views for each image patch as a single task in a multi-task learning process.

3. The method of claim 2, further comprising:

identifying commonalities between a plurality of interdependent tasks and identifying an outlier task in the multi-task learning process.

4. The method of claim 1, further comprising:

the sensor network having a plurality of sensors; and

selecting a set of data frames from each of the sensors.

5. The method of claim 4, further comprising:

calculating a view weight using the entropy of all of the image patch probabilities for each view.

6. The method of claim 5, further comprising:

calculating an object tracking probability by dynamically computing the view weights for each set of data frames.

7. The method of claim 1, further comprising:

processing the data frames with a computer processing unit (CPU) operable to perform extracting, sampling, calculating, identifying, and minimization functions; and

storing a CPU data output in a memory unit.

8. A method for obtaining a tracking target result for an object using a computer processing unit (CPU), a memory unit, and a sensor network comprising:

capturing a plurality of data frames from the sensor network and storing the data frames in the memory unit;

selecting a reference frame with the CPU from the plurality of data frames and identifying the object;

obtaining a plurality of particles from the reference frame proximate a location of the object;

sparsely representing a set of multiple features from each of the particles in a representation matrix using a multi-task formulation;

decomposing the representation matrix into a pair of collaborative weight matrices and minimizing a reconstruction error with penalty terms;

computing a probability that the particle is the tracking target result; and

identifying the tracking target result as the particle with the highest probability.

9. The method of claim 8, further comprising:

sampling a plurality of image patches proximate the location of the object in subsequent frames relative to the reference frame; and

identifying the image patch with the smallest reconstruction error relative to the tracking target result for the purpose of tracking and identifying the object.

10. The method of claim 8, further comprising:

the sensor network having a plurality of sensors; and

selecting a set of data frames from each of the sensors.

11. The method of claim 10, further comprising:

calculating a view weight using an entropy of all of the image patch probabilities for the corresponding view.

12. The method of claim 11, further comprising:

calculating an object tracking probability by dynamically computing the view weights for each set of data frames.