Method, apparatus, and computer program product for object tracking

• US 8,818,024 B2
• Filed: 03/12/2009
• Issued: 08/26/2014
• Est. Priority Date: 03/12/2009
• Status: Active Grant

• Current Assignee: WSOU Investments, LLC (WSOU Holdings, LLC)
• Original Assignee: Nokia Corporation
• Inventors: Huynh, Duy-Nguyen Ta, Pulli, Kari Antero, Gelfand, Natasha, Chen, Wei-Chao
• Primary Examiner(s): Repko, Jason
• Assistant Examiner(s): FUJITA, KATRINA R
• Application Number:
  US12/402,921
• Publication Number:
  US 20100232643A1
• Time in Patent Office: 1,993 Days
• Field of Search: 382/103, 382/107, 348/143
• US Class Current: 382/103
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 7/248 involving reference images or patches
• CPC Class Codes: G06V 10/462 Salient features, e.g. scale invariant feature transforms [SIFT]

Abstract

A method for object tracking is provided. The method may include identifying a first interest point, receiving a video frame, and detecting, via a processor, a second interest point in the video frame using a scale space image pyramid. The method may further include matching the second interest point with the first interest point, and determining a motion estimation based on the matched interest points. Similar apparatuses and computer program products are also provided.

Claims

1. A method comprising:

identifying a first interest point;

receiving a first video frame;

defining a neighborhood for the first interest point;

detecting, via a processor, a second interest point in the neighborhood of the first video frame using a scale space image pyramid, the neighborhood being a portion of the scale space image pyramid;

matching the second interest point with the first interest point;

determining a motion estimation based on the matched interest points;

generating a feature descriptor based at least on the second interest point;

identifying a type of an object associated with the feature descriptor;

augmenting the first video frame to include a label identifying the type of the object, the label being positioned within the first video frame based on the matched interest points;

tracking the object in a subsequent video frame and within the neighborhood without recalculating the feature descriptor for the subsequent video frame; and

augmenting the subsequent video frame such that the position of the label is updated with respect to the first video frame.

2. The method according to claim 1, wherein detecting the second interest point in the first video frame using a scale space image pyramid includes detecting a second interest point in the first video frame using a scale space image pyramid, the scale space image pyramid comprising a Hessian-Affine, Harris-Affine, or approximate Hessian scale space.

3. The method according to claim 1, wherein matching the second interest point with the first interest point includes identifying a relationship between dominant curvatures of the first interest point and dominant curvatures of the second interest point, the curvatures being utilized as signatures of the respective interest points.

4. The method according to claim 1, wherein matching the second interest point with the first interest point includes comparing the interest points within the scale space using normalized cross correlation.

5. The method according to claim 1 further comprising computing a feature descriptor based on at least the second interest point, the feature descriptor being a speeded up robust feature, a scale-invariant feature transform feature, or a gradient location and orientation histogram feature.

6. The method according to claim 1, wherein identifying the type of the object comprises searching a database for another object having a similar feature descriptor.

7. The method of claim 1, further comprising:

prior to tracking the object in the subsequent video frame, transforming the neighborhood based on a relative change in motion between the first video frame and the subsequent video frame.

8. The method of claim 1, further comprising:

determining based on a relative change in motion between the subsequent video frame and another subsequent video frame to recalculate the feature descriptor in the another subsequent video frame.

9. The method of claim 1, further comprising:

transforming four corners of the subsequent video frame such that the overlap between the subsequent video frame and the first video frame drops to a certain percentage.

10. An apparatus comprising a processor and a memory storing executable instructions that, in response to execution by the processor, cause the apparatus to at least:

identify a first interest point;

receive a first video frame;

define a neighborhood for the first interest point;

detect a second interest point in the neighborhood of the first video frame using a scale space image pyramid, the neighborhood being a portion of the scale space image pyramid;

match the second interest point with the first interest point;

determine a motion estimation based on the matched interest points;

generate a feature descriptor based at least on the second interest point;

identify a type of an object associated with the feature descriptor;

augment the first video frame to include a label identifying the type of the object, the label being positioned within the first video frame based on the matched interest points;

track the object in a subsequent video frame and without recalculating the feature descriptor for the subsequent video frame; and

augment the subsequent video frame such that the position of the label is updated with respect to the first video frame.

11. The apparatus according to claim 10, wherein the executable instructions that cause the apparatus to detect the second interest point in the first video frame using a scale space image pyramid include causing the apparatus to detect a second interest point in the first video frame using a scale space image pyramid, the scale space image pyramid comprising a Hessian-Affine, HarrisAffine, or approximate Hessian scale space.

12. The apparatus according to claim 10, wherein the executable instructions that cause the apparatus to match the second interest point with the first interest point include causing the apparatus to identify a relationship between dominant curvatures of the first interest point and dominant curvatures of the second interest point, the curvatures being utilized as signatures of the respective interest points.

13. The apparatus according to claim 10, wherein the executable instructions that cause the apparatus to match the second interest point with the first interest point include causing the apparatus to compare the interest points within the scale space using normalized cross correlation.

14. The apparatus according to claim 10, wherein the executable instructions further cause the apparatus to compute a feature descriptor based on at least the second interest point, the feature descriptor being a speeded up robust feature, a scale-invariant feature transform feature, or a gradient location and orientation histogram feature.

15. The apparatus according to claim 10, wherein the apparatus comprises a mobile terminal.

16. A computer program product comprising at least one non-transitory computer-readable storage medium having executable computer-readable program code instructions stored therein, the computer-readable program code instructions configured to:

identify a first interest point;

receive a first video frame;

define a neighborhood for the first interest point;

detect a second interest point in the neighborhood of the first video frame using a scale space image pyramid, the neighborhood being a portion of the scale space image pyramid;

match the second interest point with the first interest point;

determine a motion estimation based on the matched interest points;

generate a feature descriptor based at least on the second interest point;

identify a type of an object associated with the feature descriptor;

augment the first video frame to include a label identifying the type of the object, the label being positioned within the first video frame based on the matched interest points;

track the object in a subsequent video frame and within the neighborhood without recalculating the feature descriptor for the subsequent video frame; and

augment the subsequent video frame such that the position of the label is updated with respect to the first video frame.