Apparatus and method for object tracking

• US 10,210,421 B2
• Filed: 03/09/2017
• Issued: 02/19/2019
• Est. Priority Date: 05/19/2015
• Status: Active Grant

• Current Assignee: Toyota Jidosha Kabushiki Kaisha (Toyota Motor Corporation)
• Original Assignee: Toyota Motor Engineering & Manufacturing North America Incorporated (Toyota Motor Corporation), University of Technology at Sydney
• Inventors: Hong, Zhibin, Mei, Xue, Chen, Zhe, Wang, Chaohui, Prokhorov, Danil, Tao, Dacheng
• Primary Examiner(s): Woldemariam, Aklilu
• Application Number:
  US15/454,453
• Publication Number:
  US 20170185860A1
• Time in Patent Office: 712 Days
• Field of Search: 382103, 382107, 382131, 382158, 382159, 382190, 382201, 382224
• CPC Class Codes: G06F 18/00 Pattern recognition
• CPC Class Codes: G06F 18/24 Classification techniques
• CPC Class Codes: G06T 2207/20081 Training; Learning
• CPC Class Codes: G06T 3/40 Scaling of whole images or parts thereof, e.g. expanding or contracting
• CPC Class Codes: G06T 7/246 using feature-based methods, e.g. the tracking of corners or segments
• CPC Class Codes: G06T 7/248 involving reference images or patches
• CPC Class Codes: G06T 7/277 involving stochastic approaches, e.g. using Kalman filters
• CPC Class Codes: G06V 10/40 Extraction of image or video features

Abstract

An object tracking device includes a short-term processing portion and a long short-term processing portion that are implemented by circuitry and work in a collaborative manner to track an object. The short-term processing portion includes a filter that tracks the object based on short-term memory and spatiotemporal consistency. The long short-term processing portion performs key-point matching-tracking and estimation based on a key-point database in order to track the object. A controller determines an output of the object tracking device based on the processing conducted by the short-term and long short-term processing portions of the tracking device, respectively.

Claims

1. An object tracking device comprising:

circuitry configured to;

perform a short-term object tracking process to track an object from an input image;

perform a long-term object tracking process to track the object from the input image;

compare a result of the short-term object tracking process with a result of the long-term object tracking process to determine a consistency between the result of the short-term object tracking process and the result of the long-term object tracking process; and

generate a tracked object based on the consistency between the result of the short-term object tracking process and the result of the long-term object tracking process,

wherein

the short-term object tracking process includes a filtering process and tracks the object based on short-term memory and spatiotemporal consistency, and

the long-term object tracking process performs key-point matching-tracking and RANSAC estimation based on flexibility and shape generalizability of key-point based appearance models.

2. The object tracking device of claim 1, wherein the circuitry is configured to use the short-term memory to provide instant responses to the input image.

3. The object tracking device of claim 1, wherein the circuitry is configured to update the short-term memory based upon an output of the long-term object tracking process.

4. The object tracking device of claim 1, wherein the circuitry is configured to interact with the short-term memory to perform the long-term tracking process.

5. The object tracking device of claim 1, wherein

the circuitry further includes a long-term memory, and

the circuitry is configured to perform the long-term tracking process using the long-term memory.

12. The object tracking device of claim 1, wherein:

the short-term memory includes a filtering circuit that provides the filtering process, and

the circuitry is configured to reset the filtering circuit when an output of the short-term tracking process is highly inconsistent with the output of the long-term tracking process.

13. The object tracking device of claim 12, wherein the filtering process is a two-stage process and the short-term tracking process is highly inconsistent with the output of the long-term tracking process when the object is substantially occluded, leaves a field of view, or is rotated.

6. An object tracking method comprising:

performing a short-term object tracking process to track an object from an input image;

performing a long-term object tracking process to track the object from the input image;

comparing a result of the short-term object tracking process with a result of the long-term object tracking process to determine a consistency between the result of the short-term object tracking process and the result of the long-term object tracking process; and

generating a tracked object based on the consistency between the result of the short-term object tracking process and the result of the long-term object tracking process,

wherein

the short-term object tracking process includes a filtering process and tracks the object based on short-term memory and spatiotemporal consistency, and

the long-term object tracking process performs key-point matching-tracking and RANSAC estimation based on flexibility and shape generalizability of key-point based appearance models.

7. The object tracking method according to claim 6, wherein performing the short-term object tracking process includes using the short-term memory to provide instant responses to the input image.

8. The object tracking method according to claim 6, further comprising updating the short-term memory based upon an output of the long-term object tracking process.

9. The object tracking method of claim 6, further comprising interacting with the short-term memory to perform the long-term tracking process.

10. The object tracking method of claim 6, wherein performing the long-term object tracking process includes:

using a long-term memory.

14. The object tracking method of claim 6, wherein:

the short-term memory includes a filtering circuit that provides the filtering process, and

the circuitry is configured to reset the filtering circuit when an output of the short-term tracking process is highly inconsistent with the output of the long-term tracking process.

15. The object tracking method of claim 14, wherein the filtering process is a two-stage process and the short-term tracking process is highly inconsistent with the output of the long-term tracking process when the object is substantially occluded, leaves a field of view, or is rotated.

11. A non-transitory computer readable medium having stored thereon a program that when executed by a computer causes the computer to execute a method to track an object, the method comprising:

performing a short-term object tracking process to track an object from an input image;

performing a long-term object tracking process to track the object from the input image;

comparing a result of the short-term object tracking process with a result of the long-term object tracking process to determine a consistency between the result of the short-term object tracking process and the result of the long-term object tracking process; and

generating a tracked object based on the consistency between the result of the short-term object tracking process and the result of the long-term object tracking process,

wherein

the short-term object tracking process includes a filtering process and tracks the object based on short-term memory and spatiotemporal consistency, and

the long-term object tracking process performs key-point matching-tracking and RANSAC estimation based on flexibility and shape generalizability of key-point based appearance models.