Object tracking apparatus, object tracking method and recording medium

• US 7,064,776 B2
• Filed: 02/19/2002
• Issued: 06/20/2006
• Est. Priority Date: 05/09/2001
• Status: Active Grant

• Current Assignee: National Institute of Advanced Industrial Science and Technology (Government of Japan), Stanley Electric Company Limited
• Original Assignee: National Institute of Advanced Industrial Science and Technology (Government of Japan), Stanley Electric Company Limited
• Inventors: Ishiyama, Yutaka, Kubo, Fumio, Tomita, Fumiaki, Sumi, Yasushi
• Primary Examiner(s): Philippe, Gims
• Assistant Examiner(s): Rekstad, Erick
• Application Number:
  US10/078,882
• Publication Number:
  US 20020167594A1
• Time in Patent Office: 1,582 Days
• Field of Search: 348/169, 348/170, 348/171, 348/172, 348/143, 348/152, 348/208.14, 382/103, 382/165
• US Class Current: 348/169
• CPC Class Codes: G01S 3/7865 using correlation of the live video image with a stored image

Abstract

An object movement tracking apparatus and its method tracks an object with an unknown initial position and which object is continually moving, the apparatus for recovering a tracking process even when the tracking has failed and includes an observing device such as a video camera for observing a scene at a video frame rate. Time-sequentially acquired image data from the observing device are stored in an image memory device and the data processed in parallel. During data processing, an object recognition module selects a frame memory based on predetermined information. An estimated value of position/posture of the object are calculated from the image data and transmitted. A movement tracking module receives transmitted data from the object recognition module, replacing a current frame with a next frame and waits until image data in the next frame are inputted. When image data inputting is finished, the up dated replacement in position/posture of the object and its evaluated value are calculated by comparing image data and the object model. The system mode is changed and updated based on the obtained evaluated value.

Claims

1. An object movement tracking apparatus for detecting movements of an object from time-sequentially inputted images on real time basis comprising:

an observing device for observing a scene including said object and for acquiring inputted image data at a rate more than a predetermined rate;

an image memory device for temporally storing sequentially inputted image data from said observing device; and

a computer for processing said sequentially inputted image data so as to track the movement of said object, wherein;

said computer including memory for storing computer programs including;

an object recognition module for selecting a frame among image data stored in said image memory device frame by frame and detecting position/posture of said object in the selected frame; and

a movement tracking module for tracking the position/posture of said object in succeeding frames to the frame selected by said recognition module, wherein;

the movement tracking module and the object recognition module being arranged such that the movement tracking of said object is recovered by the operation of said object recognition module and said movement tracking module in parallel while the object recognition module and the movement tracking module communicate with each other when tracking movement of the object fails.

2. The object movement tracking apparatus according to claim 1 wherein said observing device comprises a single video camera equipped with an image sensor, said camera for providing said time-sequential images to said image memory device.

3. The object movement tracking apparatus according to claim 1 wherein said observing device comprises a stereo camera comprising a plurality of single cameras each equipped with image sensors, said camera for generating time-sequential stereo images which are inputted as said sequentially inputted image data into said image memory device.

4. The object movement tracking apparatus according to claim 1 wherein said observing device comprises a fast laser range finder for generating time-sequential distance images which are inputted from said finder into said image memory device as said sequentially inputted image data.

5. An object movement tracking method executed by an object movement tracking apparatus comprising:

an observing device for observing a scene including a object and for acquiring inputted image data at a rate more than a predetermined rate;

an image memory device for temporally storing sequentially inputted image data from said observing device; and

a computer for processing said sequentially inputted image data so as to track the movement of said object;

wherein;

said computer includes memory for storing computer programs including;

an object recognition module for selecting a frame among image data stored in said image memory device frame by frame and detecting position/posture of said object in the selected frame; and

a movement tracking module for tracking the position/posture of said object in succeeding frames to the frame selected by said recognition module, wherein;

said computer employs the tracking method based on the image data sequentially inputted from said observing device and temporally stored in said image memory device, the tracking method comprising the steps of;

acquiring an initial position of said object to be tracked;

detecting a movement of said object;

tracking the movement of said object on real time basis based on the image data sequentially inputted and temporally stored in said image memory device; and

recovering and resuming said tracking the movement of said object by operating said object recognition module and movement tracking module in parallel as they communicate with each other when tracking the movement of said object fails during said tracking step, wherein;

said recovering and resuming step includes a substep to run said movement tracking module.

6. The object movement tracking method according to claim 5 wherein the object to be tracked is a rigid body with known shape and size and which rigid body object exhibits movements to be tracked relative to said observing device.

7. An object movement tracking apparatus comprising:

an observing device for observing a scene including said object and for acquiring inputted image data at a rate more than a predetermined rate;

an image memory device for temporally storing sequentially inputted image data from said observing device; and

a computer for processing said sequentially inputted image data so as to track the movement of said object;

wherein;

said computer includes memory for storing computer programs including;

an object recognition module for selecting a frame among image data stored in said image memory device frame by frame and detecting position/posture of said object in the selected frame; and

a movement tracking module for tracking the position/posture of said object in succeeding frames to the frame selected by said recognition module, wherein;

said image memory device includes a recording medium which stores the following information and programs;

information on an initial position of said object to be tracked;

an object movement tracking program for detecting and tracking on real time basis based on image data inputted from said observing device and stored in said image memory device; and

a program for recovering and resuming tracking by operating said object recognition module and said movement tracking module in parallel while they communicate with each other when said tracking program fails in tracking the movement of said object during the tracking by said tracking program;

wherein

said recovering and resuming program has a subprogram to run said movement tracking module.

8. The Apparatus according to claim 7, wherein:

said recording medium includes a program stored therein for tracking a rigid body object with known shape and size and which rigid body object with the known shape and size exhibits movements to be tracked relative to said observing device, and which movements are stored in said recording medium.