Object tracking method and object tracking apparatus

• US 6,687,386 B1
• Filed: 06/13/2000
• Issued: 02/03/2004
• Est. Priority Date: 06/15/1999
• Status: Expired due to Fees

• Current Assignee: Hitachi Denshi Kabushiki Kaisha
• Original Assignee: Hitachi Denshi Kabushiki Kaisha
• Inventors: Ito, Wataru, Ueda, Hirotada
• Primary Examiner(s): Boudreau, Leo
• Assistant Examiner(s): Miller, Ryan J.
• Application Number:
  US09/592,996
• Time in Patent Office: 1,330 Days
• Field of Search: 382/103, 382/107, 382/209, 382/215, 382/216, 382/217, 382/218, 382/219, 340/937, 348/169
• US Class Current: 382/103
• CPC Class Codes: G06T 7/248 involving reference images or patches
• CPC Class Codes: G06V 10/24 Aligning, centring, orientation detection or correction of the image
• CPC Class Codes: G06V 20/54 of traffic, e.g. cars on the road, trains or boats

Abstract

A method and an apparatus for detecting an object in the image pickup field by template matching and tracking the detected object. An intruding object is detected from an image acquired by an image pickup device which takes the image of an image pickup field. A template image is formed from the detected object and is stored in a storing unit. A template matching is performed between a present input image of the image pickup field and the template image stored in the storing unit to detect the location of that part of the object which has a maximum degree of coincidence with the template image. An edge detection of the object is performed over a predetermined search area in the present input image which area includes the detected part of the object having a maximum degree of coincidence. Based on results of the edge detection, the detected part of the object is corrected and is determined as the present location of the object. A part of the present input image having the corrected location is a new template image with which the template image stored in the storing unit is updated. Instead of a single template image, a plurality of template images may be used for template matching to assure the tracking of the object with even higher accuracy and stability.

Claims

1. A tracking method for detecting an object in an image pickup field of an image pickup device, comprising the steps of:

forming from an image of said image pickup field a template image of a predetermined size including at least a part of said object;

storing said template image in a memory;

performing a template matching between an input image of said image pickup field from said image pickup device and said template image from said memory to detect a location of a part of said object in said input image which has a maximum degree of coincidence with said template image;

performing an edge density detection of said object over an expanded part of said input image which has a larger size than said predetermined size and includes said part of said object of said template image;

detecting a location of said object having a maximum edge density in said performing an edge density detection step;

compensating a location of said template image based on said location of said object having a maximum edge density so that a greater part of said template image contains said detected object; and

updating said template image, a location of which is compensated, into a new template image for a next input image.

2. A method according to claim 1, wherein said performing an edge detection step comprises substeps of:

extracting edge density components contained in said expanded part of said present input image;

displaying accumulated quantities of x-axis and y-axis edge components on x-axis and y-axis, respectively; and

detecting said portion having the maximum edge density from the accumulated quantities of x-axis and y-axis edge components on x-axis and y-axis.

3. A method according to claim 1, wherein a size of said expanded part of said input image is determined based on an apparent amount of movement of said object in said image pickup field.

4. A method according to claim 1, further comprising the step of:

controlling pan and tilt of said image pickup device so as to be directed toward said object based on a relation between said compensated location of said template image and a predetermined reference position in said image pickup field.

5. A tracking method for detecting an object in an image pickup field of an image pickup device, comprising the steps of:

forming from images of said image pickup field a predetermined number of template images, each of which has a predetermined size and includes at least a part of said object;

storing said template images in a memory;

performing a template matching between an input image of said image pickup field from said image pickup device and each of said template images from said memory to detect a location of a part of said object in said input images;

selecting a template image which has a maximum degree of coincidence in said performing a template matching step;

performing an edge density detection of said object over an expanded part of said input image which has a larger size than said predetermined size and includes said at least a part of said object at a location of said selected template image;

detecting a location of said object having a maximum edge density in said performing an edge density detection step;

compensating the location of said template image having a maximum degree of coincidence based on said location of said object having a maximum edge density so that a greater part of said template image contains said detected object; and

updating said template image, a location of which is compensated, as a new template image for a next input image.

6. A method according to claim 5, wherein said predetermined number of template images are formed from input images at different points in time and then stored.

7. A method according to claim 6, wherein the updated template image is one of said predetermined number of template images which was formed and stored from the input image at earliest point in time.

8. A method according to claim 5, wherein a picture template having the lowest level of coincidence degree among said predetermined number of template images is updated with said new template image when the degree of coincidence of at least one of said predetermined number of template images is not less than a predetermined value.

9. A method according to claim 5, further comprising the step of:

controlling said image pickup device to be directed toward said object based on a relation between said detected location of said object and a predetermined reference position in said image pickup field.

10. A method according to claim 5, wherein said performing an edge density detection step comprises substeps of:

extracting edge components contained in said expanded part of said input image;

displaying accumulated quantities of x-axis and y-axis edge components on x-axis and y-axis, respectively; and

detecting said part of said object having the maximum edge density from the accumulated quantities of x-axis and y-axis edge components on x-axis and y-axis.

11. A method according to claim 5, wherein the size of said expanded part of said input image is determined based on an apparent amount of movement of said object on said image pickup field.

12. A method according to claim 5, wherein said template matching is sequentially performed between the input image from said image pickup device and each of said predetermined number of template images, and when a location of a part of said object in said input image having a maximum degree of coincidence not less than said predetermined value is detected, the template matching using the remaining template images is terminated.

13. An object tracking apparatus for detecting and tracking an object in an image pickup field, comprising:

an image pickup device which takes an image in a range to be monitored;

a pan and tilt head with said image pickup device mounted thereon;

an image input interface for sequentially converting a video signal of an object acquired by said image pickup device in said monitor range into an input image signal;

an image processing unit connected to said image input interface which processes said input image signal;

a pan and tilt head control interface connected to said image processing unit and said pan and tilt head; and

a template image storing unit connected to said image input interface and said image processing unit,

wherein said image processing unit cuts out that part of an input image signal in advance to produce a template image of a predetermined size which includes at least a part of said object and store said template image in said template image storing unit, performs a template matching between said input image signal and said template image to detect a location of a part of said object in said input image having a maximum degree of coincidence with said template image, performs an edge detection of said object over an expanded part of said input image which has a larger size than said predetermined size and which includes said part of said object at location of said template image, detects a location of said object having a maximum edge density in said edge density detection, compensates the location of said template image based on said location of said object having a maximum edge density so that a greater part of aid template image contains said detected object, updates said template image, a location of which is compensated, into a new template image for a next input image signal, and controls said pan and tilt head based on said compensated location of said template image to direct said image pickup device toward said object.

14. A computer program product comprising:

a computer usable medium having computer readable program code means embodied in said medium for detecting and tracking an object in an image pickup field, said program code means comprising;

means for cutting out that part of an input image signal from an image pickup device which includes at least a part of said object to produce a template image of a predetermined size and storing said template image on a template image storing unit;

means for performing a template matching between an input image signal from said image pickup device and said template image from said memory to detect a location of a part of said object having a maximum degree of coincidence with said template image from said input image signal;

means for performing an edge density detection of said object over an expanded part of said input image which has a larger size than said predetermined size and includes said part of said object at a location of said template image;

means for detecting a location of said object having a maximum edge density in said means for performing an edge density detection;

means for compensating a location of said template image based on said location of said object having a maximum edge density so that a greater part of said template image contains said detected object;

means for updating said template image, the location of which is compensated, into a new template image for a next input image signal; and

means for controlling said pan and tilt head based on said compensated location of said object so that said image pickup device is directed toward said object.

15. A computer program product comprising:

a computer usable medium having computer readable program code means embodied in said medium for detecting and tracking an object in an image pickup field, said program code means comprising;

means for cutting out from input image signals each sequentially supplied from an image pickup device a plurality of template images each having a predetermined size and including at least a part of said object and storing said template images on a template image storing unit;

means for performing a template matching between an input image signal from said image pickup device and each of said template images from said memory to detect a location of a part of said object in said input image;

means for selecting a template image which has a maximum degree of coincidence in said template matching means;

means for performing an edge detection of said object over an expanded part of said input image which has a larger size than said predetermined size and includes said part of said object at a said location of said selected template image;

means for detecting a location of said object having a maximum edge density in said means for performing said edge density detection;

means for compensating a location of said template image having a maximum degree of coincidence based on said location of said object having a maximum edge density so that a greater part of said template image contains said detected object;

means for updating said template image, a location of which is compensated into, a new template image for a next input image; and

means for controlling said pan and tilt head based on the compensated location of said object so that said image pickup device is directed toward said object.

16. An object tracking apparatus for detecting and tracking an object in an image pickup field, comprising:

an image pickup device which takes an image in a range to be monitored;

a pan and tilt head with said image pickup device mounted thereon;

an image input interface for sequentially converting a video signal of an object acquired by said image pickup device in said monitor range into an input image signal;

an image processing unit connected to said image input interface which processes said input image signal;

a template image storing unit connected to said image input interface and said image processing unit; and

a pan and tilt head control interface connected to said image processing unit and said pan and tilt head,

wherein said image processing unit cuts out from sequentially input image signals a predetermined number of template images each including at least a part of said object, stores said template images in said template image storing unit, performs a template matching between an input image of said image pickup field from said image pickup device and each of said template images from said memory to detect a location of a part of said object in said input images, selects a template image which has a maximum degree of coincidence in said template matching step, performs an edge density detection of said object over an expanded part of said input image which has a larger size than said predetermined size and includes said part of said object at a location of said selected template image, detects a location of said object having a maximum edge density in said performing an edge density detection step, compensates the location of said template image having a maximum degree of coincidence based on said location of said object having a maximum edge density so that a greater part of said template image contains said detected object, updates said template image, a location of which is compensated, into a new template image for a next input image, and controls said pan and tilt head based on said compensated location of said template image so as to direct said image pickup device toward said object.