PERSON TRACKING METHOD, PERSON TRACKING APPARATUS, AND PERSON TRACKING PROGRAM STORAGE MEDIUM

• US 20100296701A1
• Filed: 04/26/2010
• Published: 11/25/2010
• Est. Priority Date: 05/21/2009
• Status: Active Grant

• Current Assignee: Fujifilm Corporation (Fujifilm Holdings Corporation)
• Original Assignee: Fujifilm Corporation (Fujifilm Holdings Corporation)
• Inventors: HU, Xuebin
• Granted Patent: US 8,369,574 B2
• Time in Patent Office: Days
• US Class Current: 382/103
• CPC Class Codes: G06F 18/254 of classification results, e.g. of results related to same input data
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/20016 Hierarchical, coarse-to-fine, multiscale or multiresolution image processing; Pyramid transform
• CPC Class Codes: G06T 2207/30201 Face
• CPC Class Codes: G06T 2207/30241 Trajectory
• CPC Class Codes: G06T 7/246 using feature-based methods, e.g. the tracking of corners or segments
• CPC Class Codes: G06V 10/245 by locating a pattern; Special marks for positioning
• CPC Class Codes: G06V 10/809 of classification results, e.g. where the classifiers operate on the same input data
• CPC Class Codes: G06V 30/2504 Coarse or fine approaches, e.g. resolution of ambiguities or multiscale approaches
• CPC Class Codes: G06V 40/173 face re-identification, e.g. recognising unknown faces across different face tracks
• CPC Class Codes: G06V 40/20 Movements or behaviour, e.g. gesture recognition recognition of facial expressions G06V40/16

Abstract

A person tracking method capable of tracking movements of a person captured by a camera through lighter processing in comparison with tracking processing that employs a Kalman filter or the like is provided. The method includes: detecting a head on each frame image; calculating a feature quantity that features a person whose head is detected on the frame images; calculating a relevance ratio that represents a degree of agreement between a feature quantity on a past frame image and a feature quantity on a current frame image, which belong to each person whose head is detected on the current frame image; and determining that, a head, which is a basis for calculation of a relevance ratio that represents a degree of agreement being a first threshold as well as being a maximum degree of agreement, is a head of the same person as the person having the head.

Claims

1. A person tracking method comprising:

a head detecting step of detecting, based on image data that is generated as a result of taking a moving image of a measured space by a camera and represents the moving image generated at a predetermined frame rate, a head on each of a plurality of measured frame images among a plurality of frame images forming the moving image, for each of the measured frame images;

a feature-quantity calculation step of calculating, based on the image data, a feature quantity that features a person whose head is detected on each of the measured frame images, for each of the measured frame images and each head;

a relevance ratio calculation step of calculating a relevance ratio that represents a degree of agreement between a feature quantity that is on a past measured frame image and belongs to each person whose head is detected on the past measured frame image and a feature quantity that is on a current measured frame image and belongs to each person whose head is detected on the current measured frame image; and

a same-person determination step of determining that, among heads detected on the current measured frame image, a head, which is a basis for calculation of a relevance ratio that represents a degree of agreement being equal to or above a first threshold as well as being a maximum degree of agreement, among relevance ratios each between a feature quantity that is on a past measured frame image and belongs to a person having a first head among heads detected on the past measured frame image and each of feature quantities that are on the current measured frame image and belong to respective persons having the respective heads detected on the current measured frame image, is a head of the same person as the person having the first head.

2. The person tracking method according to claim 1, wherein when heads determined as belonging to a same first person over a sequence of first measured frame images are detected, a head belonging to the first person is missing on subsequent one measured frame image or a plurality of subsequent sequential measured frame images whose quantity is equal to or fewer than a second threshold, and then heads determined as belonging to a same second person over a sequence of second measured frame images are detected,

the feature-quantity calculation step calculates a first average feature quantity of the first person on the plurality of first measured frame images and a second average feature quantity of the second person on the plurality of second measured frame images,

the relevance-ratio calculation step calculates a relevance ratio between the first average feature quantity and the second average feature quantity, and

when the relevance ratio between the first average feature quantity and the second average feature quantity is a relevance ratio that represents a degree of agreement equal to or above a third threshold, the same-person determination step determines that the first person and the second person are identical. 20

3. The person tracking method according to claim 1, further comprising a fragment deleting step in which when a head of a third person is detected only on one measured frame image or a plurality of sequential measured frame images whose quantity is equal to or fewer than a fourth threshold, and no head of the third person corresponding to the detected head is detected on frame images before and after the one measured frame image or the plurality of sequential measured frame images, the head of the third person is deleted from the detected heads.

4. The person tracking method according to claim 1, wherein the feature-quantity calculation step is a step of calculating, as one of the feature quantities, a histogram of pixel values of a head detected on a measured frame image.

5. The person tracking method according to claim 1, wherein the feature-quantity calculation step is a step of calculating, as one of the feature quantities, a histogram of pixel values of a body belonging to a person whose head is detected on a measured frame image.

6. The person tracking method according to claim 1, wherein the feature-quantity calculation step is a step of calculating, as one of the feature quantities, a function value based on movements of heads belonging to a same person on a plurality of past measured frame images, where an appearance position that is on a current measured frame image and belongs to a head detected on the current measured frame image is a variable.

7. A person tracking apparatus comprising:

a head detecting section that detects, based on image data that is generated as a result of taking a moving image of a measured space by a camera and represents the moving image generated at a predetermined frame rate, a head on each of a plurality of measured frame images among a plurality of frame images forming the moving image, for each of the measured frame images;

a feature-quantity calculation section that calculates, based on the image data, a feature quantity that features a person whose head is detected on each of the measured frame images, for each of the measured frame images and each head;

a relevance ratio calculation section that calculates a relevance ratio that represents a degree of agreement between a feature quantity that is on a past measured frame image and belongs to each person whose head is detected on the past measured frame image and a feature quantity that is on a current measured frame image and belongs to each person whose head is detected on the current measured frame image; and

a same-person determination section that determines that, among heads detected on the current measured frame image, a head, which is a basis for calculation of a relevance ratio that represents a degree of agreement being equal to or above a first threshold as well as being a maximum degree of agreement, among relevance ratios each between a feature quantity that is on a past measured frame image and belongs to a person having a first head among heads detected on the past measured frame image and each of feature quantities that are on the current measured frame image and belong to respective persons having the respective heads detected on the current measured frame image, is a head of the same person as the person having the first head.

8. The person tracking apparatus according to claim 7, wherein when heads determined as belonging to a same first person over a sequence of first measured frame images are detected, a head belonging to the first person is missing on subsequent one measured frame image or a plurality of subsequent sequential measured frame images whose quantity is equal to or fewer than a second threshold, and then heads determined as belonging to a same second person over a sequence of second measured frame images are detected,

the feature-quantity calculation section calculates a first average feature quantity of the first person on the plurality of first measured frame images and a second average feature quantity of the second person on the plurality of second measured frame images,

the relevance-ratio calculation section calculates a relevance ratio between the first average feature quantity and the second average feature quantity, and

when the relevance ratio between the first average feature quantity and the second average feature quantity is a relevance ratio that represents a degree of agreement equal to or above a third threshold, the same-person determination section determines that the first person and the second person are identical.

9. The person tracking apparatus according to claim 7, further comprising a fragment deleting section that deletes, when a head of a third person is detected only on one measured frame image or a plurality of sequential measured frame images whose quantity is equal to or fewer than a fourth threshold, and no head of the third person corresponding to the detected head is detected on frame images before and after the one measured frame image or the plurality of sequential measured frame images, the head of the third person is deleted from the detected heads.

10. The person tracking apparatus according to claim 7, wherein the feature-quantity calculation section calculates, as one of the feature quantities, a histogram of pixel values of a head detected on a measured frame image.

11. The person tracking apparatus according to claim 7, wherein the feature-quantity calculation section calculates, as one of the feature quantities, a histogram of pixel values of a body belonging to a person whose head is detected on a measured frame image.

12. The person tracking apparatus according to claim 7, wherein the feature-quantity section calculates, as one of the feature quantities, a function value based on movements of heads belonging to a same person on a plurality of past measured frame images, where an appearance position that is on a current measured frame image and belongs to a head detected on the current measured frame image is a variable.

13. A person tracking program storage medium that stores a person tracking program that causes, when executed in an operation apparatus running a program, the operation apparatus to operate as a person tracking apparatus comprising:

a head detecting section that detects, based on image data that is generated as a result of taking a moving image of a measured space by a camera and represents the moving image generated at a predetermined frame rate, a head on each of a plurality of measured frame images among a plurality of frame images forming the moving image, for each of the measured frame images;

a feature-quantity calculation section that calculates, based on the image data, a feature quantity that features a person whose head is detected on each of the measured frame images, for each of the measured frame images and each head;

a relevance ratio calculation section that calculates a relevance ratio that represents a degree of agreement between a feature quantity that is on a past measured frame image and belongs to each person whose head is detected on the past measured frame image and a feature quantity that is on a current measured frame image and belongs to each person whose head is detected on the current measured frame image; and

a same-person determination section that determines that, among heads detected on the current measured frame image, a head, which is a basis for calculation of a relevance ratio that represents a degree of agreement being equal to or above a first threshold as well as being a maximum degree of agreement, among relevance ratios each between a feature quantity that is on a past measured frame image and belongs to a person having a first head among heads detected on the past measured frame image and each of feature quantities that are on the current measured frame image and belong to respective persons having the respective heads detected on the current measured frame image, is a head of the same person as the person having the first head.

14. The person tracking program storage medium according to claim 13, wherein when heads determined as belonging to a same first person over a sequence of first measured frame images are detected, a head belonging to the first person is missing on subsequent one measured frame image or a plurality of subsequent sequential measured frame images whose quantity is equal to or fewer than a second threshold, and then heads determined as belonging to a same second person over a sequence of second measured frame images are detected,

the feature-quantity calculation section calculates a first average feature quantity of the first person on the plurality of first measured frame images and a second average feature quantity of the second person on the plurality of second measured frame images,

the relevance-ratio calculation section calculates a relevance ratio between the first average feature quantity and the second average feature quantity, and

when the relevance ratio between the first average feature quantity and the second average feature quantity is a relevance ratio that represents a degree of agreement equal to or above a third threshold, the same-person determination section determines that the first person and the second person are identical.

15. The person tracking program storage medium according to claim 13, further comprising a fragment deleting section that deletes, when a head of a third person is detected only on one measured frame image or a plurality of sequential measured frame images whose quantity is equal to or fewer than a fourth threshold, and no head of the third person corresponding to the detected head is detected on frame images before and after the one measured frame image or the plurality of sequential measured frame images, the head of the third person is deleted from the detected heads.

16. The person tracking program storage medium according to claim 13, wherein the feature-quantity calculation section calculates, as one of the feature quantities, a histogram of pixel values of a head detected on a measured frame image.

17. The person tracking program storage medium according to claim 13, wherein the feature-quantity calculation section calculates, as one of the feature quantities, a histogram of pixel values of a body belonging to a person whose head is detected on a measured frame image.

18. The person tracking program storage medium according to claim 13, wherein the feature-quantity section calculates, as one of the feature quantities, a function value based on movements of heads belonging to a same person on a plurality of past measured frame images, where an appearance position that is on a current measured frame image and belongs to a head detected on the current measured frame image is a variable.