IMAGE RECOGNITION APPARATUS, CONTROL METHOD, AND PROGRAM OF THE SAME

• US 20140184812A1
• Filed: 12/26/2013
• Published: 07/03/2014
• Est. Priority Date: 12/27/2012
• Status: Active Grant

• Current Assignee: Canon Kabushiki Kaisha (Canon Inc.)
• Original Assignee: Canon Kabushiki Kaisha (Canon Inc.)
• Inventors: Tsuji, Ryosuke
• Granted Patent: US 9,524,423 B2
• Time in Patent Office: Days
• US Class Current: 348/169
• CPC Class Codes: G06V 40/172 Classification, e.g. identification
• CPC Class Codes: G06V 40/173 face re-identification, e.g. recognising unknown faces across different face tracks
• CPC Class Codes: H04N 23/611 where the recognised objects include parts of the human body

Abstract

An image recognition apparatus is arranged to extract feature information of an object area detected from among images, compare with feature information of previously-registered objects, thereby calculating a degree of similarity, when the calculated degree of similarity is equal to or greater than a predetermined threshold value, determine the object included in the object area as a registered object and set a recognition confirmed state that is successively maintained between images, and when a reliability of the object tracking is low and the degree of similarity is smaller than the predetermined threshold value, set the recognition confirmed state which is successively maintained into a recognition unconfirmed state where the object included in the object area is not determined as a registered object.

Claims

1. An image recognition apparatus for recognizing a specific object from among images supplied in time series, comprising:

a storage unit configured to store feature information regarding the specific object;

an object detection unit configured to detect an object area from the supplied image;

an object recognition unit configured to extract feature information from the image of the object area detected by the object detection unit, compare the extracted feature information with the feature information of the specific object stored in the storage unit, thereby calculating a degree of similarity, and perform an object recognition on the basis of the calculated degree of similarity;

an object tracking unit configured to track the object included in the object area detected by the object detection unit, in the images supplied in time series; and

a reliability determination unit configured to determine a reliability of the object tracking which is performed by the object tracking unit,

wherein when the degree of similarity calculated by the object recognition unit is equal to or greater than a predetermined threshold value, the object included in the object area is determined as being the specific object stored in the storage unit so that a recognition confirmed state is set, and the recognition confirmed state is successively maintained between the images supplied in time series by means of the object tracking by the object tracking unit, and

when the reliability of the object tracking is low and the degree of similarity calculated by the object recognition unit is smaller than the predetermined threshold value, the recognition confirmed state which is successively maintained is set into a recognition unconfirmed state where the object included in the object area is not determined a being the specific object.

2. An apparatus according to claim 1, wherein when the reliability of the object tracking is high, the object recognition unit does not execute the object recognition for the object in which the recognition confirmed state is successively maintained by the object tracking.

3. An apparatus according to claim 1, wherein when the plurality of object areas detected by the object detection unit are densely distributed, the reliability determination unit determines that the reliability of the object tracking is low, and when the plurality of object areas detected by the object detection unit are distantly distributed, the reliability determination unit determines that the reliability of the object tracking is high.

4. An apparatus according to claim 1, wherein:

for the object in which the recognition confirmed state is not successively maintained by the object tracking, the object recognition unit calculates a degree of similarity to each of the plurality of objects stored in the storage unit; and

for the object in which the recognition confirmed state is successively maintained by the object tracking, the object recognition unit calculates a degree of similarity to the object set into the recognition confirmed state among the objects stored in the storage unit.

5. An image pickup apparatus comprising:

an image pickup unit configured to supply picked-up images in time series;

the image recognition apparatus according to claim 1; and

a control unit configured to control an image pickup condition in the image pickup unit in accordance with information of the object which is obtained by the image recognition apparatus.

6. A display apparatus comprising:

a display unit configured to display images supplied in time series;

the image recognition apparatus according to claim 1; and

a control unit configured to control a display condition of the image in the display unit in accordance with information of the object which is obtained by the image recognition apparatus.

7. A control method of an image recognition apparatus for recognizing a specific object, feature information of which is stored in a storage unit, from among images supplied in time series, comprising:

an object detection step of detecting an object area from the supplied image;

an object recognition step of extracting feature information from the image of the object area detected in the object detection step, comparing the extracted feature information with the feature information of the specific object stored in the storage unit, thereby calculating a degree of similarity, and performing an object recognition on the basis of the calculated degree of similarity;

an object tracking step of tracking the object included in the object area detected in the object detection step in the images supplied in time series; and

a reliability determination step of determining a reliability of the object tracking which is performed in the object tracking step,

wherein when the degree of similarity calculated in the object recognition step is equal to or greater than a predetermined threshold value, the object included in the object area is determined as being the specific object stored in the storage unit so that a recognition confirmed state is set, and recognition confirmed state is successively maintained between the images supplied in time series by means of the object tracking in the object tracking step, and

when the reliability of the object tracking is low and the degree of similarity calculated in the object recognition step is smaller than the predetermined threshold value, the recognition confirmed state which is successively maintained is set into a recognition unconfirmed state where the object included in the object area is not determined as being the specific object.

8. A non-transitory computer-readable storage medium storing a program for causing a computer to execute a control method of an image recognition apparatus for recognizing a specific object, feature information of which is stored in a storage unit, from among images supplied in time series,

the control method comprising:

an object detection step of detecting an object area from the supplied image;

an object recognition step of extracting feature information from the image of the object area detected in the object detection step, comparing the extracted feature information with the feature information of the specific object stored in the storage unit, thereby calculating a degree of similarity, and performing an object recognition on the basis of the calculated degree of similarity;

an object tracking step of tracking the object included in the object area detected in the object detection step in the images supplied in time series; and

a reliability determination step of determining a reliability of the object tracking which is performed in the object tracking step,

wherein when the degree of similarity calculated in the object recognition step is equal to or greater than a predetermined threshold value, the object included in the object area is determined as being the specific object stored in the storage unit so that a recognition confirmed state is set, and the recognition confirmed state is successively maintained between the images supplied in time series by means of the object tracking in the object tracking step, and

when the reliability of the object tracking is low and the degree of similarity calculated in the object recognition step is smaller than the predetermined threshold value, the recognition confirmed state which is successively maintained is set into a recognition unconfirmed state where the object included in the object area is not determined as being the specific object.