IMAGE RECOGNITION SYSTEM, IMAGE RECOGNITION APPARATUS, IMAGE RECOGNITION METHOD, AND COMPUTER PROGRAM

• US 20150363636A1
• Filed: 06/08/2015
• Published: 12/17/2015
• Est. Priority Date: 06/12/2014
• Status: Active Grant

• Current Assignee: Canon Kabushiki Kaisha (Canon Inc.)
• Original Assignee: Canon Kabushiki Kaisha (Canon Inc.)
• Inventors: Tate, Shunta, Kuroki, Tsuyoshi, Kawai, Tomoaki
• Granted Patent: US 9,743,014 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06V 40/173 face re-identification, e.g. recognising unknown faces across different face tracks
• CPC Class Codes: H04N 23/90 Arrangement of cameras or camera modules, e.g. multiple cameras in TV studios or sports stadiums

Abstract

A relative direction relationship is acquired between first and second input area images of a particular person taken from different directions. The particular person is identified by comparing a feature of the first input area image with a feature of a first one of registered area images of the particular person or another person taken from at least three directions, comparing a feature of the second input area image with a feature of a second registered area image of the same person as the person of the first registered area image, and determining whether the person in the first and second input area images is the same person in the first and second registered area images. The first and second registered area images are selected such that the relation between the first and second registered area images is similar to the relation between the first and second input area images.

Claims

1. An image recognition apparatus comprising:

an inter-image relative direction acquisition unit configured to acquire a relative direction relationship between a first input area image and a second input area image obtained by taking images of a particular person from different directions; and

a person identification unit configured to identify the particular person by comparing a feature of the first input area image with a feature of a first registered area image included in a set of registered area images including registered area images obtained by taking images of the particular person or another person from at least three directions, comparing a feature of the second input area image with a feature of a second registered area image of the same person as the person of the first registered area image included in the set of registered area images, and determining whether the person in the first and second input area images is the same as the person in the first and second registered area images,

wherein the first registered area image and the second registered area image are selected such that the relative direction relationship between the first registered area image and the second registered area image is equal or similar to the relative direction relationship between the first input area image and the second input area image.

2. The image recognition apparatus according to claim 1, further comprising:

a person tracking unit configured to track the particular person in an input image and estimate the location coordinate information of the particular person;

a person location estimation unit configured to estimate the location of the particular person in the input image;

a person area extraction unit configured to be given a first input image and a second input image obtained by taking images of the particular person from different directions, and extract the first input area image and the second input area image respectively from the first input image and the second input image based on the location of the person estimated by the person location estimation unit;

a feature extraction unit configured to extract a feature from the first input area image and a feature from the second input area image; and

a person area feature storage unit configured to store the feature of each registered area image in the set of registered area images,

while the inter-image relative direction acquisition unit calculates the relative direction relationship between the first input area image and the second input area image based on the location coordinate information associated with the first image pickup apparatus by which the first input image is taken, the location coordinate information associated with the second image pickup apparatus by which the second input image is taken, and the location coordinate information associated with the particular person estimated by the person tracking unit, and

the person identification unit receives the feature of the first input area image and the feature of the second input area image from the feature extraction unit, and receives the feature of the first registered area image and the feature of the second registered area image from the person area feature storage unit.

3. The image recognition apparatus according to claim 2, wherein the person area feature storage unit updates the features of the registered area images using a result of the determination performed by the person identification unit.

4. The image recognition apparatus according to claim 2, wherein

the person area feature storage unit stores in advance a hash table describing a hash value calculated from each registered area image in relation to the registered area image, and

the person identification unit calculates a hash value from an input area image and identifies a person using the calculated hash value and the hash table.

5. The image recognition apparatus according to claim 1, wherein the person identification unit gives a higher weight to an image with a higher resolution in comparing the features.

6. The image recognition apparatus according to claim 1, wherein the person identification unit gives a higher weight to an image of a person taken from a closer distance in comparing the features.

7. The image recognition apparatus according to claim 1, wherein the feature of each input area image and the feature of each registered area image are each obtained from a face area.

8. The image recognition apparatus according to claim 1, wherein the feature of each input area image and the feature of each registered area image are each obtained from a face area and a trunk area.

9. The image recognition apparatus according to claim 1, wherein the relative direction relationship between images is a relative relationship between the images in terms of an elevation angle and a three-dimensional rotation angle about a rotation axis extending in parallel to an image plane.

10. The image recognition apparatus according to claim 1, wherein the person identification unit estimates a face orientation direction.

11. The image recognition apparatus according to claim 10, wherein when the person identification unit performs identification, the person identification unit uses a different feature depending on the estimated face orientation direction.

12. The image recognition apparatus according to claim 10, wherein the person identification unit estimates the face orientation direction based on a direction in which a person moves.

14. An image recognition system comprising:

the image recognition apparatus according to claim 1; and

a plurality of image pickup apparatuses configured to take images of the particular person from different directions.

13. An image recognition apparatus comprising:

an inter-image relative direction acquisition unit configured to acquire a relative direction relationship between a first input area image and a second input area image obtained by taking images of a particular person from different directions; and

a person identification unit configured to identify the particular person by comparing a feature of the first input area image with a feature of a first registered area image included in a set of registered area images including registered area images obtained by taking images of the particular person or another person from at least three directions, comparing a feature of the second input area image with a feature of a second registered area image of the same person as the person of the first registered area image included in the set of registered area images, and determining whether face orientations respectively captured in the first and second input area images are the same as face orientations respectively captured in the first and second registered area images,

wherein the first registered area image and the second registered area image are selected such that the relative direction relationship between the first registered area image and the second registered area image is equal or similar to the relative direction relationship between the first input area image and the second input area image.

15. An image recognition system including a plurality of image pickup apparatuses and an image recognition apparatus,

the image pickup apparatus comprising:

an image sensing unit installed at a fixed location and configured to take an image of a particular area by performing panning or tilting as required,

the image recognition apparatus comprising;

an acquisition unit configured to acquire a first relationship in terms of a relative direction between the image pickup apparatuses at the locations in a situation in which images of a location within the particular area are taken by the image pickup apparatuses;

a setting unit configured to set a set of images of face areas extracted from a plurality of captured images obtained by taking the same location at the same time by the plurality of image pickup apparatuses;

an obtainment unit configured to obtain a second relationship in terms of a face orientation between the images of face areas; and

a recognition unit configured to recognize a face of the face area by comparing a set of registered images of a plurality of face orientations taken in advance and the set of images of the face area in a state in which the second relationship is kept.

16. The image recognition system according to claim 15, wherein the recognition unit compares a first registered image included in the set of registered images with a first image in the set of images of the face areas and compares a second registered image included in the set of registered images with a second image in the set of images of the face areas thereby to recognize whether a person captured in the images of the face areas is the same person captured in the registered images.

17. The image recognition system according to claim 15, wherein the set of registered images is registered in relation to a rotation angle of a face with respect to a reference face facing right ahead.

18. The image recognition system according to claim 15, further comprising:

a rotation angle estimation unit configured to estimate a rotation angle of a face of an image of a face area,

wherein the recognition unit adjusts the rotation angle of the image of the face area with respect to the rotation angle of the face of the registered image based on the estimated rotation angle and recognizes the face of the face area in the state in which the rotation angle is adjusted.

19. An image recognition method comprising:

calculating a relative direction relationship between a first input area image and a second input area image obtained by taking images of a particular person from different directions;

a person identification unit configured to identify the particular person by comparing a feature of the first input area image with a feature of a first registered area image included in a set of registered area images including registered area images obtained by taking images of the particular person or another person from at least three directions, comparing a feature of the second input area image with a feature of a second registered area image of the same person as the person of the first registered area image included in the set of registered area images, and determining whether the person in the first and second input area images is the same as the person in the first and second registered area images,

wherein the first registered area image and the second registered area image are selected such that the relative direction relationship between the first registered area image and the second registered area image is equal or similar to the relative direction relationship between the first input area image and the second input area image.

20. A non-transitory computer readable medium storing a computer-executable program for causing a computer to perform the image recognition method according to claim 19.