Face feature collator, face feature collating method, and program

• US 8,073,206 B2
• Filed: 11/16/2005
• Issued: 12/06/2011
• Est. Priority Date: 11/16/2004
• Status: Expired due to Fees

• Current Assignee: Panasonic Corporation (Panasonic Holdings Corporation)
• Original Assignee: Panasonic Corporation (Panasonic Holdings Corporation)
• Inventors: Kamiyama, Hirotaka, Yashio, Hitoshi
• Primary Examiner(s): Repko, Jason M
• Assistant Examiner(s): Thomas, Mia M
• Application Number:
  US11/718,738
• Publication Number:
  US 20090052747A1
• Time in Patent Office: 2,211 Days
• Field of Search: 382/115, 382/103, 382/118, 382/181, 382/190, 382/195, 382/199, 382/117, 382/282, 382/296, 382/312, 382/209, 348/77, 348/143, 348/153
• US Class Current: 382/118
• CPC Class Codes: G06V 40/171 Local features and components; Facial parts eye characteristics G06V40/18; Occluding parts, e.g. glasses; Geometrical relationships
• CPC Class Codes: H04N 7/181 for receiving images from a plurality of remote sources

Abstract

The invention provides a facial feature verification apparatus capable of synthesizing an image suitable for verification to identify a person subjected to surveillance whom surveillance staff desires to watch. The facial feature verification apparatus includes a plurality of pickup units 1, a pickup controlling unit 2 for controlling the pickup units 1, a displaying unit 3 for displaying images picked up by the pickup units 1, and a verifying unit 4 for detecting a person from the picked up images, extracting a facial image by determining a face area of the person, extracting a facial image from the images picked up by the pickup units 1, synthesizing a plurality of facial features by extracting facial features from the extracted facial images, and verifying the synthesized facial features with the facial features enrolled in advance in a facial image database 5.

Claims

1. A facial feature verification apparatus, comprising:

a plurality of pickup units including a first pickup unit and a second pickup unit that differs from the first pickup unit;

a pickup controlling unit which controls the pickup units;

a displaying unit which displays images picked up by the pickup units; and

a verifying unit which verifies the picked up images with image data enrolled in advance,

wherein the verifying unit includes;

a person detecting portion which detects a first person image in the image picked up by the first pickup unit and a second person image in the image picked up by the second pickup unit;

a face area detecting portion which determines a first face area and a second face area from the detected first person image and second person image to extract a first facial image and a second facial image, respectively;

a facial feature extracting portion which extracts a first facial feature and a second facial feature from the extracted first facial image and second facial image, respectively;

a facial feature synthesizing portion which synthesizes the extracted first facial feature with the second facial feature;

a verification portion which verifies the synthesized facial feature with a facial image enrolled in advance in a facial image database; and

a facial feature holding portion which holds the extracted first facial feature, wherein a face direction determining portion determines whether the face direction of the determined second face area is not directed to the front of a pickup unit.

2. The facial feature verification apparatus according to claim 1, wherein the pickup controlling unit includes:

an external I/F portion which serves as an interface with an external device for carrying out control externally;

a coordinates conversion portion which converts an image coordinate inputted by the external device to a predetermined world coordinate; and

a pickup device controlling portion which calculates a swivel angle for actuating the pickup units.

3. The facial feature verification apparatus according to claim 1, wherein the verification portion includes a pickup unit selecting portion which selects, among the pickup units, the pickup unit for position detection, which detects a position of the detected person, and the pickup unit for pursuit, which carries out pursuit of the detected person.

4. The facial feature verification apparatus according to claim 1, further comprising:

a face direction determining portion which determines face directions of the determined first and second face areas; and

the facial feature synthesizing portion seeks the first facial feature from the facial feature holding portion, and checks whether the face direction of the second facial feature is opposed in direction to the face direction of the first facial feature, and synthesizes the extracted second facial feature with the first facial features opposed in direction.

5. A facial feature verification method, comprising:

providing a plurality of pickup units including a first pickup unit and a second pickup unit that differs from the first pickup unit, a pickup controlling unit which controls the pickup units, a displaying unit which displays images picked up by the pickup units, and a verifying unit which verifies the picked up images with image data enrolled in advance;

detecting a first person image in the image picked up by the first pickup unit and a second person image in the picked up images image picked up by the second pickup unit;

determining a first face area and a second face area from the detected first person image and second person image to extract a first facial image and a second facial image, respectively;

extracting a first facial feature and a second facial feature from the extracted first facial image and second facial image, respectively;

synthesizing the extracted first facial feature with the second facial feature;

verifying the synthesized facial feature with a facial image in a facial image database enrolled in advance; and

holding the extracted first facial feature, wherein a face direction determining portion determined whether the face direction of the determined second face area is not directed to the front of a pickup unit.

6. The facial feature verification method of claim 5, further comprising:

determining a face direction of the determined first face area and a face direction of the determined second face area;

holding the extracted first facial feature;

seeking the first facial feature;

checking whether the face direction of the second facial feature is opposed in direction to the face direction of the first facial feature; and

synthesizing the extracted second facial feature with the first facial features opposed in direction.

7. A non-transitory recording medium on which is stored a program for facial features verifying operations to cause a computer to carry out the steps of:

providing a plurality of pickup units including a first pickup unit and a second pickup unit that differs from the first pickup unit, a pickup controlling unit which controls the pickup units, a displaying unit which displays images picked up by the pickup units, and a verifying unit which verifies the picked up images with image data enrolled in advance;

detecting a first person image in the image picked up by the first pickup unit and a second person image in the picked up images image picked up by the second pickup unit;

determining a first face area and a second face area from the detected first person image and second person image to extract a first facial image and a second facial image, respectively;

extracting a first facial feature and a second facial feature from the extracted first facial image and second facial image, respectively;

synthesizing the extracted first facial feature with the second facial feature;

verifying the synthesized facial feature with a facial image in a facial image database enrolled in advance; and

holding the extracted first facial feature, wherein a face direction determining portion determined whether the face direction of the determined second face area is not directed to the front of a pickup unit.

8. The non-transitory recording medium of claim 7, further causing the computer to carry out the steps of:

determining a face direction of the determined first face area and a face direction of the determined second face area;

holding the extracted first facial feature;

seeking the first facial feature;

checking whether the face direction of the second facial feature is opposed in direction to the face direction of the first facial feature; and

synthesizing the extracted second facial feature with the first facial features opposed in direction.