INFORMATION PROCESSING APPARATUS AND CONTROL METHOD THEREFOR

• US 20070122036A1
• Filed: 09/19/2006
• Published: 05/31/2007
• Est. Priority Date: 09/26/2005
• Status: Active Grant

• Current Assignee: Canon Kabushiki Kaisha (Canon Inc.)
• Original Assignee: Canon Kabushiki Kaisha (Canon Inc.)
• Inventors: Matsugu, Masakazu, Kaneda, Yuji, Mori, Katsuhiko
• Granted Patent: US 8,542,928 B2
• Time in Patent Office: Days
• US Class Current: 382/190
• CPC Class Codes: G06V 10/454 Integrating the filters into a hierarchical structure, e.g. convolutional neural networks [CNN]
• CPC Class Codes: G06V 40/171 Local features and components; Facial parts eye characteristics G06V40/18; Occluding parts, e.g. glasses; Geometrical relationships
• CPC Class Codes: G06V 40/174 Facial expression recognition
• CPC Class Codes: G06V 40/176 Dynamic expression

Abstract

An information processing apparatus includes an image input unit which inputs image data containing a face, a face position detection unit which detects, from the image data, the position of a specific part of the face, and a facial expression recognition unit which detects a feature point of the face from the image data on the basis of the detected position of the specific part and determines facial expression of the face on the basis of the detected feature point. The feature point is detected at a detection accuracy higher than detection of the position of the specific part. Detection of the position of the specific part is robust to a variation in the detection target.

Claims

1. An information processing apparatus comprising:

an input unit adapted to input image data containing a face;

a first detection unit adapted to detect, from the image data, a position of a specific part of the face;

a second detection unit adapted to detect a feature point of the face from the image data on the basis of the detected position of the specific part; and

a determination unit adapted to determine facial expression of the face on the basis of the detected feature point,

wherein said second detection unit has higher detection accuracy than detection accuracy of said first detection unit, and said first detection unit is robust to a variation in a detection target.

2. The apparatus according to claim 1, wherein said first detection unit detects the position of the specific part of the face on the basis of image data with a lower resolution than image data to be processed by said second detection unit.

3. The apparatus according to claim 1, further comprising a storage unit adapted to store image data of the face with predetermined facial expression,

wherein said determination unit determines first facial expression on the basis one of a difference and a ratio between a first feature amount calculated by using the feature point in the input image data and a second feature amount calculated by using the feature point in the image data stored in said storage unit.

4. The apparatus according to claim 3, wherein said determination unit executes the determination after multiplying the feature amount by a preset weighting value.

5. The apparatus according to claim 1, wherein

the input image data includes a moving image containing a plurality of frames,

said first detection unit detects the position of the specific part of the face in each of the frames,

said second detection unit detects the feature point of the face in each of the frames, and

said determination unit determines a change in the facial expression of the face on the basis of a variation in the feature point between frames adjacent to each other.

6. The apparatus according to claim 1, further comprising a correction unit adapted to correct luminance of the input image data to enhance tone of a low-luminance region.

7. The apparatus according to claim 1, wherein the specific part is at least one of a barycenter of a face region, a barycenter of an eye region, a barycenter of a mouth region, a nostril barycentric position, a nose edge, and a barycenter of a flesh color region.

8. The apparatus according to claim 1, wherein the feature point is at least one of a nostril barycenter, a barycenter of a nose edge, an end of an eyebrow, upper and lower feature points of the eyebrow, a median of the eyebrow, an end of an eye, upper and lower feature points of the eye, an end of a mouth, and upper and lower feature points of the mouth.

9. The apparatus according to claim 1, wherein the feature point includes at least one of a shape of an eyebrow, a distance between a left eyebrow and a right eyebrow, a distance between an eyebrow and an eye, a distance between an eye end and a mouth end, a distance between a nostril barycenter and the mouth end, a distance between a nose edge and the mouth end, a distance between a barycenter of a face region and the mouth end, a distance between eye ends, a distance between upper and lower edges of an eye region, a distance between mouth ends, a distance between upper and lower edges of a mouth region, wrinkles in forehead and glabella regions, and wrinkles in left and right cheek regions.

10. The apparatus according to claim 1, wherein the feature point is expressed by relative coordinates to the specific part.

11. The apparatus according to claim 1, wherein said determination unit corrects a position of the feature point on the basis of a layout of one of the feature point and the specific part in the input image data and executes the determination on the basis of the feature point with the corrected position.

12. The apparatus according to claim 1, further comprising a holding unit adapted to hold the input image data when said determination unit determines predetermined facial expression.

13. The apparatus according to claim 1, further comprising a display unit adapted to display the input image data when said determination unit determines predetermined facial expression.

14. The apparatus according to claim 1, further comprising a setting unit adapted to set a region in the image data on the basis of the position of the specific part detected by said first detection unit,

wherein said second detection unit detects the feature point of the face from the set region.

16. A computer program stored in a computer-readable storage medium to cause a computer to function as the information processing apparatus of claim 1.

17. A computer-readable storage medium which stores the computer program of claim 16.

15. A control method for an information processing apparatus for processing image data containing a face, comprising steps of:

inputting image data containing a face;

detecting, from the image data, a position of a specific part of the face;

detecting a feature point of the face from the image data on the basis of the detected position of the specific part; and

determining facial expression of the face on the basis of the detected feature point,

wherein the second detection step has higher detection accuracy than detection accuracy of the first detection step, and the first detection step is robust to a variation in a detection target.