IMAGE PROCESSING APPARATUS AND METHOD

• US 20090041312A1
• Filed: 08/06/2008
• Published: 02/12/2009
• Est. Priority Date: 08/07/2007
• Status: Abandoned Application

• Current Assignee: Kabushiki Kaisha Toshiba (Toshiba Corporation)
• Original Assignee: Kabushiki Kaisha Toshiba (Toshiba Corporation)
• Inventors: Wakasugi, Tomokazu
• Application Number:
  US12/186,916
• Publication Number:
  US 20090041312A1
• Time in Patent Office: Days
• US Class Current: 382/118
• CPC Class Codes: G06V 20/40 in video content extracting overlay text G06V20/62; video retrieval G06F16/70; processing of video elementary streams in video servers H04N21/234; processing of video elementary streams in video clients H04N21/44

Abstract

An image processing apparatus includes a unit configured to detect face areas from frames of an input moving image; a unit configured to identify face conditions, which vary depending on a face direction, from the face areas; a unit configured to classify the face areas based on the face conditions; a unit configured to correlate, when a moving distance of the face areas between adjacent frames is within a threshold value, the face areas in the frames as one sequence; a unit configured to create dictionaries in which the face areas classified based on the conditions are stored for respective sequences; a unit configured to calculate a degree of similarity between face areas, of the same condition, stored in dictionaries in different sequences, to connect sequences whose degree of similarity is high, and to determine that the face areas belonging to the connected sequences are of the same person.

Claims

1. An image processing apparatus comprising:

a face detection unit configured to detect face areas from images of respective frames of an input moving image;

a face condition identification unit configured to identify face conditions, which vary depending on a face direction, a facial expression or a way of shedding light on a face, from images of the face areas;

a face classification unit configured to classify the face areas based on the face conditions;

a sequence creation unit configured to correlate, when the face areas satisfy the condition that a moving distance of the face areas between adjacent frames is within a threshold value, the face areas in the frames as one sequence;

a dictionary creation unit configured to, using image patterns of the face areas classified based on the conditions, create dictionaries for respective sequences;

a face clustering unit configured to calculate a degree of similarity between the dictionaries, created using the image patterns of the face areas in different sequences, for each condition, to connect sequences whose degree of similarity therebetween is high, and to determine that the face areas belonging to the connected sequences are of a face of the same person.

2. The apparatus according to claim 1, wherein

the face condition identification unit extracts lips from the face areas, recognizes an open or closed condition of the lips and, based on the open or closed condition, identifies the facial expression.

3. The apparatus according to claim 2, wherein

the sequence creation unit correlates, when the face areas satisfy, in addition to the condition, the condition that a difference in a size of the face areas in the respective frames is within a predetermined range, the face areas of the respective frames as the one sequence.

4. The apparatus according to claim 2, wherein

the sequence creation unit correlates, when the face areas satisfy, in addition to the condition, the condition that there is no scene switching between the frames, the face areas of the individual frames as the one sequence.

5. An image processing method comprising steps of:

detecting face areas from images of respective frames of an input moving image;

identifying face conditions, which vary depending on a face direction, a facial expression or a way of shedding light on a face, from images of the face areas;

classifying the face areas based on the face conditions;

correlating, when the face areas satisfy the condition that a moving distance of the face areas between adjacent frames is within a threshold value, the face areas in the frames as one sequence;

creating, using image patterns of the face areas classified based on the conditions, dictionaries for respective sequences;

calculating a degree of similarity between the dictionaries, created using the image patterns of the face areas in different sequences, for each condition, connecting sequences whose degree of similarity therebetween is high, and determining that the face areas belonging to the connected sequences are of a face of the same person.

6. A program product stored in a computer readable medium, comprising the instructions of:

inputting a moving image;

detecting face areas from images of respective frames of an input moving image;

identifying face conditions, which vary depending on a face direction, a facial expression or a way of shedding light on a face, from images of the face areas;

classifying the face areas based on the face conditions;

correlating, when the face areas satisfy the condition that a moving distance of the face areas between adjacent frames is within a threshold value, the face areas in the frames as one sequence;

creating, using image patterns of the face areas classified based on the conditions, dictionaries for respective sequences;

calculating a degree of similarity between the dictionaries, created using the image patterns of the face areas in different sequences, for each condition, connecting sequences whose degree of similarity therebetween is high, and determining that the face areas belonging to the connected sequences are of a face of the same person.