Method and apparatus for capturing facial expressions

• US 8,593,523 B2
• Filed: 03/24/2011
• Issued: 11/26/2013
• Est. Priority Date: 03/24/2010
• Status: Active Grant

• Current Assignee: Industrial Technology Research Institute
• Original Assignee: Industrial Technology Research Institute
• Inventors: Wan, Shian, Huang, Yea-Shuan, Chuang, Shun-Hsu, Liao, Yuan-Shi
• Primary Examiner(s): CHEN, CHIA WEI A
• Application Number:
  US13/070,489
• Publication Number:
  US 20120169895A1
• Time in Patent Office: 978 Days
• Field of Search: 348/222.1, 348169-172, 382115-157, 382181-231, 382/103, 382286-291
• US Class Current: 348/169
• CPC Class Codes: G06V 40/176 Dynamic expression
• CPC Class Codes: H04N 23/611 where the recognised objects include parts of the human body

Abstract

A method and an apparatus for capturing facial expressions are provided, in which different facial expressions of a user are captured through a face recognition technique. In the method, a plurality of sequentially captured images containing human faces is received. Regional features of the human faces in the images are respectively captured to generate a target feature vector. The target feature vector is compared with a plurality of previously stored feature vectors to generate a parameter value. When the parameter value is higher than a threshold, one of the images is selected as a target image. Moreover, a facial expression recognition and classification procedures can be further performed. For example, the target image is recognized to obtain a facial expression state, and the image is classified according to the facial expression state.

Claims

1. A facial image capturing method, comprising:

receiving a plurality of captured images including human faces;

obtaining regional features of the human faces from the captured images, and generating a target feature vector by a feature-point-positioning procedure on the captured images, wherein a plurality of deviation-determination values are obtained therefrom and one of the captured images is selected for generating the target feature vector, wherein the target feature vector is generated according to a displacement analysis of the captured images, and wherein the displacement is generated according to a difference between coordinates of a left eye in the image currently processed and a left eye in the image previously processed as well as a difference between coordinates of a right eye in the image currently processed and a right eye in the image previously processed;

comparing the target feature vector with a plurality of previously stored feature vectors, and generating a parameter value accordingly, wherein when the parameter value is higher than a threshold, one of the captured images is selected as a target image, and the target feature vector corresponding to the target image is stored to be as a new feature vector for comparing with next target feature vector; and

recognizing the target image to obtain a facial expression state and classifying the target image according to the facial expression state.

2. The facial image capturing method according to claim 1, wherein each of the images is processed by using following expressions:

3. The facial image capturing method according to claim 1, wherein the step of comparing the target feature vector with the previously stored feature vectors comprises:

constructing a multi-dimensional feature vector as the target feature vector according to a portion or all of a left eye block, a right eye block, a nose block, and a mouth block of the human face; and

comparing the multi-dimensional feature vector with the previously stored feature vectors to obtain the parameter value.

4. The facial image capturing method according to claim 1, wherein the target image is recognized through a Hierarchical Temporal Memory (HTM) recognition technique.

5. A facial image capturing apparatus, comprising:

an image capturing unit, for capturing a plurality of images comprising human faces;

a feature-point-positioning unit, for receiving the captured images and generating a target feature vector according to regional features of the human faces in the images, wherein by a feature-point-positioning procedure is performed by the feature-point-positioning unit on the captured images, and a plurality of deviation-determination values are obtained therefrom and one of the captured images is selected for generating the target feature vector, wherein the target feature vector is generated according to a displacement analysis of the captured images, and wherein the displacement is generated according to a difference between coordinates of a left eye in the image currently processed and a left eye in the image previously processed as well as a difference between coordinates of a right eye in the image currently processed and a right eye in the image previously processed; and

an analysis unit , for receiving the target feature vector and comparing the target feature vector with a plurality of previously stored feature vectors to generate a parameter value, wherein when the parameter value is higher than a threshold, the analysis unit selects one of the images as a target image and adds the target feature vector corresponding to the target image into the feature vectors; and

a specific-expression-classification unit, wherein the specific-expression-classification unit recognizes the target image to obtain a facial expression state and classifies the target image according to the facial expression state.

6. The facial image capturing apparatus according to claim 5, wherein the image capturing unit is a digital camera, a video camera, a web camera, an IP camera, or a surveillance camera.

7. The facial image capturing apparatus according to claim 5, wherein each of the images is processed by using following expressions:

8. The facial image capturing apparatus according to claim 5, wherein the target feature vector is compared with the previously stored feature vectors by constructing a multi-dimensional feature vector according to one or all of a left eye block, a right eye block, a nose block, and a mouth block of the human face and comparing the multi-dimensional feature vector with the previously stored feature vectors to obtain the parameter value.

9. The facial image capturing apparatus according to claim 5, wherein the specific-expression-classification unit further comprises a recognizer, and the recognizer adopts a HTM recognition technique.