APPARATUS AND METHOD FOR CONTROLLING AVATAR USING EXPRESSION CONTROL POINT

• US 20120139830A1
• Filed: 11/30/2011
• Published: 06/07/2012
• Est. Priority Date: 12/01/2010
• Status: Active Grant

• Current Assignee: Samsung Electronics Co. Ltd.
• Original Assignee: Samsung Electronics Co. Ltd.
• Inventors: Kim, Jung Bae, Bang, Won Chul, Kim, Do Kyoon, HWANG, Young Kyoo
• Granted Patent: US 9,298,257 B2
• Time in Patent Office: Days
• US Class Current: 345/156
• CPC Class Codes: G06F 3/012 Head tracking input arrangements
• CPC Class Codes: G06T 2207/30196 Human being; Person
• CPC Class Codes: G06V 40/16 Human faces, e.g. facial parts, sketches or expressions
• CPC Class Codes: G06V 40/174 Facial expression recognition

Abstract

An apparatus and method for controlling an avatar using expression control points are provided. The apparatus may track positions of feature points and a head movement from an expression of a user, remove the head movement from the positions of the feature points, and generate expression control points. Accordingly, the expression of the user may be reflected to a face of the avatar.

Claims

1. An apparatus comprising:

a processor to control one or more processor-executable units;

an expression tracking unit to track positions of feature points of a facial expression of a user and a head movement of the user;

a feature point position resetting unit to reset the positions of the feature points of the facial expression by removing the head movement from the tracked positions of the feature points; and

a control point generating unit to generate expression control points using an expression parameter based on the reset positions of the feature points.

2. The apparatus of claim 1, wherein the expression tracking unit extracts the positions of the feature points of the facial expression by learning a facial appearance, and estimates the head movement based on a head rotation angle using the extracted positions of the feature points.

3. The apparatus of claim 1, wherein the feature point position resetting unit resets the positions of the feature points with a reference expression of the user looking in a forward direction, by using one or more of an inverse translation, an inverse rotation, and an inverse scale based on the head movement.

4. The apparatus of claim 1, wherein the control point generating unit generates the expression control points using the expression parameter generated from marker-based motion capture data.

5. The apparatus of claim 4, wherein the control point generating unit updates the expression parameter in accordance with variation of an expression component based on the reset positions of the feature points and variation of an expression component based on a position of the control point generated from the expression parameter.

6. The apparatus of claim 1, further comprising an expression controlling unit to control an expression of an avatar using the expression control points.

7. The apparatus of claim 6, wherein the expression controlling unit controls the expression of the avatar by converting the expression control points into control coordinates on a face mesh for control of the avatar that corresponds to a face of the user.

8. A method comprising:

tracking positions of feature points of a facial expression of a user and a head movement of the user;

resetting the positions of the feature points of the facial expression by removing the head movement from the tracked positions of the feature points; and

generating, by way of a processor, expression control points using an expression parameter based on the reset positions of the feature points.

9. The method of claim 8, wherein the tracking extracts the positions of the feature points of the facial expression by learning a facial appearance, and estimates the head movement based on a head rotation angle using the extracted positions of the feature points.

10. The method of claim 8, wherein the resetting resets the positions of the feature points with a reference expression of the user looking in a forward direction, by using one or more of an inverse translation, an inverse rotation, and an inverse scale based on the head movement.

11. The method of claim 8, wherein the generating of the expression control points generates the expression control points using the expression parameter generated from marker-based motion capture data.

12. The method of claim 11, wherein the generating of the expression control points updates the expression parameter, in accordance with variation of an expression component based on the reset positions of the feature points and in accordance with variation of an expression component based on a position of the control point generated from the expression parameter.

13. The method of claim 8, further comprising controlling an expression of an avatar using the control point.

14. The method of claim 13, wherein the controlling of the expression of the avatar controls the expression of the avatar by converting the expression control points into control coordinates on a face mesh for control of the avatar that corresponds to a face of the user.

15. A non-transitory computer readable recording medium storing a program to cause a computer to implement the method of claim 8.

16. A method of controlling an avatar, the method comprising:

tracking positions of feature points of a facial expression of a user and a head movement of the user; and

generating, by way of a processor, expression control points for controlling the avatar using an expression parameter obtained by removing the head movement of the user from the tracked positions of the feature points.

17. A non-transitory computer readable recording medium storing a program to cause a computer to implement the method of claim 16.