Avatar facial expression techniques

• US 9,330,483 B2
• Filed: 04/11/2011
• Issued: 05/03/2016
• Est. Priority Date: 04/11/2011
• Status: Active Grant

• Current Assignee: Intel Corporation
• Original Assignee: Intel Corporation
• Inventors: Du, Yangzhou, Tong, Xiaofeng, Hu, Wei, Zhang, Yimin
• Primary Examiner(s): Hoang, Peter
• Application Number:
  US13/977,682
• Publication Number:
  US 20140035934A1
• Time in Patent Office: 1,849 Days
• Field of Search: 345/474
• US Class Current: 1/1
• CPC Class Codes: G06T 13/40 of characters, e.g. humans, animals or virtual beings
• CPC Class Codes: G06T 13/80 2D [Two Dimensional] animation, e.g. using sprites
• CPC Class Codes: G06T 17/20 Finite element generation, e.g. wire-frame surface description, tesselation

Abstract

A method and apparatus for capturing and representing 3D wireframe, color and shading of facial expressions are provided, wherein the method includes the following steps: storing a plurality of feature data sequences, each of the feature data sequences corresponding to one of the plurality of facial expressions; and retrieving one of the feature data sequences based on user facial feature data; and mapping the retrieved feature data sequence to an avatar face. The method may advantageously provide improvements in execution speed and communications bandwidth.

Claims

1. An apparatus comprising:

a camera to capture a user'"'"'s facial expressions and head movements;

a face tracking module to track face feature point positions and to decompose the face feature point positions into a facial expression movement and a rigid head movement;

a database module to store a plurality of feature data sequences, each of the feature data sequences corresponding to one of a plurality of facial expression movements;

the face tracking module to retrieve one of the stored feature data sequences based on the captured facial expression movement; and

a mapping module to map the retrieved feature data sequence to an avatar face and to present a similar facial expression movement and a rigid head movement on the avatar face, the facial expression movement being mapped by determining motion vectors for each face feature point position, transforming the motion vectors to avatar face motion vectors on a model avatar face, and smoothing the avatar face motion vectors, the rigid head movement being manned by applying three-axis rotation to the avatar face.

2. The apparatus of claim 1, wherein the plurality of feature data sequences include data for multiple features over a plurality of frames.

3. The apparatus of claim 2, wherein the database module is to further store trend data for each of the multiple features.

4. The apparatus of claim 3, wherein the face tracking module is to retrieve one of the feature data sequences based on trend data corresponding to the user facial feature data.

5. The apparatus of claim 1, wherein the tracking module is to determine the user facial feature data based on predetermined facial wireframe models.

6. The apparatus of claim 1, wherein the mapping module is to map x and v translations of the rigid head movement to the avatar face.

7. The apparatus of claim 1, wherein the mapping module is to determine a lip line for the avatar by finding a shortest path from a left mouth corner to a right mouth corner for each lip segmented from the other lip.

17. The apparatus of claim 1, further comprising a rendering engine to drive the avatar using the mapped feature data sequence.

18. The apparatus of claim 17, further comprising a communications interface to communicate the mapped feature data sequence to a second device, the second device having the rendering engine.

8. A method comprising:

capturing a user'"'"'s facial expressions and head movements;

tracking face feature point positions of the facial expressions and head movements;

decomposing the face feature point positions into a facial expression movement and a rigid head movement;

retrieving one of a plurality of stored feature data sequences using the facial expression movement, each of the feature data sequences corresponding to one of a plurality of facial expression movements; and

mapping the retrieved feature data sequence to an avatar face and presenting a similar facial expression movement and a rigid head movement on the avatar face, the facial expression movement being mapped by determining motion vectors for each face feature point position, transforming the motion vectors to avatar face motion vectors on a model avatar face, and smoothing the avatar face motion vectors, the rigid head movement being mapped by applying three-axis rotation to the avatar face.

9. The method of claim 8 wherein the plurality of feature data sequences include data for multiple features over a plurality of frames.

10. The method of claim 9, further comprising storing trend data for each of the multiple features.

11. The method of claim 8, wherein said retrieving is based on trend data corresponding to the user facial feature data.

12. The method of claim 8, further comprising determining the user facial feature data based on predetermined facial wireframe models.

13. The method of claim 8, further comprising mapping x and y translations of the rigid head movement to the avatar face.

14. The method of claim 13, wherein said mapping comprises determining a lip line for the avatar by finding a shortest path from a left mouth corner to a right mouth corner for each lip segmented from the other lip.

19. The method of claim 8, further comprising communicating the mapped feature data sequence to a second device to render the avatar at the second device.

20. The method of claim 19, further comprising driving the avatar using the mapped feature data sequence.

15. An article comprising a non-transitory machine-accessible medium having stored thereon instructions that, when executed by a machine, cause the machine to perform operations comprising:

capturing a user'"'"'s facial expressions and head movements;

tracking face feature point positions of the facial expressions and head movements;

decomposing the face feature point positions into a facial expression movement and a rigid head movement;

retrieving one of a plurality of stored feature data sequences using the facial expression movement, each of the feature data sequences corresponding to one of a plurality of facial expression movements; and

mapping the retrieved feature data sequence to an avatar face and presenting a similar facial expression movement and a rigid head movement on the avatar face, the facial expression movement being mapped by determining motion vectors for each face feature point position, transforming the motion vectors to avatar face motion vectors on a model avatar face, and smoothing the avatar face motion vectors, the rigid head movement being mapped by applying three-axis rotation to the avatar face.

16. The article of claim 15, wherein the instructions, when executed by a machine, further cause the machine to:

map x and y translation of the rigid head movement to the avatar face.