EMOTION AUGMENTED AVATAR ANIMATION

• US 20170206694A1
• Filed: 07/30/2015
• Published: 07/20/2017
• Est. Priority Date: 07/30/2015
• Status: Active Grant

• Current Assignee: Intel Corporation
• Original Assignee: Intel Corporation
• Inventors: JIAO, Shaohui, ZHANG, Lidan, LI, Qiang, SHEN, Xiaolu, LI, Wenlong
• Granted Patent: US 10,176,619 B2
• Time in Patent Office: Days
• CPC Class Codes: G06F 2203/011 Emotion or mood input determined on the basis of sensed human body parameters such as pulse, heart rate or beat, temperature of skin, facial expressions, iris, voice pitch, brain activity patterns
• CPC Class Codes: G06F 3/012 Head tracking input arrangements
• CPC Class Codes: G06T 13/40 of characters, e.g. humans, animals or virtual beings
• CPC Class Codes: G06T 2207/20164 Salient point detection; Corner detection
• CPC Class Codes: G06T 2207/30201 Face
• CPC Class Codes: G06T 2207/30241 Trajectory
• CPC Class Codes: G06T 2210/56 Particle system, point based geometry or rendering
• CPC Class Codes: G06T 7/246 using feature-based methods, e.g. the tracking of corners or segments
• CPC Class Codes: G06V 40/174 Facial expression recognition

Abstract

Apparatuses, methods and storage medium associated with emotion augmented animation of avatars are disclosed herein. In embodiments, an apparatus may comprise an animation augmentation engine to receive facial data of a user, analyze the facial data to determine an emotion state of the user, and drive additional animation that supplements animation of the avatar based at least in part on a result of the determination of the emotion state of the user. Other embodiments may be described and/or claimed.

Claims

1. An apparatus for animating an avatar, comprising:

one or more processors; and

an animation augmentation engine to be operated by the processor to receive facial data of a user, analyze the facial data to determine an emotion state of the user, and drive additional animation that supplements animation of the avatar based at least in part on a result of the determination of the emotion state of the user.

2. The apparatus of claim 1, wherein to receive facial data, the animation augmentation engine is to receive facial landmark data of a facial mesh of the user.

3. The apparatus of claim 1, wherein to analyze the facial data, the animation augmentation engine comprises a plurality of emotion classifiers to respectively process the facial data, and generate a plurality of confidence values for corresponding emotion states of the emotion classifiers.

4. The apparatus of claim 3, wherein the plurality of emotion classifiers comprise at least one of an angry state classifier, a disgusted state classifier, a fear state classifier, a happy state classifier, a sad state classifier, or a surprise state classifier.

5. The apparatus of claim 1, wherein to drive additional animation, the animation augmentation engine is to create an emitter for the additional animation, and generate a collection of particles to be emitted from the emitter.

6. The apparatus of claim 5, wherein the animation augmentation engine is to further acquire a movement trajectory, and calculate velocity in the movement trajectory for the particles to be emitted from the emitter.

7. The apparatus of claim 1, further comprising a facial expression tracker to be operated by the one or more processors to receive one or more image frames of the user, analyze the one or more image frames to detect and track facial expressions of the user, and to provide the facial data to the animation augmentation engine based at least in part on a result of the analysis of the one or more image frames.

8. The apparatus of claim 7 further comprising an avatar animation engine to be operated by the one or more processors;

wherein the facial expression tracker is to drive the avatar animation engine to animate the avatar based on facial expressions of the user detected and tracked; and

wherein to drive additional animation, the animation augmentation engine is to drive the avatar animation engine to supplement the animation of the avatar based on facial expressions of the user detected and tracked, with the emotion state based additional animation.

9. The apparatus of claim 8, wherein the facial expression tracker comprises a primary animation message generator to generate primary animation messages based on facial expressions of the user detected and tracked, and send the primary animation messages to the avatar animation engine to drive the avatar animation engine to animate the avatar based on facial expressions of the user;

and the animation augmentation engine comprises a supplemental animation message generator to generate supplemental animation messages based on the emotion state of the user detected, and send the supplemental animation messages to the avatar animation engine to drive the avatar animation engine to supplement the animation of the avatar based on facial expressions of the user, with the emotion state based additional animation.

10. The apparatus of claim 7, wherein the apparatus is a selected one of a smartphone, a computing tablet, an ultrabook, an ebook, or a laptop computer.

11. A method for animating an avatar, comprising:

receiving, by a computing device, facial data of a user;

analyzing, by the computing device, the facial data to determine an emotion state of the user; and

driving, by the computing device, additional animation that supplements animation of the avatar based at least in part on a result of the determination of the emotion state of the user.

12. The method of claim 11, wherein receiving comprises receiving facial landmark data of a facial mesh of the user.

13. The method of claim 11, wherein analyzing comprises respectively processing the facial data with a plurality of emotion classifiers to generate a plurality of confidence values for corresponding emotion states of the emotion classifiers;

wherein the emotion states comprise at least one of an angry state, a disgusted state, a fear state, a happy state, a sad state, or a surprise state.

15. The method of claim 11, wherein driving additional animation comprises creating an emitter for the additional animation, and generating a collection of particles to be emitted from the emitter;

wherein driving additional animation further comprises acquiring a movement trajectory, and calculating velocity in the movement trajectory for the particles to be emitted from the emitter.

17. The method of claim 11, further comprising receiving one or more image frames of the user, analyzing the one or more image frames to detect and track facial expressions of the user, and providing the facial data based at least in part on a result of the analysis of the one or more image frames.

18. The method of claim 17 further comprising driving animation of the avatar based on facial expressions of the user detected and tracked;

and wherein driving additional animation comprises supplementing the animation of the avatar based on facial expressions of the user detected and tracked, with the emotion state based additional animation.

14. (canceled)

16. (canceled)

19. (canceled)

20. (canceled)

21. (canceled)

22. (canceled)

23. (canceled)

24. (canceled)

25. (canceled)

26. One or more computer-readable media comprising instructions that cause an computing device, in response to execution of the instructions by the computing device, to operate an animation augmentation engine to:

receive facial data of a user,

analyze the facial data to determine an emotion state of the user, and

drive additional animation that supplements animation of the avatar based at least in part on a result of the determination of the emotion state of the user.

27. The computer-readable media of claim 26, wherein to receive facial data comprises to receive facial landmark data of a facial mesh of the user.

28. The computer-readable media of claim 26, wherein to analyze the facial data comprises to respectively process the facial data with a plurality of emotion classifiers to generate a plurality of confidence values for corresponding emotion states of the emotion classifiers.

29. The computer-readable media of claim 28, wherein the plurality of emotion classifiers comprise at least one of an angry state classifier, a disgusted state classifier, a fear state classifier, a happy state classifier, a sad state classifier, or a surprise state classifier.

30. The computer-readable media of claim 28, wherein to drive additional animation comprises to create an emitter for the additional animation, and generate a collection of particles to be emitted from the emitter.

31. The computer-readable media of claim 30, wherein to drive additional animation further comprises to further acquire a movement trajectory, and calculate velocity in the movement trajectory for the particles to be emitted from the emitter.

32. The computer-readable media of claim 26, wherein the computing device is further caused to receive one or more image frames of the user, analyze the one or more image frames to detect and track facial expressions of the user, and to provide the facial data based at least in part on a result of the analysis of the one or more image frames.

33. The computer-readable media of claim 32, wherein the computing device is further caused to drive the avatar animation engine to animate the avatar based on facial expressions of the user detected and tracked;

and wherein to drive additional animation comprises to supplement the animation of the avatar based on facial expressions of the user detected and tracked, with the emotion state based additional animation.

34. The computer-readable media of claim 33, wherein to drive animation comprises to generate primary animation messages based on facial expressions of the user detected and tracked, and send the primary animation messages to the avatar animation engine to drive the avatar animation engine to animate the avatar based on facial expressions of the user;

and to drive additional animation comprises to generate supplemental animation messages based on the emotion state of the user detected, and send the supplemental animation messages to the avatar animation engine to drive the avatar animation engine to supplement the animation of the avatar based on facial expressions of the user, with the emotion state based additional animation.