Avatar animation, social networking and touch screen applications

• US 9,460,541 B2
• Filed: 03/29/2013
• Issued: 10/04/2016
• Est. Priority Date: 03/29/2013
• Status: Active Grant

• Current Assignee: Tahoe Research Limited (f/k/a Learndale Limited) (Vector Capital Corporation)
• Original Assignee: Intel Corporation
• Inventors: Li, Wenlong, Tong, Xiaofeng, Du, Yangzhou, Sachson, Thomas, Wang, Yunzhen
• Primary Examiner(s): Nguyen, Kimbinh T
• Application Number:
  US13/997,259
• Publication Number:
  US 20160005206A1
• Time in Patent Office: 1,285 Days
• Field of Search: 345/473, 345/474
• US Class Current: 1/1
• CPC Class Codes: G06T 13/40 of characters, e.g. humans, animals or virtual beings
• CPC Class Codes: G06T 13/80 2D [Two Dimensional] animation, e.g. using sprites
• CPC Class Codes: G06T 7/20 Analysis of motion motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51
• CPC Class Codes: G06V 40/168 Feature extraction; Face representation
• CPC Class Codes: G09G 2370/022 Centralised management of display operation, e.g. in a server instead of locally
• CPC Class Codes: G09G 5/003 Details of a display terminal, the details relating to the control arrangement of the display terminal and to the interfaces thereto specific for a CRT G09G1/165; for a flat panel G09G3/2092
• CPC Class Codes: H04N 2007/145 Handheld terminals
• CPC Class Codes: H04N 7/157 defining a virtual conference space and using avatars or agents computer conference optimisation or adaptation H04L12/1827

Abstract

Systems and methods may provide for detecting a condition with respect to one or more frames of a video signal associated with a set of facial motion data and modifying, in response to the condition, the set of facial motion data to indicate that the one or more frames lack facial motion data. Additionally, an avatar animation may be initiated based on the modified set of facial motion data. In one example, the condition is one or more of a buffer overflow condition and a tracking failure condition.

Claims

1. An apparatus to animate avatars, comprising:

a frame monitor, implemented as least partly in one or more of configurable logic or fixed-functionality logic hardware, to detect a condition with respect to one or more frames of a video signal associated with a set of facial motion data related to an image capture of a face of one or more individuals;

a motion module, implemented as least partly in one or more of configurable logic or fixed-functionality logic hardware, to modify, in response to the condition, the set of facial motion data to indicate that the one or more frames of the video signal associated with the captured image of the one or more individuals lack facial motion data; and

an avatar module, implemented as least partly in one or more of configurable logic or fixed-functionality logic hardware, to initiate an avatar animation based on the modified set of facial motion data,

wherein frames located proximate to the frames lacking facial motion data are leveraged to compute replacement facial motion data for the frames lacking facial motion data.

2. The apparatus of claim 1, wherein the condition is to be one or more of a buffer overflow condition and a tracking failure condition.

3. The apparatus of claim 1, wherein the avatar module further includes:

a smoothing module, implemented as least partly in one or more of configurable logic or fixed-functionality logic hardware, to apply a smoothing process to the one or more frames to obtain the replacement facial motion data for the one or more frames;

a snapshot module, implemented as least partly in one or more of configurable logic or fixed-functionality logic hardware, to identify a plurality of avatar images based on the modified set of facial motion data and the replacement facial motion data; and

an animation module, implemented as least partly in one or more of configurable logic or fixed-functionality logic hardware, to generate the avatar animation based on the plurality of avatar images and an audio signal associated with the video signal.

4. The apparatus of claim 1, wherein the avatar module is to send the modified set of facial motion data and an audio signal associated with the video signal to a remote server.

5. The apparatus of claim 4, further including a tone module, implemented as least partly in one or more of configurable logic or fixed-functionality logic hardware, to identify a voice tone setting based on a user input, wherein the avatar module is to send the voice tone setting to the remote server.

6. The apparatus of claim 1, further including:

a snapshot module, implemented as least partly in one or more of configurable logic or fixed-functionality logic hardware, to identify a plurality of avatar images based on the modified set of facial motion data;

an animation module, implemented as least partly in one or more of configurable logic or fixed-functionality logic hardware, to generate the avatar animation based on the plurality of avatar images and an audio signal associated with the video signal; and

a content module, implemented as least partly in one or more of configurable logic or fixed-functionality logic hardware, to combine touch screen content with the avatar animation.

7. At least one non-transitory computer readable storage medium comprising a set of instructions which, if executed by a client device, cause the client device to:

detect a condition with respect to one or more frames of a video signal associated with a set of facial motion data related to an image capture of a face of one or more individuals;

modify, in response to the condition, the set of facial motion data to indicate that the one or more frames of the video signal associated with the captured image of the one or more individuals lack facial motion data; and

initiate an avatar animation based on the modified set of facial motion data,

wherein frames located proximate to the frames lacking facial motion data are leveraged to compute replacement facial motion data for the frames lacking facial motion data.

8. The at least one non-transitory computer readable storage medium of claim 7, wherein the condition is to be one or more of a buffer overflow condition and a tracking failure condition.

9. The at least one non-transitory computer readable storage medium of claim 7, wherein the instructions, if executed, cause the client device to:

apply a smoothing process to the one or more frames to obtain the replacement facial motion data for the one or more frames;

identify a plurality of avatar images based on the modified set of facial motion data and the replacement facial motion data; and

generate the avatar animation based on the plurality of avatar images and an audio signal associated with the video signal.

10. The at least one non-transitory computer readable storage medium of claim 7, wherein the instructions, if executed, cause the client device to send the modified set of facial motion data and an audio signal associated with the video signal to a remote server.

11. The at least one non-transitory computer readable storage medium of claim 10, wherein the instructions, if executed, cause the client device to:

identify a voice tone setting based on a user input; and

send the voice tone setting to the remote server.

12. The at least one non-transitory computer readable storage medium of claim 7, wherein the instructions, if executed, cause the client device to:

identify a plurality of avatar images based on the modified set of facial motion data;

generate the avatar animation based on the plurality of avatar images and an audio signal associated with the video signal; and

combine touch screen content with the avatar animation.

13. A method to animate avatars, comprising:

detecting, by a mobile communication device, a condition with respect to one or more frames of a video signal associated with a set of facial motion data related to an image capture of a face of one or more individuals;

modifying, by the mobile communication device, and in response to the condition, the set of facial motion data to indicate that the one or more frames of the video signal associated with the captured image of the one or more individuals lack facial motion data; and

initiating, by one or more of the mobile communication device and a remote server, an avatar animation based on the modified set of facial motion data,

wherein frames located proximate to the frames lacking facial motion data are leveraged to compute replacement facial motion data for the frames lacking facial motion data.

14. The method of claim 13, wherein the condition is one or more of a buffer overflow condition and a tracking failure condition.

15. The method of claim 13, further including:

applying a smoothing process to the one or more frames to obtain the replacement facial motion data for the one or more frames;

identifying a plurality of avatar images based on the modified set of facial motion data and the replacement facial motion data; and

generating the avatar animation based on the plurality of avatar images and an audio signal associated with the video signal.

16. The method of claim 13, wherein initiating the avatar animation includes sending the modified set of facial motion data and an audio signal associated with the video signal to a remote server.

17. The method of claim 16, further including:

identifying a voice tone setting based on a user input; and

sending the voice tone setting to the remote server.

18. The method of claim 13, further including:

identifying a plurality of avatar images based on the modified set of facial motion data;

generating the avatar animation based on the plurality of avatar images and an audio signal associated with the video signal; and

combining touch screen content with the avatar animation.

19. At least one non-transitory computer readable storage medium comprising a set of instructions which, if executed by a server, cause the server to:

receive a set of facial motion data related to an image capture of a face of one or more individuals, and associated with a video signal;

detect an indication in the set of facial motion data that one or more frames of the video signal associated with the captured image of the one or more individuals lack facial motion data;

generate replacement facial motion data for the one or more frames; and

initiate an avatar animation based on the set of facial motion data and the replacement facial motion data,

wherein frames located proximate to the frames lacking facial motion data are leveraged to compute the replacement facial motion data for the frames lacking facial motion data.

20. The at least one non-transitory computer readable storage medium of claim 19, wherein the instructions, if executed, cause a server to:

identify a plurality of avatar images based on the set of facial motion data and the replacement facial motion data; and

generate an avatar animation based on the plurality of avatar images and an audio signal associated with the video signal.

21. The at least one non-transitory computer readable storage medium of claim 19, wherein the instructions, if executed, cause a server to:

receive an audio signal associated with the video signal;

receive a voice tone setting; and

translate the audio signal to an avatar voice based on the voice tone setting, wherein the avatar animation is to be initiated further based on the avatar voice.

22. The at least one non-transitory computer readable storage medium of claim 21, wherein the instructions, if executed, cause a server to stream the set of facial motion data and the avatar voice to a social networking recipient.

23. The at least one non-transitory computer readable storage medium of claim 19, wherein the replacement facial motion data is to be generated based on one or more of a smoothing process and a speech recognition process.