Animating Speech Of An Avatar Representing A Participant In A Mobile Communication

• US 20100060647A1
• Filed: 11/20/2009
• Published: 03/11/2010
• Est. Priority Date: 10/11/2007
• Status: Active Grant

• Current Assignee: Activision Publishing Incorporated (Microsoft Corporation)
• Original Assignee: International Business Machines Corporation
• Inventors: Wolfe, Martin A., Muirhead, Richard W., Reddington, Francis X., Brown, William A.
• Granted Patent: US 8,125,485 B2
• Time in Patent Office: Days
• US Class Current: 345/473
• CPC Class Codes: G06T 13/205 driven by audio data
• CPC Class Codes: G06T 13/40 of characters, e.g. humans, animals or virtual beings
• CPC Class Codes: Y10S 345/956 Language driven animation
• CPC Class Codes: Y10S 345/957 Actor

Abstract

Animating speech of an avatar representing a participant in a mobile communication including selecting one or more images; selecting a generic animation template; fitting the one or more images with the generic animation template; texture wrapping the one more images over the generic animation template; and displaying the one or more images texture wrapped over the generic animation template. Receiving an audio speech signal; identifying a series of phonemes; and for each phoneme: identifying a new mouth position for the mouth of the generic animation template; altering the mouth position to the new mouth position; texture wrapping a portion of the one or more images corresponding to the altered mouth position; displaying the texture wrapped portion of the one or more images corresponding to the altered mouth position of the mouth of the generic animation template; and playing the portion of the audio speech signal represented by the phoneme.

Claims

1. A computer implemented method of animating speech of an avatar representing a participant in a mobile communication, the avatar displayed on a display screen of a mobile communications device, the method comprising:

displaying, by an animation module installed on the mobile communications device, the avatar representing the participant including;

receiving a text source;

identifying, in dependence upon the text source, the participant;

selecting, from data storage in dependence upon the identification of the participant, one or more images to represent the participant;

selecting from data storage a generic animation template for the participant, the generic animation template having a mouth, the mouth characterized by a mouth position;

fitting the one or more images representing the participant with the generic animation template;

texture wrapping the one more images representing the participant over the generic animation template; and

displaying, by the animation module, the one or more images texture wrapped over the generic animation template;

converting, by the animation module, the text source to an audio speech signal;

identifying, by the animation module from the audio speech signal, a series of phonemes, each phoneme representing a portion of the audio speech signal;

and for each phoneme;

identifying a new mouth position for the mouth of the generic animation template, altering the mouth position of the mouth of the generic animation template to the new mouth position, texture wrapping a portion of the one or more images corresponding to the altered mouth position of the mouth of the generic animation template, displaying the texture wrapped portion of the one or more images corresponding to the altered mouth position of the mouth of the generic animation template, and playing, synchronously with the displayed texture wrapped portion of the one or more images, the portion of the audio speech signal represented by the phoneme.

2. The method of claim 1 wherein:

the text source comprises one or more email messages; and

identifying the participant in dependence upon the text source further comprises identifying the participant as a sender of the one or more email messages.

3. The method of claim 1 further comprising:

storing the texture wrapped generic animation template; and

upon a subsequent mobile communication with the participant, selecting, in dependence upon the identification of the participant, the stored texture wrapped generic animation template to represent the participant.

4. The method of claim 1 wherein:

the text source comprises a file; and

receiving a text source further comprises retrieving the file from remote data storage.

5. The method of claim 1 wherein:

the text source comprises one or more small message system (‘

SMS’

) text messages; and

identifying the participant in dependence upon the text source further comprises identifying the participant as a sender of the SMS text messages.

6. The method of claim 1 wherein fitting the one or more images representing the participant with the generic animation template further comprises:

resizing the one or more images to conform with the size of the generic animation template;

identifying specific facial features of the one or more images;

aligning the specific facial features with corresponding facial features of the generic animation template; and

reshaping the generic animation template to conform with the one or more images.

7. The method of claim 1 wherein identifying, for each phoneme, a mouth position for the mouth of the generic animation template further comprises retrieving the coordinates of the mouth position from a data structure in dependence upon an identification of the phoneme.

8. The method of claim 1 further comprising capturing one or more digital images of the participant and storing the captured digital images for use as the one or more images to represent the participant.

9. The method of claim 1 wherein selecting one or more images to represent the participant further comprises:

identifying the participant in the mobile communication; and

selecting the one or more images in dependence upon the identified participant.

10. The method of claim 1 further comprising:

identifying, by the animation module, one or more background images; and

providing, by the animation module, as a background to the displayed avatar the one or more background images.

11. A computer implemented method of animating speech of an avatar representing a participant in a mobile communication, the avatar displayed on a display screen of a mobile communications device, the method comprising:

displaying, by an animation module installed on the mobile communications device, the avatar representing the participant including;

selecting from data storage one or more images to represent the participant;

selecting from data storage a generic animation template for the participant, the generic animation template having a mouth and at least one emotive feature, the mouth characterized by a mouth position;

fitting the one or more images representing the participant with the generic animation template;

texture wrapping the one more images representing the participant over the generic animation template; and

displaying, by the animation module, the one or more images texture wrapped over the generic animation template;

receiving, by the animation module, an audio speech signal derived from the mobile communication of the participant;

identifying, by the animation module from the audio speech signal, a series of phonemes and one or more points of voice inflection greater than a predetermined threshold, each phoneme representing a portion of the audio speech signal;

for each phoneme;

identifying a new mouth position for the mouth of the generic animation template, altering the mouth position of the mouth of the generic animation template to the new mouth position, texture wrapping a portion of the one or more images corresponding to the altered mouth position of the mouth of the generic animation template, displaying the texture wrapped portion of the one or more images corresponding to the altered mouth position of the mouth of the generic animation template, and playing, synchronously with the displayed texture wrapped portion of the one or more images, the portion of the audio speech signal represented by the phoneme; and

for each point of voice inflection greater than the predetermined threshold, triggering a motion key-frame caption that alters display of the emotive feature synchronously with the playing of a portion of the audio speech signal including the point of voice inflection greater than the predetermined threshold.

12. The method of claim 11 wherein the emotive feature comprises one or more facial features including at least one of an ear, a dimple, an eyebrow, a brow, a chin, and a nose.

13. The method of claim 12 wherein triggering the motion key-frame caption further comprises altering display of the one or more facial features including at least one of moving one or both ears, depressing at least one dimple, raising at least one eyebrow, furrowing the brow, moving the chin, and wrinkling the nose.

14. The method of claim 11 wherein the emotive feature comprises one or more non-facial features including at least one of a finger, a hand, an arm, a shoulder, and a chest.

15. The method of claim 14 wherein triggering the motion key-frame caption further comprises altering display of the one or more non-facial features including at least one of moving one or more fingers, waving one or more hands, raising one or more arms, shrugging one or more shoulders, and expanding and retracting the chest.

16. The method of claim 11 wherein fitting the one or more images representing the participant with the generic animation template further comprises:

resizing the one or more images to conform with the size of the generic animation template;

identifying specific facial features of the one or more images;

aligning the specific facial features with corresponding facial features of the generic animation template; and

reshaping the generic animation template to conform with the one or more images.

17. The method of claim 11 wherein identifying, for each phoneme, a mouth position for the mouth of the generic animation template further comprises retrieving the coordinates of the mouth position from a data structure in dependence upon an identification of the phoneme.

18. A computer implemented method of animating speech of an avatar representing a participant in a mobile communication, the avatar displayed on a display screen of a mobile communications device, the method comprising:

displaying, by an animation module installed on the mobile communications device, the avatar representing the participant including;

selecting from data storage one or more images to represent the participant;

selecting from data storage a generic animation template for the participant, the generic animation template having a mouth, the mouth characterized by a mouth position;

fitting the one or more images representing the participant with the generic animation template;

texture wrapping the one more images representing the participant over the generic animation template; and

displaying, by the animation module, the one or more images texture wrapped over the generic animation template; and

receiving, by the animation module, an audio speech signal derived from the mobile communication of the participant;

identifying a vocal pattern from a particular portion of the audio speech signal;

determining whether the vocal pattern matches a predetermined vocal pattern;

identifying, by the animation module from the audio speech signal, a series of phonemes, each phoneme representing a portion of the audio speech signal; and

for each phoneme;

identifying a new mouth position for the mouth of the generic animation template, altering the mouth position of the mouth of the generic animation template to the new mouth position, texture wrapping a portion of the one or more images corresponding to the altered mouth position of the mouth of the generic animation template, displaying the texture wrapped portion of the one or more images corresponding to the altered mouth position of the mouth of the generic animation template, and playing, synchronously with the displayed texture wrapped portion of the one or more images, the portion of the audio speech signal represented by the phoneme; and

if the vocal pattern of the particular portion of the audio speech signal matches a predetermined vocal pattern, displaying an indication of the predetermined vocal pattern synchronously with the playing of the particular portion of the audio speech signal.

19. The method of claim 18 wherein the predetermined vocal pattern comprises one of:

a vocal pattern used to authenticate a speaker;

a vocal pattern indicating a lying speaker;

a vocal pattern indicating a sad speaker;

a vocal pattern indicating a frightened speaker;

a vocal pattern indicating a happy speaker;

a vocal pattern indicating an excited speaker; and

a vocal pattern indicating a drowsy speaker.

20. The method of claim 18 wherein:

identifying a vocal pattern from a particular portion of the audio speech signal further comprises converting the audio speech signal from an analog audio signal to a digital audio signal and analyzing the digital audio signal to identify the vocal pattern; and

determining whether the vocal pattern matches a predetermined vocal pattern further comprises comparing the identified vocal pattern to a plurality of predetermined vocal patterns stored in a repository of predetermined vocal patterns.