Smooth morphing between personal video calling avatars

• US 20070188502A1
• Filed: 02/09/2007
• Published: 08/16/2007
• Est. Priority Date: 02/09/2006
• Status: Active Grant

• Current Assignee: Dialogic, Inc. (Enghouse Systems Limited)
• Original Assignee: Dialogic, Inc. (Enghouse Systems Limited)
• Inventors: Bishop, Wendell E.
• Granted Patent: US 8,421,805 B2
• Time in Patent Office: Days
• US Class Current: 345/473
• CPC Class Codes: G06T 13/40 of characters, e.g. humans, animals or virtual beings
• CPC Class Codes: G06T 13/80 2D [Two Dimensional] animation, e.g. using sprites
• CPC Class Codes: G06T 2210/44 Morphing
• CPC Class Codes: H04L 12/1827 Network arrangements for conference optimisation or adaptation

Abstract

Dynamic animated avatars selectively morph to reveal or depict the user'"'"'s identity while simultaneously emulating or tracking active movements associated with the user'"'"'s verbal stream or perceived movements of the user. A user elects morphing to transition the rendered avatar to another, more revealing avatar, for example from a cartoon caricature to a posed photographic image. Animation processing identifies active movements derived from an input stream to compute animations of the mouth and profile based on speech, text, or captured video from the user. The computed animations appear as active movements to the currently rendered avatar, and emulate the user speaking or moving according to text, audio or video being transmitted. A user maintains an anonymous or posed identity with respect to the receiving party, and chooses to reveal a different avatar depicting a truer identity at the user'"'"'s discretion, while continuing to display active movements paralleling the user'"'"'s verbal activities.

Claims

1. A method for rendering image data associated with a user comprising:

selecting a graphical rendition corresponding to a user;

receiving an indication of active movements applicable to the selected graphical rendition;

rendering the graphical rendition as a rendered image depicting the active movements;

receiving an initiator signal to transform the rendered image to a progressed image;

transitioning the rendered image to the progressed image; and

maintaining the active movements of the rendered image during the transitioning.

2. The method of claim 1 wherein rendering depicting the active movements further comprises:

capturing expressive input via an input stream; and

displaying the graphical rendition and animating the rendered image using the expressive input, the expressive input corresponding to facial features of spoken verbiage.

3. The method of claim 2 wherein rendering further comprises depicting the active movements by emulating the user performing the speech conveyed by the expressive input, the expressive input contained in an input stream including the indication of active movements.

4. The method of claim 3 further comprising:

performing feature extraction on the input stream to derive facial features;

deriving, from the input stream, facial features including a perspective angle and jaw movements corresponding to the spoken verbiage; and

computing, from the facial features, the active movements.

5. The method of claim 2 wherein capturing expressive input includes:

applying feature extraction to an input stream;

deriving an audio stream corresponding to text from the input stream; and

computing mouth movements corresponding to spoken text.

6. The method of claim 1 wherein the graphical rendition is a base avatar derived from a base image, transitioning further comprising:

selecting a transformed base image corresponding to a transition avatar; and

applying a set of changes to the rendered image such that the rendered image appears more like the transition avatar than the base avatar.

7. The method of claim 6 wherein transitioning comprises morphing a first rendered image corresponding to the graphical rendition into a second rendered image corresponding to the progressed image.

8. The method of claim 7 wherein morphing includes transitioning the rendered image, the rendered image corresponding to a first base image, to correspond to a second base image.

9. The method of claim 8 wherein the morphing includes:

receiving an input stream, the morphing responsive to DTMF signals inband in an audio component of the input stream; and

combining the active movements and transitioning simultaneously in the rendered image.

10. The method of claim 2 wherein capturing expressive input includes capturing at least one of verbal signals corresponding to an audio stream or visual input corresponding to captured video signals.

11. The method of claim 10 wherein transitioning comprises rendering a progression of static images, each of the static images having an incremental change over a previous static image in the progression.

12. The method of claim 11 wherein graphical rendition is a base image including at least one of:

a caricature;

a posed photographic image;

or

a current video feed.

13. The method of claim 1 wherein transitioning further comprises:

extracting features from an input stream, the features indicative of spoken verbiage;

identifying active animations corresponding to the extracted features; and

applying the active animations to the rendered image to generate the appearance of the rendered image verbalizing the spoken language corresponding to the extracted features.

14. The method of claim 13 wherein extracting features further comprises:

extracting audio features and video features from the input stream, wherein the extracted features further comprise audio features and video features, further comprising applying active animations to effect mouth, eye and profile appearances of the rendered image.

15. A communications device for rendering image data associated with a user comprising:

an interface to a base image repository responsive to a selection of a graphical rendition corresponding to a user;

an extractor operable to receiving an indication of active movements applicable to the selected graphical rendition;

an animator operable to render the graphical rendition as a rendered image depicting the active movements; and

the morpher responsive to an initiator signal to transform the rendered image to a progressed image, the morpher further operable to transitioning the rendered image to the progressed image, the animator further operable to maintain the active movements of the rendered image during the transitioning.

16. The device of claim 15 wherein the extractor is further operable to:

capture expressive input via an input stream;

the animator responsive to the extractor and operable to display the graphical rendition and animating the rendered image using the expressive input, the expressive input corresponding to facial features of spoken verbiage.

17. The device of claim 16 wherein the animator is further operable to depict the active movements by emulating the user performing the speech conveyed by the expressive input, the expressive input contained in an input stream including the indication of active movements.

18. The device of claim 17 wherein the extractor is further operable to

perform feature extraction on the input stream to derive facial features;

derive, from the input stream, facial features including a perspective angle and jaw movements corresponding to the spoken verbiage; and

computing, from the facial features, the active movements.

19. The device of claim 16 wherein the extractor further comprises an audio feature extractor operable to:

apply feature extraction to an input stream;

derive an audio stream corresponding to text from the input stream; and

compute mouth movements corresponding to spoken text, further including a video feature extractor operable to extracting video features from the input stream;

wherein the extracted features further comprise audio features and video features, further comprising applying active animations to effect mouth, eye and profile appearances of the rendered image.

20. The device of claim 15 wherein the graphical rendition is a base avatar derived from a base image, the morpher further operable to comprising:

receive a selection a transformed base image corresponding to a transition avatar; and

apply a set of changes to the rendered image such that the rendered image appears more like the transition avatar than the base avatar, wherein transitioning comprises morphing a first rendered image corresponding to the graphical rendition into a second rendered image corresponding to the progressed image, the morpher further operable to;

receive extracted features from an input stream, the features indicative of spoken verbiage;

identify active animations corresponding to the extracted features; and

apply the active animations to the rendered image to generate the appearance of the rendered image verbalizing the spoken language corresponding to the extracted features.

21. The device of claim 20 wherein the morpher is further operable to:

receive an input stream, the morphing responsive to DTMF signals inband in an audio component of the input stream; and

combine the active movements and transitioning simultaneously in the rendered image.

22. The device of claim 16 wherein extractor further includes an audio feature extractor and a video feature extractor, the audio feature extractor operable to capture verbal signals corresponding to an audio stream, the video feature extractor operable to capture visual input corresponding to captured video signals.

23. A computer program product having a computer readable medium operable to store a set of encoded instructions for execution by a processor responsive to the instructions for rendering image data associated with a user comprising:

computer program code for selecting a graphical rendition corresponding to a user;

computer program code for rendering the graphical rendition as a rendered image using active movements, further comprising;

computer program code for capturing expressive input via an input stream;

computer program code for displaying the graphical rendition; and

computer program code for animating the rendered image using the expressive input, the expressive input corresponding to facial features of spoken verbiage;

computer program code for receiving an indication to transform the rendered image to a progressed image

computer program code for transitioning the rendered image to the progressed image, further including computer program code for morphing a first rendered image corresponding to the graphical rendition into a second rendered image corresponding to the progressed image; and

computer program code for maintaining the active movements during the transitioning.