Smooth morphing between personal video calling avatars

• US 8,421,805 B2
• Filed: 02/09/2007
• Issued: 04/16/2013
• Est. Priority Date: 02/09/2006
• Status: Active Grant

• Current Assignee: Dialogic, Inc. (Enghouse Systems Limited)
• Original Assignee: Dialogic, Inc. (Enghouse Systems Limited)
• Inventors: Bishop, Wendell E.
• Primary Examiner(s): Hajnik, Daniel
• Assistant Examiner(s): YANG, ANDREW GUS
• Application Number:
  US11/704,745
• Publication Number:
  US 20070188502A1
• Time in Patent Office: 2,258 Days
• Field of Search: None
• US Class Current: 345/473
• CPC Class Codes: G06T 13/40 of characters, e.g. humans, animals or virtual beings
• CPC Class Codes: G06T 13/80 2D [Two Dimensional] animation, e.g. using sprites
• CPC Class Codes: G06T 2210/44 Morphing
• CPC Class Codes: H04L 12/1827 Network arrangements for conference optimisation or adaptation

Abstract

Dynamic animated avatars selectively morph to reveal or depict the user'"'"'s identity while simultaneously emulating or tracking active movements associated with the user'"'"'s verbal stream or perceived movements of the user. A user elects morphing to transition the rendered avatar to another, more revealing avatar, for example from a cartoon caricature to a posed photographic image. Animation processing identifies active movements derived from an input stream to compute animations of the mouth and profile based on speech, text, or captured video from the user. The computed animations appear as active movements to the currently rendered avatar, and emulate the user speaking or moving according to text, audio or video being transmitted. A user maintains an anonymous or posed identity with respect to the receiving party, and chooses to reveal a different avatar depicting a truer identity at the user'"'"'s discretion, while continuing to display active movements paralleling the user'"'"'s verbal activities.

Claims

1. A computer implemented method for generating image data associated with a user comprising:

selecting a graphical rendition corresponding to a user that obscures a visual identity of the user;

receiving an indication of active movements applicable to the selected graphical rendition and applicable to the progressed image;

rendering the graphical rendition in each frame of a first series of image frames as a rendered image depicting the active movements;

selecting a progressed image that is more revealing of the visual identity of the user than the graphical rendition;

receiving an initiator signal provided by the user to begin transforming the rendered image to the progressed image;

in response to receipt of the user-provided initiator signal, generating the progressed image depicting the active movements in each frame of a second series of image frames concurrently with the first series of image frames;

further in response to the user-provided initiator signal, generating a third series of image frames depicting the active movements, each frame in the third series of image frames being generated by combining a frame from the first series of image frames depicting the active movements with a corresponding frame from the second series of image frames depicting the active movements, where the combining is performed by progressive interpolation to cause the third series of image frames to progress from the rendered image to the progressed image while maintaining the active movements; and

rendering the third series of image frames.

2. The method of claim 1 wherein rendering the graphical rendition in each frame of a first series of image frames as a rendered image depicting the active movements further comprises:

capturing expressive input via an input stream; and

displaying the graphical rendition and animating the rendered image with the first series of image frames using the expressive input, the expressive input corresponding to facial features of spoken verbiage.

3. The method of claim 2 wherein rendering the graphical rendition in each frame of a first series of image frames as a rendered image depicting the active movements further comprises depicting the active movements by emulating the user performing the speech conveyed by the expressive input, the expressive input contained in an input stream including the indication of active movements.

4. The method of claim 3 further comprising:

performing feature extraction on the input stream to derive facial features;

deriving, from the input stream, facial features including a perspective angle and jaw movements corresponding to the spoken verbiage; and

computing, from the facial features, the active movements.

5. The method of claim 2 wherein capturing expressive input via the input stream includes:

applying feature extraction to an input stream;

deriving an audio stream corresponding to text from the input stream; and

computing mouth movements corresponding to spoken text.

6. The method of claim 1 wherein generating the progressed image further comprises:

selecting a transition image that is more revealing of the visual identity of the user than the graphical rendition;

capturing expressive input via an input stream; and

applying the expressive input to the transition image to form the progressed image in each frame of the second series of image frames.

7. The method of claim 6 wherein combining by progressive interpolation further comprises morphing the rendered image depicting the active movements corresponding to the graphical rendition into another rendered image corresponding to the progressed image.

8. The method of claim 7 wherein morphing the rendered image corresponding to the graphical rendition into another rendered image corresponding to the progressed image includes transitioning the rendered image, the rendered image corresponding to a first base image, to correspond to a second base image.

9. The method of claim 8 wherein the morphing the rendered image corresponding to the graphical rendition into another rendered image corresponding to the progressed image includes responding to DTMF signals in band in an audio component of the input stream.

10. The method of claim 2 wherein capturing expressive input via an input stream includes capturing at least one of verbal signals corresponding to an audio stream or visual input corresponding to captured video signals.

11. The method of claim 10 wherein combining by progressive interpolation further comprises rendering a progression of static images, each of the static images including an incremental change over a previous static image in the progression.

12. The method of claim 11 wherein the graphical rendition is a base image including at least one of:

a caricature; and

a posed photographic image;

or a current video feed.

13. The method of claim 1 wherein depicting the active movements further comprises:

extracting features from an input stream, the features indicative of spoken verbiage;

identifying active animations corresponding to the extracted features; and

applying the active animations to the rendered image to generate the appearance of the rendered image verbalizing the spoken language corresponding to the extracted features.

14. The method of claim 13 wherein extracting features from the input stream further comprises:

extracting audio features and video features from the input stream, wherein the extracted features further comprise audio features and video features, further comprising applying active animations to affect mouth, eye and profile appearances of the rendered image.

15. A communications device for generating image data associated with a user comprising:

at least one processor;

an interface to a base image repository being operable to provide to the base image repository a selection of a graphical rendition corresponding to a user that obscures a visual identity of the user;

a selection interface being operable to permit selection of a progressed image that is more revealing of the visual identity of the user than the graphical rendition;

an extractor operable to receive an indication of active movements applicable to the selected graphical rendition and to the selected progressed image;

an animator operable to render the graphical rendition in each frame of a first series of image frames as a rendered image depicting the active movements, and being further operable to be responsive to an initiator signal provided by the user to generate the progressed image depicting the active movements in each frame of a second series of image frames concurrently with the first series of image frames; and

a morpher operable to be responsive to the user-provided initiator signal to generate a third series of image frames depicting the active movements, each frame in the third series of image frames being generated by combining a frame from the first series of image frames depicting the active movements with a corresponding frame from the second series of image frames depicting the active movements, where the combining is performed by progressive interpolation to cause the third series of image frames to progress from the rendered image to the progressed image while maintaining the active movements.

16. The device of claim 15 wherein the extractor is further operable to:

capture expressive input via an input stream;

the animator being responsive to the extractor and operable to display the graphical rendition and animate the rendered image with the first series of image frames using the expressive input, the expressive input corresponding to facial features of spoken verbiage.

17. The device of claim 16 wherein the animator is further operable to:

depict the active movements by emulating user performing the speech conveyed by the expressive input, the expressive input contained in the input stream including the indication of active movements.

18. The device of claim 17 wherein the extractor is further operable to:

perform feature extraction on the input stream to derive facial features;

derive, from the input stream, facial features including a perspective angle and jaw movements corresponding to the spoken verbiage; and

compute, from the facial features, the active movements.

19. The device of claim 16 wherein the extractor further comprises an audio feature extractor operable to:

apply feature extraction to the input stream;

derive an audio stream corresponding to text from the input stream; and

compute mouth movements corresponding to spoken text, further including a video feature extractor operable to extract video features from the input stream;

wherein the extracted features further comprise audio features and video features, the video feature extractor being further operable to apply active animations to affect mouth, eye and profile appearances of the rendered image.

20. The device of claim 15 wherein the morpher is further operable to:

receive a selection of a transition avatar that is more revealing of the visual identity of the user than the graphical rendition;

capture expressive input via an input stream; and

apply the expressive input to the transition avatar to form the progressed image in each frame of the second series of image frames.

21. The device of claim 20 wherein the morpher is further operable respond to DTMF signals in band in an audio component of the input stream.

22. The device of claim 16 wherein the extractor further includes an audio feature extractor and a video feature extractor, the audio feature extractor being operable to capture verbal signals corresponding to an audio stream, the video feature extractor operable to capture visual input corresponding to captured video signals.

24. The device of claim 15 wherein the morpher is further operable to:

receive extracted features from an input stream, the extracted features being indicative of spoken verbiage;

identify active animations corresponding to the extracted features; and

apply the active animations to the rendered image to generate an appearance of the rendered image verbalizing the spoken verbiage corresponding to the extracted features.

23. A computer program product having a non-transitory computer readable medium operable to store a set of encoded instructions for execution by a processor responsive to the instructions for generating image data associated with a user, the set of encoded instructions comprising:

computer program code for selecting a graphical rendition corresponding to a user that obscures a visual identity of the user;

computer program code for capturing expressive input via an input stream;

computer program code for rendering the graphical rendition as a rendered image using the expressive input in each frame of a first series of image frames, the expressive input corresponding to facial features of spoken verbiage;

computer program code for selecting a progressed image that is more revealing of the visual identity of the user than the graphical rendition;

computer program code for receiving an indication provided by the user to transform the rendered image to the progressed image;

computer program code for responding to receiving the indication by generating the progressed image using the expressive input in each frame of a second series of image frames concurrently with the first series of image frames;

computer program code for responding to receiving the indication by generating a third series of image frames that maintain the usage of the expressive input, each frame in the third series of image frames being generated by combining a frame from the first series of image frames with a corresponding frame from the second series of image frames, where the combining is performed by progressive interpolation to generate a progression of resulting morphed image frames that maintain the usage of the expressive input while gradually transforming the rendered image to the progressed image; and

computer program code for rendering the progression of resulting morphed image frames.