Interactive Avatars for Telecommunication Systems

• US 20130258040A1
• Filed: 04/02/2012
• Published: 10/03/2013
• Est. Priority Date: 04/02/2012
• Status: Active Grant

• Current Assignee: Argela Yazilim Ve Bilisim Teknolojileri San. Ve Tic A.S.
• Original Assignee: Argela Yazilim Ve Bilisim Teknolojileri San. Ve Tic A.S.
• Inventors: Kaytaz, Bulent, Eliacik, C. Bora, Aykac, Alper, Uluderya, Serkant, Cakici, Fatih, Idilkurt, Ceyda, Tekdas, Okan, Tanriverdi, Tolga, Dursun, Galip, Ozcu, Neva
• Granted Patent: US 9,402,057 B2
• Time in Patent Office: Days
• US Class Current: 348/14.07
• CPC Class Codes: H04L 65/1046 Call controllers; Call servers
• CPC Class Codes: H04L 65/1069 Session establishment or de-establishment
• CPC Class Codes: H04L 67/131 Protocols for games, networked simulations or virtual reality
• CPC Class Codes: H04M 3/42068 where the identifier is used to access a profile
• CPC Class Codes: H04N 7/157 defining a virtual conference space and using avatars or agents computer conference optimisation or adaptation H04L12/1827
• CPC Class Codes: H04W 4/21 for social networking applications

Abstract

This invention is an apparatus an methods which integrates users'"'"' avatars into IP-based calling and messaging telecommunication services in such a way that avatar can be animated in real-time according to spoken words, anticipated gestures and pre-set and stored emotions. The invention includes an online editor to generate and customize users avatars.

Claims

1. An avatar-calling service processing a call from a calling IP client to a called IP client wherein calling IP clients'"'"' avatar is integrated into said calling service, said service is accessible through said IP clients'"'"' communication device wherein said calling IP client with avatar has video calling and real-time rendering capabilities and said called IP client'"'"'s end-user communication device has real-time rendering and video/graphics display capabilities, said calling IP client has an application which integrates avatar with lip and motion-syncs into avatar calling service, said avatar-calling service comprising:

an avatar call control server communicating with an avatar inserter using an HTTP/XML interface retrieving said calling IP client'"'"'s avatar information from an avatar repository, said avatar repository storing avatars of users;

said avatar inserter retrieving said calling IP client'"'"'s avatar information from said avatar repository and transferring said retrieved avatar information to said avatar call control server;

said avatar call control server sending said avatar information with calling information.

2. The method of claim 1, wherein said communication device is any of the following:

smartphone, a software-based telephone, a cellular telephone, a personal computer (PC), a laptop, a personal digital assistant (PDA), an IP-based phone, a personal tablet computer.

3. The method of claim 2, wherein the communication device is haptic-capable.

4. The method of claim 1, wherein the avatar is an animated 2D or 3D digital representations of said IP client'"'"'s user.

5. The method of claim 4, wherein said animated 2D and/or 3D avatars are synchronized to the users lip, face and body movements in real time.

6. The method of claim 1, wherein said Avatar-Calling service is one of the following:

3G call, 4G call, video call, VoIP call or video conferencing.

7. The method of claim 1, wherein the method further comprises sending emotions and facial expressions using in-band DTMF signaling.

8. The method of claim 1, wherein said calling party'"'"'s voice is modified during an Avatar-Calling Service.

9. The method of claim 1, wherein, in action-reply mode, the replying or called party'"'"'s finger interactions on the avatar seen in the message or call on the touchscreen of said communication device are returned to said messaging or calling party in animated action form.

10. The method of claim 3, wherein said finger interactions are on a touchscreen, creating viewable effects and/or change the appearance of said avatars.

11. The method of claim 1, wherein avatars are customized by morphing one avatar into another avatar.

12. An avatar portal service providing an online editor to one or more IP clients wherein each of said IP clients create and customize digital avatars based on detected key facial features detected from at least one uploaded photograph of one or more subscribers associated with said IP clients, said portal service comprising:

an avatar web server storing data corresponding to digital avatars associated with said IP clients and preferences associated with subscribers associated with each of said IP clients and rendering an avatar portal interface to subscribers;

a photofit server creating the 3D model of uploaded photographs of said subscribers;

an avatar editor server rendering said user'"'"'s avatars according to a plurality of customization operations;

a video renderer creating pre-encoded stock footages of said avatars;

an avatar repository receiving said pre-encoded stock footages and storing said uploaded photographs, avatars and pre-encoded stock footages of said subscribers.

said pre-encoded stock footages created by said video renderer being used in said avatar-calling service for lip, face and body movement synchronization operations.

13. The portal service of claim 12, wherein said IP client is implemented as part of any of the following communication devices:

Smartphone, a software-based telephone, a cellular telephone, a personal computer (PC), a laptop, a personal digital assistant (PDA), an IP-based phone, a personal tablet computer.

14. The portal service of claim 12, wherein avatars are animated 2D or 3D digital representations of said IP client'"'"'s user.

15. The portal service of claim 14, wherein said animated 2D and/or 3D avatars are synchronized to the users lip, face and body movements in real time.

16. The portal service of claim 12, wherein said users can create a profile, said profile comprising a set of user information including an address book, play-rules and preferences, which define how avatars are inserted into calling services.

17. The portal service of claim 12, wherein said portal service is used to create clothes and accessories for avatars via his/her IP client.

18. The portal service of claim 12, wherein said created clothes and accessories can be sold to or shared with other IP clients through said portal service.

19. The portal service of claim 12, wherein users can choose from pre-created avatars available on portal service.

20. The portal service of claim 12, wherein avatars are further customized by morphing one avatar into another avatar.

21. The portal service of claim 12, avatars are based on at least one uploaded photograph, by detecting and calculating key facial features automatically.

22. The portal service of claim 21, wherein hairstyle seen on said photograph of said user is detected and modeled automatically for use in created avatars.

23. The portal service of claim 21, wherein said users assign virtual human-like functions to avatars.

24. A method implementing an avatar-messaging service, processing a message from a messaging IP client to a messaged IP client wherein messaging IP clients'"'"' avatar is integrated into said messaging service, said service is accessible through said called IP clients'"'"' communication device wherein, said messaging IP client with avatar has video messaging and real time rendering capabilities;

said messaged IP client'"'"'s end-user communication device has messaging, real time rendering and video/graphics display capabilities, said messaging IP client has an application which integrates avatar with lip-syncs into avatar messaging service, said method comprises the steps of;

logging into an avatar portal through said IP client;

an email server communicating with both clients through POP3/IMAP protocol for retrieving messages and SMTP protocol for sending mail messages;

a notification server communicating with both clients through HTTP/HTTPS;

a transcoder server communicating with said avatar repository for optimizing said avatar files stored in avatar repository, said optimized avatars used in avatar-messaging service.

25. A computer-based method as implemented at a server facilitating communication between a first and second IP client, said method comprising:

said server receiving a first pre-assigned key at said server from said first IP client, said first IP client communicating with said second IP client, said server storing a plurality of avatars and a plurality of emotions, at least one avatar among said stored plurality of avatars in said server associated with said first IP client and each emotion associated with a pre-assigned key;

said server identifying a first emotion corresponding to said received first pre-assigned key;

said server retrieving data corresponding to said first emotion and modifying avatar associated with said first IP client to show said first emotion based on said retrieved data corresponding to said first emotion; and

said server rendering said modified avatar associated with said first IP client showing said first emotion at said second IP client.

26. The computer-based method of claim 25, wherein said first pre-assigned key is sent through In-band Dual-Tone Multi-Frequency Signaling (DTMF).

27. The computer-based method of claim 25, wherein said computer-based method further comprises:

said server receiving a second pre-assigned key at said server from said first IP client;

said server identifying a second emotion corresponding to said received second pre-assigned key;

said server retrieving data corresponding to said second emotion and modifying avatar associated with said first IP client to show said second emotion based on said retrieved data corresponding to said second emotion; and

said server rendering said modified avatar associated with said first IP client showing said second emotion at said second IP client.

28. The computer-based method of claim 27, wherein said first pre-assigned key is sent through In-band Dual-Tone Multi-Frequency Signaling (DTMF).

29. The computer-based method of claim 25, wherein data corresponding to said plurality of emotions are previously generated and stored in said server.

30. The computer-based method of claim 29, wherein said plurality of emotions are any of the following:

happy, sad, scared, or surprised.

31. A computer-based method as implemented at a server facilitating communication between a first and second IP client, said server storing a plurality of avatars and data corresponding to a plurality of moods, said method comprising:

said server rendering, at said second IP client, an avatar modified to represent a first mood associated with said first IP client, said first IP client communicating with said second IP client;

said server receiving an indication of alteration of a mood from said first mood to a second mood based on a detection of a change in voice tone at said first IP client;

said server retrieving data corresponding to said second mood and modifying said avatar associated with said first IP client to reflect said second mood; and

said server rendering said modified avatar reflective of said second mood at said second IP client.

32. The computer-based method of claim 31, wherein data corresponding to said plurality of moods are previously generated and stored in said server.

33. The computer-based method of claim 31, wherein said plurality of moods are any of the following:

happy, sad, scared, or surprised.

34. A computer-based method as implemented at a server facilitating communication between a first and second IP client, said server storing a plurality of avatars and data corresponding to a plurality of seasons, said method comprising:

said server rendering, at said second IP client, an avatar modified to represent a first season associated with said first IP client, said first IP client communicating with said second IP client;

said server receiving an indication of alteration of a season from said first season to a second season;

said server retrieving data corresponding to said second season and modifying said avatar associated with said first IP client to reflect said second season; and

said server rendering, at said second IP client, said modified avatar reflective of said second season.

35. A computer-based method as implemented at a server facilitating communication between a first and second IP client, said server storing a plurality of avatars, said method comprising:

said server rendering, at said second IP client, an avatar associated with said first IP client, said first IP client communicating with said second IP client;

said server receiving an indication of a change in voice tone, sent from said first IP client, corresponding to a physical condition;

said server retrieving data corresponding to an effect indicative of said physical condition and modifying said avatar associated with said first IP client to reflect said physical condition; and

said server rendering, at said second IP client, said modified avatar reflective of said physical condition.

36. The computer-based method of claim 35, wherein said physical condition is a physical ailment.

37. The computer-based method of claim 36, wherein said physical ailment is a cold.

38. The computer-based method of claim 35, wherein said physical condition indicates tanning and said modified avatar reflects the tanned condition.

39. A computer-based method as implemented at a server facilitating communication between a first and second IP client, said server storing a plurality of avatars and data corresponding to a plurality of interactions, said method comprising:

said server rendering, at said second IP client, an avatar associated with said first IP client, said first IP client communicating with said second IP client;

said server receiving an indication of an physical interaction, at said second IP client, with said avatar associated with said first IP client;

said server retrieving data corresponding to said physical interaction and modifying said avatar associated with said first IP client to reflect said interaction; and

said server rendering, at said second IP client, said modified avatar reflective of an image corresponding to said physical interaction.

40. The computer-based method of claim 39, wherein said physical interaction is a touch screen interaction.

41. The computer-based method of claim 39, wherein said first and second IP clients interact using their corresponding avatars based on speech-to-text or text-to-speech techniques.