Avatars in social interactive television

• US 8,098,905 B2
• Filed: 05/24/2011
• Issued: 01/17/2012
• Est. Priority Date: 05/01/2008
• Status: Active Grant

• Current Assignee: AT&T Intellectual Property I LP (AT&T, Inc.)
• Original Assignee: AT&T Intellectual Property I LP (AT&T, Inc.)
• Inventors: Amento, Brian Scott, Harrison, Christopher, Stead, Larry
• Primary Examiner(s): PERUNGAVOOR, SATHYANARAYA V
• Application Number:
  US13/114,879
• Publication Number:
  US 20110225603A1
• Time in Patent Office: 238 Days
• Field of Search: 382/118
• US Class Current: 382/118
• CPC Class Codes: G06T 13/40 of characters, e.g. humans, animals or virtual beings
• CPC Class Codes: H04N 21/23412 for generating or manipulating the scene composition of objects, e.g. MPEG-4 objects
• CPC Class Codes: H04N 21/42203 sound input device, e.g. microphone
• CPC Class Codes: H04N 21/4223 Cameras H04N23/00 takes precedence
• CPC Class Codes: H04N 21/44012 involving rendering scenes according to scene graphs, e.g. MPEG-4 scene graphs
• CPC Class Codes: H04N 21/4788 communicating with other users, e.g. chatting arrangements for providing for computer conferences, e.g. chat rooms, to substation in data switching networks H04L12/1813; distributed application using peer-to-peer [P2P] networks H04L67/104
• CPC Class Codes: H04N 7/17318 Direct or substantially direct transmission and handling of requests

Abstract

Virtual environments are presented on displays along with multimedia programs to permit viewers to participate in a social interactive television environment. The virtual environments include avatars that are created and maintained in part using continually updated animation data that may be captured from cameras that monitor viewing areas in a plurality of sites. User input from the viewers may be processed in determining which viewers are presented in instances of the virtual environment. Continually updating the animation data results in avatars accurately depicting a viewer'"'"'s facial expressions and other characteristics. Presence data may be collected and used to determine when to capture background images from a viewing area that may later be subtracted during the capture of animation data. Speech recognition technology may be employed to provide callouts within a virtual environment.

Claims

1. A method of presenting avatars with a multimedia program, the method comprising:

generating animation input data indicative of a viewer watching the multimedia program;

generating an avatar corresponding to the viewer based at least in part on the animation input data;

displaying the avatar in a virtual environment that includes further avatars corresponding to further viewers;

subtracting background data in the animation input data to create viewer data that does not indicate a viewing area around the viewer;

responsive to detecting the viewer entering the viewing area, simulating the avatar entering the virtual environment; and

responsive to detecting the viewer leaving the viewing area, simulating the avatar leaving the virtual environment.

2. The method of claim 1, wherein the viewer data includes facial expression data indicative of facial expressions of the viewer while watching the multimedia program.

3. The method of claim 2, wherein the facial expression data indicate laughter by the viewer.

4. The method of claim 1, further comprising superimposing the virtual environment on the multimedia program for presentation on a display.

5. The method of claim 1, further comprising:

detecting a level of motion in the viewing area around the viewer;

when the level of motion is low, capturing background data from the viewing area; and

wherein, when the level of motion is high, generating the animation input data includes excluding the background data.

6. A system for presenting avatars combined with multimedia programs, comprising:

a processor configured to access memory media, wherein the memory media include program instructions executable by the processor, the program instructions including instructions to;

during a viewing session, receive animation input data harvested by monitoring a viewer;

add an avatar corresponding to the viewer in a virtual environment that includes further avatars corresponding to further viewers;

update the avatar in response to receiving the animation input data;

output display data for the virtual environment concurrently with display data for the multimedia program;

modify the animation input data by subtracting background data to result in viewer data that does not include a viewing area around the viewer; and

wherein the updating further includes when the viewer is detected entering the viewing area updating the avatar as entering the virtual environment and when the viewer is detected leaving the viewing area updating the avatar as leaving the virtual environment.

7. The system of claim 6, wherein the program instructions include instructions to:

detect a level of motion in the viewing area around the viewer;

when a low level of motion is detected, capture background data from the viewing area; and

when a high level of motion is detected, monitor the viewer to result in animation input data generated by excluding the background data.

8. The system of claim 6, wherein the viewer data include facial expression data captured from a facial expression of the viewer.

9. The system of claim 8, wherein the facial expression data represents laughter by the viewer.

10. The system of claim 6, wherein the instructions to update the avatar include instructions to:

update the avatar to perform an avatar motion based on the animation input data, wherein the avatar motion corresponds to a viewer motion detected in the animation input data.

11. A non-transitory computer readable storage media including program instructions for simultaneously presenting avatars and a multimedia program, the program instructions including instructions to:

receive a request from a viewer of the multimedia program to display a virtual environment concurrent with the multimedia program, wherein the virtual environment is configured to display a plurality of avatars for a respective plurality of viewers, including an avatar for the viewer;

responsive to receiving animation data for the viewer by monitoring the viewer, the animation data indicative of a real action of the viewer, update the avatar in the virtual environment according to the animation data, wherein an avatar action mimics the real action; and

extract the facial animation data from the animation data by subtracting background data, wherein the facial animation data substantially represents the viewer and does not represent the background area comprising a monitoring area around the viewer.

12. The storage media of claim 11, wherein the program instructions include instructions to:

send a rendering of the virtual environment to the user for display.

13. The storage media of claim 12, wherein the rendering is sent substantially simultaneously to the respective plurality of viewers.

14. The storage media of claim 11, wherein the facial animation data represent laughter by the viewer.

15. The storage media of claim 11, wherein the program instructions include instructions to:

detect a level of motion in a monitoring area around the viewer;

when a low level of motion is detected, capture background data from the monitoring area; and

when a high level of motion is detected, monitor the viewer to result in animation input data generated by excluding the background data.

16. The storage media of claim 15, wherein the program instructions include instructions to:

when the viewer is detected entering the monitoring area, update the avatar entering the virtual environment; and

when the viewer is detected leaving the monitoring area, update the avatar leaving the virtual environment.