Avatars in social interactive television

• US 8,818,054 B2
• Filed: 10/30/2012
• Issued: 08/26/2014
• Est. Priority Date: 05/01/2008
• Status: Active Grant

• Current Assignee: AT&T Intellectual Property I LP (AT&T, Inc.)
• Original Assignee: AT&T Intellectual Property I LP (AT&T, Inc.)
• Inventors: Amento, Brian Scott, Harrison, Christopher, Stead, Larry
• Primary Examiner(s): STREGE, JOHN B
• Application Number:
  US13/664,117
• Publication Number:
  US 20130057556A1
• Time in Patent Office: 665 Days
• Field of Search: 382/118, 382/173
• US Class Current: 382/118
• CPC Class Codes: G06T 13/40 of characters, e.g. humans, animals or virtual beings
• CPC Class Codes: H04N 21/23412 for generating or manipulating the scene composition of objects, e.g. MPEG-4 objects
• CPC Class Codes: H04N 21/42203 sound input device, e.g. microphone
• CPC Class Codes: H04N 21/4223 Cameras H04N23/00 takes precedence
• CPC Class Codes: H04N 21/44012 involving rendering scenes according to scene graphs, e.g. MPEG-4 scene graphs
• CPC Class Codes: H04N 21/4788 communicating with other users, e.g. chatting arrangements for providing for computer conferences, e.g. chat rooms, to substation in data switching networks H04L12/1813; distributed application using peer-to-peer [P2P] networks H04L67/104
• CPC Class Codes: H04N 7/17318 Direct or substantially direct transmission and handling of requests

Abstract

Virtual environments are presented on displays along with multimedia programs to permit viewers to participate in a social interactive television environment. The virtual environments include avatars that are created and maintained in part using continually updated animation data that may be captured from cameras that monitor viewing areas in a plurality of sites. User input from the viewers may be processed in determining which viewers are presented in instances of the virtual environment. Continually updating the animation data results in avatars accurately depicting a viewer'"'"'s facial expressions and other characteristics. Presence data may be collected and used to determine when to capture background images from a viewing area that may later be subtracted during the capture of animation data. Speech recognition technology may be employed to provide callouts within a virtual environment.

Claims

1. A method, comprising:

processing background data indicative of a viewing area and animation data indicative of a viewer in the viewing area watching a multimedia program to generate a viewer avatar indicative of the viewer watching the multimedia program with a background area removed;

displaying the multimedia program, the viewer avatar, and further avatars corresponding to further viewers; and

responsive to detecting the viewer exiting the viewing area, terminating the displaying of the viewer avatar.

2. The method of claim 1, wherein the viewer avatar reflects facial expression data, indicative of changing facial expressions of the viewer.

3. The method of claim 2, wherein the facial expression data indicate laughter by the viewer.

4. The method of claim 1, wherein displaying the viewer avatar and the further avatars includes superimposing at least one of the viewer avatar and the further avatars over the multimedia program.

5. The method of claim 1, further comprising:

capturing images of the viewing area responsive to determining a level of motion in the viewing area below a predetermined threshold.

6. The method of claim 1, further comprising:

responsive to detecting a second viewer in the viewing area, displaying a second viewer avatar adjacent the viewer avatar.

7. A system for presenting multimedia programs, the system comprising:

a processor having access to a computer readable medium, wherein the computer readable medium includes program instructions executable by the processor, to perform operations including;

processing background data indicative of a viewing area and animation data indicative of a viewer in the viewing area watching the multimedia program to generate a viewer avatar indicative of the viewer watching the multimedia program with a background area removed;

displaying the multimedia program, the viewer avatar, and further avatars corresponding to further viewers; and

responsive to detecting the viewer exiting the viewing area, terminating the displaying of the viewer avatar.

8. The system of claim 7, wherein the operations include:

responding to detecting a level of motion in the viewing area around the viewer below a first predetermined threshold level of motion by capturing background data from the viewing area.

9. The system of claim 7, wherein the operations include:

responsive to determining that the viewer is entering the viewing area, causing the displaying of the avatar entering the virtual environment.

10. The system of claim 7, wherein the viewer data includes time varying facial expression data indicative of changing facial expressions of the viewer.

11. The system of claim 10, wherein the facial expression data includes data indicative of laughter by the viewer.

12. The system of claim 7, wherein the operations include:

representing avatar motion based on the animation data, wherein the avatar motion corresponds to a viewer motion detected in the animation data.

16. The storage media of claim 7, wherein the animation data includes facial animation data.

17. The storage media of claim 16, wherein the facial animation data indicates variations in facial expressions of the viewer.

18. The storage media of claim 16, wherein the operations include:

extracting the facial animation data from the animation data by subtracting background data, wherein the facial animation data substantially represents the viewer and does not represent the background area comprising a monitoring area around the viewer.

13. A non-transitory computer readable storage media including program instructions executable by a processor, to perform operations, comprising:

processing background data indicative of a viewing area and animation data indicative of a viewer in the viewing area watching the multimedia program to generate a viewer avatar indicative of the viewer watching the multimedia program with a background area removed;

displaying the multimedia program, the viewer avatar, and further avatars corresponding to further viewers; and

responding to detecting the viewer exiting the viewing area by terminating the displaying of the viewer avatar.

14. The storage media of claim 13, wherein the operations include:

sending a rendering of the viewer avatar and the further avatars to the user for display.

15. The storage media of claim 14, wherein sending the rendering includes sending the rendering substantially simultaneously to the viewer avatar and the further viewers.