Avatars in social interactive television

• US 8,311,295 B2
• Filed: 11/30/2011
• Issued: 11/13/2012
• Est. Priority Date: 05/01/2008
• Status: Active Grant

• Current Assignee: AT&T Intellectual Property I LP (AT&T, Inc.)
• Original Assignee: AT&T Intellectual Property I LP (AT&T, Inc.)
• Inventors: Amento, Brian Scott, Harrison, Christopher, Stead, Larry
• Primary Examiner(s): STREGE, JOHN B
• Application Number:
  US13/308,335
• Publication Number:
  US 20120075312A1
• Time in Patent Office: 349 Days
• Field of Search: 382/118, 382/173
• US Class Current: 382/118
• CPC Class Codes: G06T 13/40 of characters, e.g. humans, animals or virtual beings
• CPC Class Codes: H04N 21/23412 for generating or manipulating the scene composition of objects, e.g. MPEG-4 objects
• CPC Class Codes: H04N 21/42203 sound input device, e.g. microphone
• CPC Class Codes: H04N 21/4223 Cameras H04N23/00 takes precedence
• CPC Class Codes: H04N 21/44012 involving rendering scenes according to scene graphs, e.g. MPEG-4 scene graphs
• CPC Class Codes: H04N 21/4788 communicating with other users, e.g. chatting arrangements for providing for computer conferences, e.g. chat rooms, to substation in data switching networks H04L12/1813; distributed application using peer-to-peer [P2P] networks H04L67/104
• CPC Class Codes: H04N 7/17318 Direct or substantially direct transmission and handling of requests

Abstract

Virtual environments are presented on displays along with multimedia programs to permit viewers to participate in a social interactive television environment. The virtual environments include avatars that are created and maintained in part using continually updated animation data that may be captured from cameras that monitor viewing areas in a plurality of sites. User input from the viewers may be processed in determining which viewers are presented in instances of the virtual environment. Continually updating the animation data results in avatars accurately depicting a viewer'"'"'s facial expressions and other characteristics. Presence data may be collected and used to determine when to capture background images from a viewing area that may later be subtracted during the capture of animation data. Speech recognition technology may be employed to provide callouts within a virtual environment.

Claims

1. A method of presenting avatars with a multimedia program, the method comprising:

processing background data indicative of a viewing area and animation input data indicative of a viewer in the viewing area watching the multimedia program to generate viewer data indicative of the viewer watching the multimedia program with a background area removed;

displaying the multimedia program, a viewer avatar derived from the viewer data, and further avatars corresponding to further viewers and

responsive to detecting the viewer exiting the viewing area, terminating the displaying of the viewer avatar.

2. The method of claim 1, wherein the viewer data includes facial expression data, indicative of changing facial expressions of the viewer, the avatar changes to reflect the changing facial expressions.

3. The method of claim 2, wherein the facial expression data indicate laughter by the viewer.

4. The method of claim 1, wherein displaying the viewer avatar and the further avatars includes superimposing the viewer avatar and the further avatars over the multimedia program.

5. The method of claim 1, further comprising:

acquiring the background data, wherein acquiring the background data includes capturing images of the viewing area responsive to determining a level of motion in the viewing area below predetermined threshold.

6. The method of claim 1, further comprising:

responsive to detecting a second viewer in the viewing, displaying a second viewer avatar adjacent the viewer avatar.

7. A system for presenting avatars combined with multimedia programs, comprising:

a processor configured to access a computer readable medium, wherein the computer readable medium includes program instructions executable by the processor, the program instructions including instructions for;

processing background data indicative of a viewing area and animation input data indicative of a viewer in the viewing area watching the multimedia program to generate viewer data indicative of the viewer watching the multimedia program with a background area removed;

displaying the multimedia program, a viewer avatar derived from the viewer data, and further avatars corresponding to further viewers and

responsive to detecting the viewer exiting the viewing area, terminating the displaying of the viewer avatar.

8. The system of claim 7, wherein the program instructions include instructions for:

responding to detecting a level of motion in the viewing area around the viewer below a first predetermined threshold level of motion by capturing the background data from the viewing area.

9. The system of claim 7, wherein the program instructions include instructions for:

responding to determining that the viewer is entering the viewing area causing the displaying of the avatar entering the virtual environment.

10. The system of claim 7, wherein the viewer data includes time varying facial expression data indicative of changing facial expressions of the viewer.

11. The system of claim 10, wherein the facial expression data includes data indicative of laughter by the viewer.

12. The system of claim 7, wherein the program instructions include instructions for:

update the avatar to perform an avatar motion based on the animation input data, wherein the avatar motion corresponds to a viewer motion detected in the animation input data.

13. A non-transitory computer readable storage media including program instructions for simultaneously presenting avatars and a multimedia program, the program instructions including instructions for:

processing background data indicative of a viewing area and animation data indicative of a viewer in the viewing area watching the multimedia program to generate viewer data indicative of the viewer watching the multimedia program with the a background area removed;

displaying the multimedia program, a viewer avatar derived from the viewer data, and further avatars corresponding to further viewers and

responding to detecting the viewer exiting the viewing area by terminating the displaying of the viewer avatar.

14. The storage media of claim 13, wherein the program instructions include instructions for:

sending a rendering of the viewer avatar and the further avatars to the user for display.

15. The storage media of claim 14, wherein sending the rendering includes sending the rendering substantially simultaneously to the viewer avatar and the further viewers.

16. The storage media of claim 13, wherein the animation data indicates facial animation data.

17. The storage media of claim 16, wherein the facial animation data indicates variations in facial expressions of the viewer.

18. The storage media of claim 16, wherein the instructions for processing the background data and the animation data include instructions for:

extract the facial animation data from the animation data by subtracting background data, wherein the facial animation data substantially represents the viewer and does not represent the background area comprising a monitoring area around the viewer.