Avatars in social interactive television

• US 7,953,255 B2
• Filed: 05/01/2008
• Issued: 05/31/2011
• Est. Priority Date: 05/01/2008
• Status: Active Grant

• Current Assignee: AT&T Knowledge Ventures, L.P. (AT&T, Inc.)
• Original Assignee: AT&T Intellectual Property I LP (AT&T, Inc.)
• Inventors: Amento, Brian Scott, Harrison, Christopher, Stead, Larry
• Primary Examiner(s): PERUNGAVOOR, SATHYANARAYA V
• Application Number:
  US12/113,704
• Publication Number:
  US 20090276802A1
• Time in Patent Office: 1,125 Days
• Field of Search: 382/118
• US Class Current: 382/118
• CPC Class Codes: G06T 13/40 of characters, e.g. humans, animals or virtual beings
• CPC Class Codes: H04N 21/23412 for generating or manipulating the scene composition of objects, e.g. MPEG-4 objects
• CPC Class Codes: H04N 21/42203 sound input device, e.g. microphone
• CPC Class Codes: H04N 21/4223 Cameras H04N23/00 takes precedence
• CPC Class Codes: H04N 21/44012 involving rendering scenes according to scene graphs, e.g. MPEG-4 scene graphs
• CPC Class Codes: H04N 21/4788 communicating with other users, e.g. chatting arrangements for providing for computer conferences, e.g. chat rooms, to substation in data switching networks H04L12/1813; distributed application using peer-to-peer [P2P] networks H04L67/104
• CPC Class Codes: H04N 7/17318 Direct or substantially direct transmission and handling of requests

Abstract

Virtual environments are presented on displays along with multimedia programs to permit viewers to participate in a social interactive television environment. The virtual environments include avatars that are created and maintained in part using continually updated animation data that may be captured from cameras that monitor viewing areas in a plurality of sites. User input from the viewers may be processed in determining which viewers are presented in instances of the virtual environment. Continually updating the animation data results in avatars accurately depicting a viewer'"'"'s facial expressions and other characteristics. Presence data may be collected and used to determine when to capture background images from a viewing area that may later be subtracted during the capture of animation data. Speech recognition technology may be employed to provide callouts within a virtual environment.

Claims

1. A method of presenting avatars with a multimedia program, the method comprising:

continually monitoring a viewer to result in animation input data;

displaying an avatar corresponding to the viewer in a virtual environment that includes further avatars corresponding to further viewers;

continually updating the avatar in response to the animation input data;

subtracting background data to result in animation input data that corresponds substantially to the monitored viewer and not a viewing area around the monitored viewer;

detecting a level of motion in the viewing area around the monitored viewer;

in response to a low level of motion, capturing background data from the viewing area; and

in response to a high level of motion, continually monitoring the viewer to result in animation input data achieved in part by excluding the captured background data.

2. The method of claim 1, wherein continually monitoring the viewer includes capturing facial expressions, wherein the animation input data includes facial expression data.

3. The method of claim 2, further comprising:

detecting laughter from the viewer; and

wherein the animation input data includes laughter data.

4. The method of claim 1, wherein the virtual environment is superimposed on the multimedia program for presentation on a display.

5. The method of claim 1, wherein the virtual environment is placed within a perimeter outside the presentation of a multimedia program on a display.

6. The method of claim 1, further comprising:

detecting the viewer entering the viewing area; and

simulating the avatar entering the virtual environment in response to the detecting the viewer entering.

7. The method of claim 6, further comprising:

detecting the viewer leaving the viewing area; and

simulating the avatar leaving the virtual environment in response to the detecting the viewer leaving.

8. The method of claim 1, further comprising:

receiving updated animation data corresponding to the further viewers; and

updating the further avatars in response to the updated animation data.

9. The method of claim 8, wherein receiving updated animation data occurs continually during a viewing session.

10. A computer program product stored on one or more computer readable media for simultaneously presenting avatars and a multimedia program, the computer program product having instructions operable for:

during a viewing session, receiving animation input data harvested by monitoring a viewer;

displaying an avatar corresponding to the viewer in a virtual environment that includes further avatars corresponding to further viewers;

during the animation session, continually updating the avatar in response to the animation input data;

subtracting background data to result in animation input data that corresponds substantially to the monitored viewer and not a viewing area around the monitored viewer;

detecting the viewer entering the viewing area; and

simulating the avatar entering the virtual environment in response to the detecting the viewer entering.

11. The computer program product of claim 10, wherein the received animation input data includes captured facial expression data.

12. The computer program product of claim 11, further comprising instructions operable for:

detecting laughter from the viewer, and wherein the animation input data includes laughter data.

13. The computer program product of claim 10, wherein the virtual environment is superimposed on the multimedia program for presentation on a display.

14. The computer program product of claim 10, wherein the virtual environment is placed within a perimeter outside the presentation of a multimedia program on a display.

15. The computer program product of claim 14, wherein receiving updated animation data occurs continually during a viewing session.

16. The computer program product of claim 10, further comprising instructions operable for:

detecting a first level of motion in the viewing area for the monitored person;

in response to the first level of motion, capturing background data from the viewing area;

detecting a second level of motion in the viewing area to determine a viewing session is occurring, wherein continually monitoring the person to result in animation input data is accomplished in part by excluding the captured background data.

17. The computer program product of claim 10, further comprising instructions operable for:

detecting the viewer leaving the viewing area; and

simulating the avatar leaving the virtual environment in response to the detecting the viewer leaving.

18. The computer program product of claim 10, further comprising instructions operable for:

receiving updated animation data corresponding to the further viewers; and

updating the further avatars in response to the updated animation data.