Bringing a visual representation to life via learned input from the user

• US 9,159,151 B2
• Filed: 07/13/2009
• Issued: 10/13/2015
• Est. Priority Date: 07/13/2009
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: Perez, Kathryn Stone, Kipman, Alex, Burton, Nicholas D., Wilson, Andrew
• Primary Examiner(s): Wu, Xiao
• Assistant Examiner(s): Salvucci, Matthew D
• Application Number:
  US12/501,964
• Publication Number:
  US 20110007079A1
• Time in Patent Office: 2,283 Days
• Field of Search: 345/473, 715706-707
• US Class Current: 1/1
• CPC Class Codes: A63F 13/213 comprising photodetecting means, e.g. cameras, photodiodes or infrared cells A63F13/219, A63F13/655 take precedence
• CPC Class Codes: A63F 13/56 Computing the motion of game characters with respect to other game characters, game objects or elements of the game scene, e.g. for simulating the behaviour of a group of virtual soldiers or for path finding
• CPC Class Codes: A63F 13/58 by computing conditions of game characters, e.g. stamina, strength, motivation or energy level
• CPC Class Codes: A63F 13/67 adaptively or by learning from player actions, e.g. skill level adjustment or by storing successful combat sequences for re-use
• CPC Class Codes: G01S 17/36 with phase comparison between the received signal and the contemporaneously transmitted signal
• CPC Class Codes: G01S 17/89 for mapping or imaging
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06T 13/40 of characters, e.g. humans, animals or virtual beings
• CPC Class Codes: H04N 21/4223 Cameras H04N23/00 takes precedence
• CPC Class Codes: H04N 21/44218 Detecting physical presence or behaviour of the user, e.g. using sensors to detect if the user is leaving the room or changes his face expression during a TV program methods or arrangements for recognising human body or animal bodies or body parts G06V40/10; methods or arrangements for acquiring or recognising human faces, facial parts, facial sketches, facial expressions G06V40/16; methods or arrangements for recognising movements or behaviour G06V40/20; arrangements for identifying users in broadcast systems H04H60/45
• CPC Class Codes: H04N 21/4532 involving end-user characteristics, e.g. viewer profile, preferences monitoring of user activities for profile generation for accessing a video database G06F16/739; user profiles in network data switching protocols H04L67/306; processing of user preferences or user profiles in wireless networks H04W8/18
• CPC Class Codes: H04N 21/4667 Processing of monitored end-user data, e.g. trend analysis based on the log file of viewer selections
• CPC Class Codes: H04N 21/4781 Games

Abstract

Data captured with respect to a human may be analyzed and applied to a visual representation of a user such that the visual representation begins to reflect the behavioral characteristics of the user. For example, a system may have a capture device that captures data about the user in the physical space. The system may identify the user'"'"'s characteristics, tendencies, voice patterns, behaviors, gestures, etc. Over time, the system may learn a user'"'"'s tendencies and intelligently apply animations to the user'"'"'s avatar such that the avatar behaves and responds in accordance with the identified behaviors of the user. The animations applied to the avatar may be animations selected from a library of pre-packaged animations, or the animations may be entered and recorded by the user into the avatar'"'"'s avatar library.

Claims

1. A method for applying an animation to a visual representation of a user, the method comprising:

selecting a first animation from a plurality of stored animations, based on a first image indicative of a user presently being in a physical capture area, the first image being received from a capture device;

applying the first animation to a visual representation of the user on a display device;

updating at least one animation of the plurality of stored animations based at least in part on a characteristic of the user identified based at least in part on the first image;

in response to determining from a second image of the physical capture area that the user is not located in the physical capture area, selecting a second animation of the plurality of stored animations based at least in part on a characteristic of the user from a prior image indicative of the user in the physical capture area or another physical capture area; and

in response to determining that a second user is interacting with a computer application in which the visual representation of the user is made, applying the second animation to a visual representation of the user on the display device to animate the visual representation of the user while the user is not located in the physical capture area whereby the visual representation of the user is animated to behave in accordance with the characteristic of the user in the second animation.

2. The method of claim 1, wherein the first animation does not correspond to data presently captured by the capture device of the user present in the physical space.

3. The method of claim 1, wherein the plurality of stored animations is a pre-packaged plurality of animations, and wherein selecting the second animation from the plurality of stored animations comprises:

selecting the second animation from the pre-packaged plurality of animations.

4. The method of claim 1, further comprising:

rewriting the first animation in the plurality of stored animations to correspond to the user'"'"'s motion captured by a capture device, the rewriting comprising replacing a gesture in the first animation with a gesture of the user.

5. The method of claim 4, wherein rewriting the animation is prompted by a user'"'"'s gesture in the physical capture area.

6. The method of claim 1, wherein the visual representation of the user is animated such that the user appears to be present in the physical capture area and/or controlling the visual representation of the user.

7. A computer-readable memory, bearing computer-readable instructions that, when executed on a computer, cause the computer to perform operations comprising:

selecting a first animation from a plurality of stored animations, based on a first image indicative of a user presently being in a physical capture area, the first image received from a capture device;

applying the first animation to a visual representation of the user on a display device;

updating at least one animation of the plurality of stored animations based at least in part on a characteristic of the user identified based at least in part on the first image;

in response to determining from a second image of the physical capture area that the user is not located in the physical capture area, selecting a second animation of the plurality of stored animations based at least in part on a characteristic of the user from a prior image indicative of the user in the physical capture area or another physical capture area; and

in response to determining that a second user is interacting with a computer application in which the visual representation of the user is made, applying the second animation to a visual representation of the user on the display device to animate the visual representation of the user while the user is not located in the physical capture area whereby the visual representation of the user is animated to behave in accordance with the characteristic of the user in the second animation.

8. The computer-readable memory of claim 7, wherein the first animation does not correspond to data presently captured by the capture device of the user present in the physical space.

9. The computer-readable memory of claim 7, wherein the plurality of stored animations is a pre-packaged plurality of animations, and wherein selecting the second animation from the plurality of stored animations comprises:

selecting the second animation from the pre-packaged plurality of animations.

10. The computer-readable memory of claim 7, further bearing computer-readable instructions that, when executed on the computer, cause the computer to perform operations comprising:

rewriting the first animation in the plurality of stored animations to correspond to the user'"'"'s motion captured by a capture device.

11. The computer-readable memory of claim 10, wherein rewriting the animation is prompted by a user'"'"'s gesture in the physical capture area.

12. The computer-readable memory of claim 7, wherein the visual representation of the user is animated such that the user appears to be present in the physical capture area and/or controlling the visual representation of the user.

13. A system, comprising:

a processor; and

a memory communicatively coupled to the processor when the system is operational, the memory bearing processor-executable instructions that, when executed on the processor, cause the system at least to;

select a first animation from a plurality of stored animations, based on a first image indicative of a user presently being in a physical capture area, the first image received from a capture device;

apply the first animation to a visual representation of the user on a display device;

updating at least one animation of the plurality of stored animations based at least in part on a characteristic of the user identified based at least in part on the first image;

in response to determining from a second image of the physical capture area that the user is not located in the physical capture area, select a second animation of the plurality of stored animations based at least in part on a characteristic of the user from a prior image indicative of the user in the physical capture area or another physical capture area; and

in response to determining that a second user is interacting with a computer application in which the visual representation of the user is made, apply the second animation to a visual representation of the user on the display device to animate the visual representation of the user while the user is not located in the physical capture area whereby the visual representation of the user is animated to behave in accordance with the characteristic of the user in the second animation.

14. The system of claim 13, wherein the first animation does not correspond to data presently captured by the capture device of the user present in the physical space.

15. The system of claim 13, wherein the plurality of stored animations is a pre-packaged plurality of animations, and wherein the instructions that, when executed on the processor, cause the system at least to select the second animation from the plurality of stored animations further cause the system at least to:

select the second animation from the pre-packaged plurality of animations.

16. The system of claim 13, further bearing processor-executable instructions that, when executed on the processor, cause the system at least to:

rewrite the first animation in the plurality of stored animations to correspond to the user'"'"'s motion captured by a capture device.

17. The system of claim 16, wherein rewriting the animation is prompted by a user'"'"'s gesture in the physical capture area.

18. The system of claim 13, wherein the visual representation of the user is animated such that the user appears to be present in the physical capture area and/or controlling the visual representation of the user.