Visual representation expression based on player expression

• US 8,390,680 B2
• Filed: 07/09/2009
• Issued: 03/05/2013
• Est. Priority Date: 07/09/2009
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Perez, Kathryn Stone, Kipman, Alex, Burton, Nicholas D., Wilson, Andrew
• Primary Examiner(s): Czekaj, David
• Assistant Examiner(s): Beck, Leron
• Application Number:
  US12/500,251
• Publication Number:
  US 20110007142A1
• Time in Patent Office: 1,335 Days
• Field of Search: 348/77, 345/474, 382/236, 340/573.1
• US Class Current: 348/77
• CPC Class Codes: A63F 13/213 comprising photodetecting means, e.g. cameras, photodiodes or infrared cells A63F13/219, A63F13/655 take precedence
• CPC Class Codes: A63F 13/42 by mapping the input signals into game commands, e.g. mapping the displacement of a stylus on a touch screen to the steering angle of a virtual vehicle
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06T 13/80 2D [Two Dimensional] animation, e.g. using sprites

Abstract

Using facial recognition and gesture/body posture recognition techniques, a system can naturally convey the emotions and attitudes of a user via the user'"'"'s visual representation. Techniques may comprise customizing a visual representation of a user based on detectable characteristics, deducting a user'"'"'s temperament from the detectable characteristics, and applying attributes indicative of the temperament to the visual representation in real time. Techniques may also comprise processing changes to the user'"'"'s characteristics in the physical space and updating the visual representation in real time. For example, the system may track a user'"'"'s facial expressions and body movements to identify a temperament and then apply attributes indicative of that temperament to the visual representation. Thus, a visual representation of a user, such as an avatar or fanciful character, can reflect the user'"'"'s expressions and moods in real time.

Claims

1. A method for applying attributes indicative of a user'"'"'s temperament to a visual representation, the method comprising:

rendering the visual representation of a user;

receiving first data indicative of the user in a physical space;

analyzing the first data indicative of the user in the physical space to deduce the user'"'"'s temperament;

determining a pre-existing animation of the visual representation to display based on the user'"'"'s temperament;

displaying the pre-existing animation for the visual representation on a display device;

determining a state of an executing application when the first data indicative of the user in the physical space was received;

receiving second data indicative of the user in the physical space;

determining that the executing application has returned to the state;

determining to display the pre-existing animation independent of the second data, because the application has returned to the state; and

displaying the pre-existing animation on a display device.

2. The method of claim 1, wherein the visual representation is at least one of an avatar, a character, a cursor, or a stock model.

3. The method of claim 1, wherein analyzing the first data of the user in the user'"'"'s temperament comprises:

analyzing a user'"'"'s characteristics, a user'"'"'s physical features, a user'"'"'s behavior, a user'"'"'s speech pattern, a user'"'"'s voice, a gesture, or history data.

4. The method of claim 1, wherein the first data indicative of the user in the physical space is representative of at least one of a user'"'"'s characteristics in the physical space.

5. The method of claim 1, further comprising:

applying at least one of a user'"'"'s characteristics to the visual representation.

6. The method of claim 1, wherein receiving first data indicative of the user in the physical space comprises:

receiving data via a capture device.

7. The method of claim 1, wherein the user'"'"'s temperament comprises at least one of generally negative, generally positive, ambivalent, bored, happy, sad, frustrated, excited, or angry.

8. The method of claim 1, further comprising:

receiving a second data indicative of the user in the physical space;

determining a change to the user'"'"'s temperament based on a difference between the second data and the first data;

determining a second pre-existing animation for the visual representation based on the change to the user'"'"'s temperament; and

displaying the second pre-existing animation for the visual representation on a display device.

9. The method of claim 1, further comprising:

selecting the pre-existing animation from a plurality of pre-existing animations that correspond to the user'"'"'s temperament.

21. The method of claim 1, wherein determining a pre-existing animation of the visual representation to display based on the user'"'"'s temperament comprises:

determining a pre-existing animation of the visual representation to display based on the user'"'"'s temperament, the pre-existing animation differing from a motion or pose of the user captured in the first data indicative of the user in the physical space.

10. A system for applying attributes indicative of a user'"'"'s temperament to a visual representation, the system comprising:

a processor; and

a memory communicatively coupled to the processor when the system is operational, the memory bearing processor-executable instructions that, when executed on the processor, cause the system to at least;

render the visual representation of the user;

receive data indicative of the user in a physical space

analyze the data indicative of the user in the physical space to deduce the user'"'"'s temperament;

determine a pre-existing animation of the visual representation to display based on the user'"'"'s temperament;

display the pre-existing animation for the visual representation on a display device;

determine a state of an executing application when the first data indicative of the user in the physical space was received;

receive second data indicative of the user in the physical space;

determine that the executing application has returned to the state;

determine to display the pre-existing animation independent of the second data, because the application has returned to the state; and

display the pre-existing animation on a display device.

11. The system of claim 10, wherein the instructions that, when executed on the processor, cause the system to at least display the pre-existing animation for the visual representation on a display device, further cause the system to at least:

display the pre-existing animation for the visual representation on a display device in real time with respect to the receipt of the data indicative of the user in the physical space.

12. The system of claim 10, further comprising:

a memory that stores a table that provides characteristics that correlate to a particular temperament.

13. A tangible computer-readable storage device for applying attributes indicative of a temperament to a visual representation, bearing computer-executable instructions that, when executed on a computer, cause the computer to perform operations comprising:

rendering the visual representation of a user;

receiving data indicative of the user in a physical space;

analyzing the data indicative of the user in the physical space to deduce the user'"'"'s temperament;

determining a pre-existing animation of the visual representation to display based on the user'"'"'s temperament;

displaying the pre-existing animation for the visual representation on a display device;

determining a state of an executing application when the first data indicative of the user in the physical space was received;

receiving second data indicative of the user in the physical space;

determining that the executing application has returned to the state;

determining to display the pre-existing animation independent of the second data, because the application has returned to the state; and

displaying the pre-existing animation on a display device.

14. The computer-readable storage medium of claim 13, wherein displaying the pre-existing animation for the visual representation on a display device is performed in real time with respect to the receipt of the data indicative of the user in the physical space.

15. The computer-readable storage medium of claim 13, wherein the visual representation is at least one of an avatar, a character, a cursor, or a stock model.

16. The computer-readable storage medium of claim 13, further bearing computer-executable instructions that, when executed on the computer, cause the computer to perform operations comprising:

applying at least one of a user'"'"'s characteristics to the visual representation, the data comprising data representative of the user'"'"'s characteristics.

17. The computer-readable storage medium of claim 13, wherein receiving data indicative of the user in the physical space comprises:

receiving data via a capture device.

18. The computer-readable storage medium of claim 13, wherein the user'"'"'s temperament comprises at least one of generally negative, generally positive, ambivalent, bored, happy, sad, frustrated, excited, or angry.

19. The computer-readable storage medium of claim 13, further bearing computer-executable instructions that, when executed on the computer, cause the computer to perform operations comprising:

providing a filter representing the temperament identity gesture, the filter comprising base information about the temperament identity gesture; and

applying the filter to the data and determining an output from the base information about the temperament identity gesture.

20. The computer-readable storage medium of claim 13, further comprising:

selecting the pre-existing animation from a plurality of pre-existing animations that correspond to the user'"'"'s temperament.