Visual representation expression based on player expression

• US 9,519,989 B2
• Filed: 03/04/2013
• Issued: 12/13/2016
• Est. Priority Date: 07/09/2009
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: Perez, Kathryn Stone, Kipman, Alex, Burton, Nicholas D., Wilson, Andrew
• Primary Examiner(s): Beck, Leron
• Application Number:
  US13/784,480
• Publication Number:
  US 20130187929A1
• Time in Patent Office: 1,380 Days
• Field of Search: 348/77
• US Class Current: 1/1
• CPC Class Codes: A63F 13/213 comprising photodetecting means, e.g. cameras, photodiodes or infrared cells A63F13/219, A63F13/655 take precedence
• CPC Class Codes: A63F 13/42 by mapping the input signals into game commands, e.g. mapping the displacement of a stylus on a touch screen to the steering angle of a virtual vehicle
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06T 13/80 2D [Two Dimensional] animation, e.g. using sprites

Abstract

Using facial recognition and gesture/body posture recognition techniques, a system can naturally convey the emotions and attitudes of a user via the user'"'"'s visual representation. Techniques may comprise customizing a visual representation of a user based on detectable characteristics, deducting a user'"'"'s temperament from the detectable characteristics, and applying attributes indicative of the temperament to the visual representation in real time. Techniques may also comprise processing changes to the user'"'"'s characteristics in the physical space and updating the visual representation in real time. For example, the system may track a user'"'"'s facial expressions and body movements to identify a temperament and then apply attributes indicative of that temperament to the visual representation. Thus, a visual representation of a user, which may be, for example, an avatar or fanciful character, can reflect the user'"'"'s expressions and moods in real time.

Claims

1. A method for applying attributes indicative of a user'"'"'s temperament to a visual representation of the user, the method comprising:

rendering a visual representation of a user;

receiving, at a computing system, first data indicative of the user in a physical space, the first data comprising image data indicating a user body gesture while playing an application game and audible data indicating utterances made by the user while playing the application game;

analyzing, at the computing system, the first data indicative of the user in the physical space and data indicative of an application game result identifying success or failure of the user playing the application game to deduce an emotional temperament, the analyzing comprising querying a database to identify the emotional temperament based upon the user body gesture, the utterances made by the user, the application game result identifying success or failure of the user playing the application game, and past tendencies of the user'"'"'s temperament;

determining, at the computing system, a pre-existing animation to display based on the deduced emotional temperament, the pre-existing animation having a motion, pose or facial expression not performed by the user as indicated in the first data; and

displaying, at the computing system, the pre-existing animation for the visual representation on a display device.

2. The method of claim 1, wherein the visual representation is at least one of an avatar, a character, a cursor, or a stock model.

3. The method of claim 1, wherein analyzing the first data of the user in the physical space comprises:

analyzing a user'"'"'s characteristics, a user'"'"'s physical features, a user'"'"'s behavior, a user'"'"'s speech pattern, a user'"'"'s voice, a gesture, or history data.

4. The method of claim 1, wherein the first data indicative of the user in the physical space is representative of at least one of a user'"'"'s characteristics in the physical space.

5. The method of claim 1, further comprising:

applying at least one of a user'"'"'s characteristics to the visual representation.

6. The method of claim 1, wherein receiving first data indicative of the user in the physical space comprises:

receiving data via a capture device.

7. The method of claim 1, wherein the deduced emotional temperament comprises generally negative, generally positive, ambivalent, bored, happy, sad, frustrated, excited, or angry.

8. The method of claim 1, further comprising:

receiving a second data indicative of the user in the physical space;

determining a change to the user'"'"'s emotional temperament based on a difference between the second data and the first data;

determining a second pre-existing animation based on the change to the user'"'"'s emotional temperament, the second pre-existing animation having a motion, pose or facial expression not expressed in the second data; and

displaying the second pre-existing animation for the visual representation on the display device.

9. The method of claim 1, further comprising:

selecting the pre-existing animation from a plurality of pre-existing animations that correspond to the deduced emotional temperament.

10. A system for applying attributes indicative of a user'"'"'s temperament to a visual representation, the system comprising:

a processor; and

memory communicatively coupled with the processor, the memory bearing processor-executable instructions that, upon execution on the processor, cause the system to at least;

render a visual representation of a user;

receive data indicative of the user in a physical space, the first data comprising image data indicating a user body gesture while playing an application game and audible data indicating utterances made by the user while playing the application game;

analyze the data indicative of the user in the physical space and data indicative of an application game result identifying success or failure of the user playing the application game to deduce an emotional temperament, the analyzing comprising querying a database to identify the emotional temperament based upon the user body gesture, the utterances made by the user, the application game result identifying success or failure of the user playing the application game, and past tendencies of the user'"'"'s temperament;

determine a pre-existing animation to display based on the deduced emotional temperament, the pre-existing animation having a motion, pose or facial expression not performed by the user as indicated in the first data; and

display the pre-existing animation for the visual representation on a display device.

11. The system of claim 10, further comprising processor-executable instructions that cause the system to at least:

display the pre-existing animation for the visual representation on a display device in real time with respect to the receipt of data indicative of the user in the physical space.

12. The system of claim 11,

the memory having stored thereon data correlating characteristics to a particular emotional temperament.

13. A computer-readable storage device that is not only a propagating signal comprising computer-executable instructions that, upon execution on a computer, cause the computer to perform operations comprising:

rendering a visual representation of a user;

receiving data indicative of the user in a physical space, the first data comprising image data indicating a user body gesture while playing an application game and audible data indicating utterances made by the user while playing the application game;

analyzing the first data indicative of the user in the physical space and data indicative of an application game result identifying success or failure of the user playing the application game to deduce an emotional temperament, the analyzing comprising querying a database to identify the emotional temperament based upon the user body gesture, the utterances made by the user, the application game result identifying success or failure of the user playing the application game, and past tendencies of the user'"'"'s temperament;

determining a pre-existing animation of to display based on the deduced emotional temperament, the pre-existing animation having a motion, pose or facial expression not performed by the user as indicated in the first data; and

displaying the pre-existing animation for the visual representation on a display device.

14. The computer-readable storage device of claim 13, wherein displaying the pre-existing animation for the visual representation on a display device is performed in real time with respect to receiving data indicative of the user in the physical space.

15. The computer-readable storage device of claim 13, wherein the visual representation is at least one of an avatar, a character, a cursor, or a stock model.

16. The computer-readable storage device of claim 13, comprising instructions that, upon execution on the computer, cause the computer to perform further operations comprising:

applying at least one of a user'"'"'s characteristics to the visual representation, the data comprising data representative of the user'"'"'s characteristics.

17. The computer-readable storage device of claim 13, wherein receiving data indicative of the user in the physical space comprises:

receiving data via a capture device.

18. The computer-readable storage device of claim 13, wherein the deduced emotional temperament comprises at least one of generally negative, generally positive, ambivalent, bored, happy, sad, frustrated, excited, or angry.

19. The computer-readable storage device of claim 13, comprising instructions that, upon execution on the computer, cause the computer to perform further operations comprising:

providing a filter representing an emotional temperament identity gesture, the filter comprising base information about the emotional temperament identity gesture; and

applying the filter to the first data and determining an output from the base information about the emotional temperament identity gesture.

20. The computer-readable storage device of claim 13, comprising instructions that, upon execution on the computer, cause the computer to perform further operations comprising:

selecting the pre-existing animation from a plurality of pre-existing animations that correspond to the deduced temperament.