Interacting with user interface via avatar

• US 9,292,083 B2
• Filed: 05/29/2014
• Issued: 03/22/2016
• Est. Priority Date: 06/11/2010
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: Evertt, Jeffrey, Bennett, Darren, Vance, Dylan, Galloway, David, Markovic, Relja, Latta, Stephen, Santos, Oscar Omar Garza, Geisner, Kevin, Deaguero, Joel
• Primary Examiner(s): Du, Haixia
• Application Number:
  US14/290,749
• Publication Number:
  US 20140267311A1
• Time in Patent Office: 663 Days
• US Class Current: 1/1
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06T 13/40 of characters, e.g. humans, animals or virtual beings

Abstract

Embodiments are disclosed that relate to interacting with a user interface via feedback provided by an avatar. One embodiment provides a method comprising receiving depth data, locating a person in the depth data, and mapping a physical space in front of the person to a screen space of a display device. The method further comprises forming an image of an avatar representing the person, outputting to a display an image of a user interface comprising an interactive user interface control, and outputting to the display device the image of the avatar such that the avatar faces the user interface control. The method further comprises detecting a motion of the person via the depth data, forming an animated representation of the avatar interacting with the user interface control based upon the motion of the person, and outputting the animated representation of the avatar interacting with the control.

Claims

1. In a computing device, a method of presenting a user interface, the method comprising:

receiving depth data from a depth-sensing camera;

locating a plurality of persons in the depth data;

determining a selected person of the plurality of persons to be in control of the user interface relative to one or more other persons of the plurality of persons from one or more of a posture and a gesture of the selected person in the depth data relative to the one or more other persons of the plurality of persons, the one or more of the posture and the gesture comprising one or more characteristics indicative of an intent of the selected person to assume control of the user interface;

forming an image of an avatar representing the selected person;

outputting to the display device an image of a user interface, the user interface comprising an interactive user interface control;

outputting to the display device the image of the avatar such that the avatar appears to face the user interface control;

detecting a motion of the selected person via the depth data; and

outputting to the display device an animated representation of the avatar interacting with the user interface control based upon the motion of the selected person.

2. The method of claim 1, wherein outputting the image of the avatar comprises outputting the image of the avatar such that the avatar appears to face the selected person.

3. The method of claim 1, wherein detecting the motion of the selected person comprises detecting a motion of a limb of the selected person, and wherein forming an animated representation of the avatar comprises forming an animated representation of a limb of the avatar moving relative to the user interface control.

4. The method of claim 1, further comprising mapping the physical space in front of the selected person by performing skeletal tracking of the selected person, determining an orientation and size of the selected person from the skeletal tracking of the selected person, and determining an estimated extent of limb motion of the selected person from a perspective of the depth-sensing camera.

5. The method of claim 1, wherein outputting to the display device the image of the avatar comprises outputting images of a plurality of avatars such that a selected avatar representing the selected person is visually distinguished from other avatars representing other persons.

6. The method of claim 5, wherein the selected avatar is visually distinguished by one or more of size, location in the image, and posture relative to the other avatars.

7. The method of claim 5, further comprising detecting an input configured to cause a change of person in control of the user interface, and in response, changing which avatar is visually distinguished from the other avatars, wherein the input configured to cause the change of person in control of the user interface comprises one or more of a gesture and a posture.

8. A computing device, comprising:

a processor; and

memory comprising instructions executable by the processor to;

receive depth data from a depth-sensing camera;

locate a plurality of persons in the depth data;

determine a selected person of the plurality of persons to be in control of the user interface relative to one or more other persons of the plurality of persons from one or more of a posture and a gesture of the selected person in the depth data relative to the one or more other persons of the plurality of persons, the one or more of the posture and the gesture comprising one or more characteristics indicative of an intent of the selected person to assume control of the user interface;

form an image of an avatar representing the selected person;

output to the display device an image of a user interface, the user interface comprising an interactive user interface control;

output to the display device the image of the avatar such that the avatar appears to face the user interface control;

detect a motion of the selected person via the depth data; and

output to the display device an animated representation of the avatar interacting with the user interface control based upon the motion of the selected person.

9. The computing device of claim 8, wherein the instructions are executable to output the image of the avatar such that the avatar appears to face the selected person.

10. The computing device of claim 8, wherein the instructions are executable to detect the motion of the selected person by detecting a motion of a limb of the selected person, and to form an animated representation of the avatar by forming an animated representation of a limb of the avatar moving relative to the user interface control.

11. The computing device of claim 8, wherein the instructions are further executable to map the physical space in front of the selected person by performing skeletal tracking of the selected person, determine an orientation and size of the selected person from the skeletal tracking of the selected person, and determine an estimated extent of limb motion of the selected person from a perspective of the depth-sensing camera.

12. The computing device of claim 8, wherein the instructions are executable to output to the display device the image of the avatar by outputting images of a plurality of avatars such that a selected avatar representing the selected person is visually distinguished from other avatars representing other persons.

13. The computing device of claim 12, wherein the selected avatar is visually distinguished by one or more of size, location in the image, and posture relative to the other avatars.

14. The computing device of claim 8, wherein the instructions are further executable to detect an input configured to cause a change of person in control of the user interface, and in response, change which avatar is visually distinguished from the other avatars, wherein the input configured to cause the change of person in control of the user interface comprises one or more of a gesture and a posture.

15. A storage device, comprising instructions executable by a computing device to:

receive depth data from a depth-sensing camera;

locate a plurality of persons in the depth data;

determine a selected person of the plurality of persons to be in control of the user interface relative to one or more other persons of the plurality of persons from one or more of a posture and a gesture of the selected person in the depth data relative to the one or more other persons of the plurality of persons, the one or more of the posture and the gesture comprising one or more characteristics indicative of an intent of the selected person to assume control of the user interface;

form an image of an avatar representing the selected person;

output to the display device an image of a user interface, the user interface comprising an interactive user interface control;

output to the display device the image of the avatar such that the avatar appears to face the user interface control;

detect a motion of the selected person via the depth data; and

output to the display device an animated representation of the avatar interacting with the user interface control based upon the motion of the selected person.

16. The storage device of claim 15, wherein the instructions are executable to output the image of the avatar such that the avatar appears to face the selected person.

17. The storage device of claim 15, wherein the instructions are executable to detect the motion of the selected person by detecting a motion of a limb of the selected person, and to form an animated representation of the avatar by forming an animated representation of a limb of the avatar moving relative to the user interface control.

18. The storage device of claim 15, wherein the instructions are further executable to map the physical space in front of the selected person by performing skeletal tracking of the selected person, determine an orientation and size of the selected person from the skeletal tracking of the selected person, and determine an estimated extent of limb motion of the selected person from a perspective of the depth-sensing camera.

19. The storage device of claim 15, wherein the instructions are executable to output to the display device the image of the avatar by outputting images of a plurality of avatars such that a selected avatar representing the selected person is visually distinguished from other avatars representing other persons.

20. The storage device of claim 15, wherein the instructions are further executable to detect an input configured to cause a change of person in control of the user interface, and in response, change which avatar is visually distinguished from the other avatars, wherein the input configured to cause the change of person in control of the user interface comprises one or more of a gesture and a posture.