Recognizing user intent in motion capture system

• US 9,195,305 B2
• Filed: 11/08/2012
• Issued: 11/24/2015
• Est. Priority Date: 01/15/2010
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: Markovic, Relja, Latta, Stephen G, Geisner, Kevin A, Steed, Jonathan T, Bennett, Darren A, Vance, Amos D
• Primary Examiner(s): Baderman, Scott
• Assistant Examiner(s): Mrabi, Hassan
• Application Number:
  US13/671,875
• Publication Number:
  US 20130074002A1
• Time in Patent Office: 1,111 Days
• Field of Search: 715/781
• US Class Current: 1/1
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883

Abstract

Techniques for facilitating interaction with an application in a motion capture system allow a person to easily begin interacting without manual setup. A depth camera system tracks a person in physical space and determines a probabilistic measure of the person'"'"'s intent to engage of disengage with the application based on location, stance and movement. Absolute location in a field of view of the depth camera, and location relative to another person, can be evaluated. Stance can include facing a depth camera, indicating a willingness to interact. Movements can include moving toward or away from a central area in the physical space, walking through the field of view, and movements which occur while standing generally in one location, such as moving one'"'"'s arms around, gesturing, or shifting weight from one foot to another.

Claims

1. Tangible computer readable memory having computer readable software embodied thereon for programming a processor to perform a method for recognizing an intent of a person to engage with an application in a motion capture system, the method comprising:

receiving images of a field of view of the motion capture system;

based on the images, distinguishing a person'"'"'s body;

determining a bounding shape of the person'"'"'s body, the bounding shape is represented by a point within the bounding shape;

tracking a location of the person'"'"'s body in the field of view based on the point within the bounding shape relative to a location in the field of view in which an intent to engage with the application is most probable;

based on the tracking determining a probabilistic measure of an intent by the person to engage with the application;

based on the probabilistic measure of the intent by the person to engage with the application, determining that the person intends to engage with the application;

in response to determining that the person intends to engage with the application, allowing the person to engage with the application by automatically associating a profile and an avatar with the person in the application, and displaying the avatar in a virtual space on a display; and

updating the display by controlling the avatar as the person engages with the application by moving the person'"'"'s body.

2. The tangible computer readable memory of claim 1, wherein:

the probabilistic measure of the intent by the person to engage with the application is based on a stance of the person'"'"'s body in the field of view and how long the stance is maintained.

3. The tangible computer readable memory of claim 1, wherein the method performed further comprises:

determining movement of the person'"'"'s body in the field of view relative to an optical axis of a depth camera in the motion capture system, wherein the probabilistic measure of the intent by the person to engage with the application considers that is based on movement of the person'"'"'s body perpendicular to the optical axis is an indication that the person does not intend to engage with the application.

4. The tangible computer readable memory of claim 1, wherein:

the probabilistic measure of the intent by the person to engage with the application results in a higher probability of intent to engage when the person is within a specified distance of another person who is engaging with the application.

5. The tangible computer readable memory of claim 1, wherein the method performed further comprises:

based on the distinguishing, determining a probabilistic measure of an intent by the person to disengage with the application; and

based on the probabilistic measure of the intent by the person to disengage with the application, determining that the person intends to disengage with the application, in response to which the profile and the avatar are automatically stored indexed to an identifier of the person.

6. The tangible computer readable memory of claim 5, wherein:

the probabilistic measure of the intent by the person to disengage with the application provides a higher probability of the intent by the person to disengage with the application when the person'"'"'s body makes a movement which indicates the intent by the person to disengage with the application.

7. The tangible computer readable memory of claim 6, wherein:

the movement which indicates the intent by the person to disengage with the application comprises the person walking away from the display.

8. The tangible computer readable memory of claim 6, wherein:

the movement which indicates the intent by the person to disengage with the application comprises the person turning their back on the display while standing still.

9. The tangible computer readable memory of claim 6, wherein:

the movement which indicates the intent by the person to disengage with the application comprises the person walking away from a location in the field of view in which the intent to engage with the application is most probable.

10. The tangible computer readable memory of claim 6, wherein:

the movement which indicates the intent by the person to disengage with the application comprises the person walking toward a boundary of the field of view and then out of the field of view.

11. The tangible computer readable memory of claim 1, wherein:

the location in the field of view in which an intent to engage with the application is most probable is in a first zone of the field of view; and

a center of the first zone is along an optical axis of a depth camera in the motion capture system.

12. The tangible computer readable memory of claim 1, wherein:

the location in the field of view in which an intent to engage with the application is most probable is in a first zone of the field of view; and

the probabilistic measure is based on a distance between a center of the first zone and the point within the bounding shape.

13. The tangible computer readable memory of claim 1, wherein:

the probabilistic measure is based on one area of multiple areas in the field of view in which the point within the bounding shape is located; and

the multiple areas include a first area which an intent to engage with the application is most probable, a second area in which an intent to engage with the application is moderately probable and a third area in which an intent to engage with the application is less probable than in the second area.

14. The tangible computer readable memory of claim 13, wherein:

the second area surrounds the first area and the third area is outside the second area.

15. A motion capture system, comprising:

a depth camera system to receive images of a field of view;

a display; and

a processor to communicate with the depth camera system and the display, the processor is configured to execute instructions to;

implement an application in the motion capture system,

based on the images, distinguish a person'"'"'s body and, in response, determine a probabilistic measure of an intent by the person to engage with the application,

based on the probabilistic measure of the intent by the person to engage with the application, determine that the person intends to engage with the application and, in response, an avatar of on the display in an active state, where the avatar moves based on movements of the person and has an appearance consistent with the active state,

determine that the person'"'"'s body is no longer distinguished in the images;

when a time since the person'"'"'s body is no longer distinguished in the images is less than a first threshold, allow the avatar to remain in the active state, where the avatar has the appearance consistent with the active state,

when a time since the person'"'"'s body is no longer distinguished in the images is more than the first threshold, provide the avatar in an inactive state in which the avatar does not move based on movements of the person and in which the avatar has an appearance consistent with the inactive state, and

determine that the person'"'"'s body is again distinguished in the images and, in response;

(a) if a time in which the avatar is in the inactive state is less than a time out period, provide the avatar again in the active state, where the avatar again moves based on movements of the person and has the appearance consistent with the active state, without determining that the person intends to engage with the application based on a location and a stance of the person in the field of view, and

(b) if a time in which the avatar is in the inactive state is more than the time out period, require a determination that the person intends to engage with the application based on the location and the stance of the person in the field of view as a condition to provide the avatar again in the active state.

16. The motion capture system of claim 15, wherein:

the probabilistic measure of the intent by the person to engage with the application uses a different probability for different zones of the field of view.

17. The motion capture system of claim 15, wherein:

the probabilistic measure of the intent by the person to engage with the application is based on a location and stance of the person'"'"'s body in the field of view.

18. A processor-implemented method for recognizing an intent of a person to engage with an application in a motion capture system, comprising the processor-implemented steps of:

receiving images of a field of view of the motion capture system;

based on the images, distinguishing a person'"'"'s body, determining that the person intends to engage with the application and allowing engagement of the person with the application by automatically associating a profile and an avatar with the person in the application, and displaying the avatar in a virtual space on a display;

based on the images, determining that the person'"'"'s body has not been detected for a first period of time and determining whether the person left the field of view;

in response to the determining that the person'"'"'s body has not been detected for the first period of time, suspending the engagement of the person with the application;

based on the images, while the engagement of the person with the application is suspended, determining that the person'"'"'s body is detected again in the field of view;

when the person'"'"'s body is detected again in the field of view and when the person did not leave the field of view, allowing the person to re-engage with the application without re-determining that the person intends to engage with the application based on a location and a stance of the person in the field of view; and

when the person'"'"'s body is detected again in the field of view and when the person left the field of view;

determining how long the engagement of the person with the application has been suspended, and if the engagement of the person with the application has been suspended for more than a specified period of time, re-determining that the person intends to engage with the application based on the location and the stance of the person in the field of view before allowing the person to re-engage with the application.

19. The processor-implemented method of claim 18, wherein:

when the person'"'"'s body is detected again in the field of view and when the person left the field of view, and if the engagement of the person with the application has not been suspended for more than the specified period of time, allowing the person to re-engage with the application without re-determining that the person intends to engage with the application.

20. The processor-implemented method of claim 18, wherein:

the allowing engagement of the person with the application comprises displaying the avatar in the virtual space on the display and moving the avatar based on movements of the person'"'"'s body; and

the suspending the engagement of the person with the application comprises not displaying the avatar in the virtual space on the display and not moving the avatar based on movements of the person'"'"'s body.