Recognition system for sharing information

• US 8,803,888 B2
• Filed: 06/02/2010
• Issued: 08/12/2014
• Est. Priority Date: 06/02/2010
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Buban, Garret J.
• Primary Examiner(s): Tung, Kee M
• Assistant Examiner(s): Du, Haixia
• Application Number:
  US12/792,549
• Publication Number:
  US 20110302293A1
• Time in Patent Office: 1,532 Days
• Field of Search: 345/473, 715/972, 715/977, 715/861
• US Class Current: 345/473
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06T 2207/10028 Range image; Depth image; 3D point clouds
• CPC Class Codes: G06T 2207/30196 Human being; Person
• CPC Class Codes: G06T 7/251 involving models
• CPC Class Codes: G06V 40/20 Movements or behaviour, e.g. gesture recognition recognition of facial expressions G06V40/16
• CPC Class Codes: G06V 40/23 Recognition of whole body movements, e.g. for sport training
• CPC Class Codes: H04L 65/4061 Push-to services, e.g. push-to-talk or push-to-video
• CPC Class Codes: H04L 67/06 specially adapted for file transfer, e.g. file transfer protocol [FTP]
• CPC Class Codes: H04L 67/131 Protocols for games, networked simulations or virtual reality

Abstract

A system and method for sharing information between users based on recognition of the users and their associated processing devices in a scene. Interactions can be physical, verbal or a combination of physical and verbal gestures. Movements of the user and other users are tracked and interactions detected between them. User processing devices are connected by detecting users within view of a capture device, the capture device detecting motion tracking data for the user, such as a skeletal model. Information sharing may be controlled by the processing devices directly, by an intermediary server, or by a combination of the processing device and an intermediary server.

Claims

1. A computer implemented method for sharing information between users, comprising:

monitoring a scene using a capture device including a processor having a field of view including the scene;

detecting a first user in the scene using the capture device and creating a skeletal model for the first user;

associating the first user in the scene with a first user processing device;

tracking movements of the first user in a scene using the skeletal model of the first user;

detecting a second user in the scene using the capture device and creating a skeletal model for the second user;

associating the second user in the scene with a second user processing device;

tracking movements of the second user in a scene using the skeletal model of the second user;

providing skeletal data identifying the first user and the second user to an information sharing application;

detecting interactions between skeletal models of the first user and the second user to the information sharing application; and

responsive to detecting an interaction, sharing information stored in the first processing device with the second processing device.

2. The computer implemented method of claim 1 wherein detecting includes detecting depth information for each user and recognition information for each user.

3. The computer implemented method of claim 1 wherein the method is performed by a server having coupled thereto the capture device.

4. The computer implemented method of claim 3 wherein the information sharing application is provided on the first user processing device.

5. The computer implemented method of claim 3 wherein the information sharing application is provided on the server.

6. The computer implemented method of claim 1 further including:

transferring user information stored on the first user processing device to the second user processing device responsive to the detecting.

7. The computer implemented method of claim 6 wherein detecting the interaction includes detecting a sound associated with the relationship between the first user and the second user temporally related to the relationship.

8. The computer implemented method of claim 1 wherein the method further includes providing tracking information for the skeletal model of the first user and the skeletal model of the second user to at least the first user processing device.

9. The computer implemented method of claim 1 wherein associating includes:

receiving a request from the first processing device for an identification of an unknown user in the scene;

providing skeletal data and recognition information associated with a skeletal model of the unknown to the processing device responsive to the request;

receiving a request for ownership of the skeletal model of the unknown user; and

linking the processing device and the skeletal model of the unknown user.

10. A method for sharing information between at least two users, each user having a processing device, comprising:

detecting a detection and tracking server proximate to a scene, the server coupled to a capture device having a view of the scene including each of the users, the detection and tracking server creating a skeletal model of each of the users in the scene;

associating a first of the at least two users with a first user processing device by requesting a first skeletal model identifying the first of the at least two users in the scene;

associating a second of the at least two users with a second user processing device by requesting a second skeletal model identifying the second of the at least two users in the scene;

determining a physical interaction between the at least two users based on an interaction between the first skeletal model with the second skeletal model in the scene via the capture device; and

responsive to the interaction determining a predefined sharing action and sharing user information stored on the first user processing device and the second user processing device based on the interaction and the predefined action.

11. The method of claim 10 wherein associating includes receiving skeletal model tracking information for each of the users in the scene and wherein the tracking information comprises image data associated with each of the skeletal models, and matching the image data with known image data for a user.

12. The method of claim 11 wherein associating further includes requesting ownership of a skeletal model, and wherein determining a physical interaction includes:

receiving tracking information on skeletal models of each of the users in the scene;

detecting an interaction between the users comprising one of;

a gesture indicating a relationship between the users, a proximal interaction between two skeletal models, a proximal interaction between two skeletal models accompanied by a sound, or an audible command.

13. The method of claim 12 wherein the method further includes associating one of a predefined sharing action or a sharing action defined by a type of interaction detected.

14. The method of claim 13 wherein sharing information includes creating a connection between the first processing device associated with the first user and a second processing device associated with the second user.

15. A computer storage device including instructions causing a computer to perform a process, the process comprising:

monitoring a scene using depth and recognition information provided by a capture device having an associated processor, the scene within the field of view of the capture device;

tracking movements of targets comprising at least a first user and a second user in a scene using a skeletal model for each target in the scene;

associating a first user in the scene with a first skeletal model and a first user processing device;

associating a second user in the scene with a second skeletal model and a second user processing device;

detecting interactions between the first user and the second user in the scene by reference to movements of the first and second skeletal models of the users; and

upon detecting an interaction, sharing information stored on the the first user processing device with the second user processing device by transferring the information from the first user device to the second user device.

16. The computer storage device of claim 15 wherein monitoring is performed by a first computer and tracking movements is performed by the first user processing device associated with the first user.

17. The computer storage device of claim 15 wherein sharing information between users comprises executing a predefined action.

18. The computer storage device of claim 17 wherein executing a predefined action includes sharing user contact information automatically upon detecting the interaction.

19. The computer storage device of claim 15 wherein the interaction comprises detecting a relationship between a first skeletal model and a second skeletal model.

20. The computer storage device of claim 15 wherein associating includes:

receiving a request from the first user processing device for an identification of a new user in the scene;

providing skeletal data and recognition information associated with the skeletal model to the first user processing device responsive to the request;

receiving a request for ownership of the skeletal model; and

linking the processing device and the skeletal model.