RECOGNITION SYSTEM FOR SHARING INFORMATION

• US 20110302293A1
• Filed: 06/02/2010
• Published: 12/08/2011
• Est. Priority Date: 06/02/2010
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Buban, Garret J.
• Granted Patent: US 8,803,888 B2
• Time in Patent Office: Days
• US Class Current: 709/224
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06T 2207/10028 Range image; Depth image; 3D point clouds
• CPC Class Codes: G06T 2207/30196 Human being; Person
• CPC Class Codes: G06T 7/251 involving models
• CPC Class Codes: G06V 40/20 Movements or behaviour, e.g. gesture recognition recognition of facial expressions G06V40/16
• CPC Class Codes: G06V 40/23 Recognition of whole body movements, e.g. for sport training
• CPC Class Codes: H04L 65/4061 Push-to services, e.g. push-to-talk or push-to-video
• CPC Class Codes: H04L 67/06 specially adapted for file transfer, e.g. file transfer protocol [FTP]
• CPC Class Codes: H04L 67/131 Protocols for games, networked simulations or virtual reality

Abstract

A system and method for sharing information between users based on recognition of the users and their associated processing devices in a scene. Interactions can be physical, verbal or a combination of physical and verbal gestures. Movements of the user and other users are tracked and interactions detected between them. User processing devices are connected by detecting users within view of a capture device, the capture device detecting motion tracking data for the user, such as a skeletal model. Information sharing may be controlled by the processing devices directly, by an intermediary server, or by a combination of the processing device and an intermediary server.

Claims

1. A computer implemented method for sharing information between users, comprising:

monitoring a scene;

detecting at least one user in the scene and creating a skeletal model for the at least one user;

associating the at least one user in the scene with a processing device;

tracking movements of the at least one user in a scene using the skeletal model of the user;

providing skeletal data identifying the user to an information sharing application; and

providing interaction information between skeletal models of at least two users to the sharing application.

2. The computer implemented method of claim 1 wherein detecting includes detecting depth information for the target and recognition information for the user.

3. The computer implemented method of claim 1 wherein the method is performed by a server having coupled thereto a capture device, the capture device having a field of view comprising the scene.

4. The computer implemented method of claim 3 wherein the information sharing application is provided on the processing device.

5. The computer implemented method of claim 3 wherein the information sharing application is provided on the server.

6. The computer implemented method of claim 1 wherein detecting includes detecting at least a first user and a second user, the first user having a first skeletal model and the second user having a second skeletal model, and the method further includes detecting an interaction between the first user and the second user by detecting a relationship between the first skeletal model and the second skeletal model.

7. The computer implemented method of claim 6 wherein detecting the interaction includes detecting a sound associated with the relationship between the first user and the second user temporally related to the relationship.

8. The computer implemented method of claim 1 wherein detecting includes detecting at least a first user and a second user, the first user having a first skeletal model and the second user having a second skeletal model, and the method further includes providing tracking information for the first skeletal model and the second skeletal model to at least the first processing device.

9. The computer implemented method of claim 1 wherein associated includes:

receiving a request from a processing device for an identification of a user in the scene;

providing skeletal data and recognition information associated with the skeletal model to the processing device responsive to the request;

receiving a request for ownership of the skeletal model; and

linking the processing device and the skeletal model.

10. A method for sharing information between two users, each user having a processing device, comprising:

detecting the presence of a detection and tracking server having a view of a scene including each of the users, the detection and tracking server creating a skeletal model of each of the users in the scene;

associating one of the at least two users with a processing device by requesting a skeletal model identifying the one of the at least two users in the scene;

determining a physical interaction between the two users based on an interaction of the skeletal model with another skeletal model in the scene; and

sharing information between the users based on the interaction.

11. The method of claim 10 wherein associating includes receiving skeletal model tracking information for each of the users in the scene and wherein the recognition information comprises image data associated with each of the skeletal models, and matching the image data with known image data for a user.

12. The method of claim 11 wherein associating further includes requesting ownership of a skeletal model, and wherein determining a physical interaction includes:

receiving tracking information on skeletal models of each of the users in the scene;

detecting an interaction between the users comprising one of;

a gesture indicating a relationship between the users, a proximal interaction between two skeletal models, a proximal interaction between two skeletal models accompanied by a sound, or an audible command.

13. The method of claim 12 wherein the method further includes associating one of a predefined sharing action or a sharing action defined by a type of interaction detected.

14. The method of claim 13 wherein sharing information includes creating a connection between the processing device associated with the user and a second processing devices associated with a user associated with a second skeletal model.

15. A computer readable medium including instructions causing a computer to perform a process, the process comprising:

monitoring a scene using depth and recognition information provided by a capture device;

tracking movements of targets comprising users in a scene using a skeletal model of each target in the scene;

associating a user in the scene with a skeletal model and a processing device;

detecting interactions between one or more users in the scene by reference to at least movements of the skeletal models of the users in relation to other users; and

upon detecting an interaction, sharing information between users in the scene.

16. The computer readable medium of claim 15 wherein monitoring is performed by a first computer and tracking movements is performed by the processing device associated with the user.

17. The computer readable medium of claim 15 wherein sharing information between users comprises executing a predefined action.

18. The computer readable medium of claim 15 wherein the interaction comprises detecting a relationship between a first skeletal model and a second skeletal model.

19. The computer readable medium of claim 15 wherein associating includes:

receiving a request from the processing device for an identification of a user in the scene;

providing skeletal data and recognition information associated with the skeletal model to the processing device responsive to the request;

receiving a request for ownership of the skeletal model; and

linking the processing device and the skeletal model.

20. The computer readable medium of claim 17 wherein executing a predefined action includes sharing user contact information automatically upon detecting the interaction.