Cooperative federation of digital devices via proxemics and device micro-mobility

• US 9,774,653 B2
• Filed: 03/04/2016
• Issued: 09/26/2017
• Est. Priority Date: 03/14/2013
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: Hinckley, Kenneth P., Marquardt, Nicolai
• Primary Examiner(s): Mejia, Anthony
• Application Number:
  US15/062,036
• Publication Number:
  US 20160261666A1
• Time in Patent Office: 571 Days
• Field of Search: 709219
• CPC Class Codes: H04L 67/02 based on web technology, e.g. hypertext transfer protocol [HTTP]
• CPC Class Codes: H04L 69/16 Implementation or adaptation of Internet protocol [IP], of transmission control protocol [TCP] or of user datagram protocol [UDP]

Abstract

The subject disclosure is directed towards co-located collaboration/data sharing that is based upon detecting the proxemics of people and/or the proxemics of devices. A federation of devices is established based upon proxemics, such as when the users have entered into a formation based upon distance between them and orientation. User devices may share content with other devices in the federation based upon micro-mobility actions performed on the devices, e.g., tilting and/or otherwise interacting with a sending device.

Claims

1. A system comprising:

a memory;

a server; and

a processor configured to execute instructions in the memory to perform the following;

process image data to determine a formation based upon proxemics of users having digital devices;

maintain federation data corresponding to;

an identity of the digital devices in the formation;

a distance between users associated with the digital devices; and

a relative body orientation of the users; and

communicate the federation data to the digital devices in the formation to establish a federation between two or more digital devices in the formation based on the federation data, the federation between the digital devices in the formation enabling the two or more digital devices in the formation to communicate content with one another based upon one or more micro-mobility actions detected by at least one of the two or more digital devices in the formation.

2. The system of claim 1, wherein the processor is further configured to dynamically update the federation data and the two or more digital devices in the formation upon a change to the formation.

3. The system of claim 1, wherein the image data comprises depth data, and wherein the server includes a processing pipeline that determines the users within the depth data and the relative body orientation of the users relative to one another.

4. The system of claim 3, wherein the relative body orientation of the users relative to one another comprises a face to face orientation, a side-to-side orientation, or a corner-to-corner orientation.

5. The system of claim 1, wherein the server comprises a central server coupled to one or more cameras that provide the image data.

6. The system of claim 1, wherein at least one of the digital devices in the federation is associated with the federation by proxemics but not associated with any one of the users.

7. The system of claim 1, wherein at least one of the digital devices comprises a digital whiteboard or other display mechanism configured to render content.

8. The system of claim 1, wherein the processor is further configured to exclude a user from the federation based on the relative body orientation of the users.

9. A method comprising:

determining proxemics among a plurality of devices, the plurality of digital devices comprising one or more of the following;

at least two users having digital devices, at least one user and at least two digital devices, and at least two digital devices;

maintaining federation data indicative of a distance between users associated with the plurality of digital devices and a relative body orientation of the users;

establishing a federation between a set of the plurality of digital devices based upon the federation data, the federation enabling the set of the plurality of digital devices to communicate data between one another using micro-mobility actions, the micro-mobility actions comprising one or more of the following;

device tilting, touch gesture, pen gesture, and multi-touch gesture.

10. The method of claim 9, wherein maintaining the federation data further comprises updating the federation data upon a change to the federation.

11. The method of claim 9, further comprising associating detected users with the devices.

12. The method of claim 9, further comprising excluding a user from the federation based on the relative body orientation of the users.

13. The method of claim 9, wherein the relative body orientation comprises a head orientation.

14. The method of claim 9, further comprising detecting a tilting of a device in the federation prior to sharing content of the device.

15. The method of claim 14 further comprising, taking one of a plurality of sharing actions based upon an amount of tilt detected.

16. One or more computer-readable storage devices having computer-executable instructions, which when executed on at least one processor perform operations comprising:

receiving content at a recipient device from a sending device, the sending device and the recipient device being participants in a federation established by relative body orientations between a user of the sending device and a user of the recipient device; and

rendering the content on the recipient device in partial, increasing amounts over time until the content is fully rendered, the rendering in partial, increasing amounts occurring from one direction based upon a location of the sending device relative to the recipient device to have the content appear to have come from a direction of the sending device relative to the recipient device.

17. The one or more computer-readable storage devices of claim 16 having further computer-executable instructions comprising:

detecting, at the sending device, one of a first micro-mobility action or a second micro-mobility action; and

sharing a full screen of content based on the first micro-mobility action or sharing less than a full screen of content based upon the second micro-mobility action.

18. The one or more computer-readable storage devices of claim 17, wherein the first micro-mobility action and the second micro-mobility action comprises one or more of the following:

device tilting, touch gesture, pen gesture, and multi-touch gesture.

19. The one or more computer-readable storage devices of claim 17, wherein a difference between the first micro-mobility action and the second micro-mobility action is an amount of tilt detected by the sending device.

20. The one or more computer-readable storage devices of claim 17 having further computer-executable instructions comprising excluding a device from the federation based on a relative body orientation of a user associated with the device.