MIXED REALITY COLLABORATION

• US 20180063205A1
• Filed: 08/25/2017
• Published: 03/01/2018
• Est. Priority Date: 08/30/2016
• Status: Active Application

• Current Assignee: Augre Mixed Reality Technologies, LLC
• Original Assignee: Augre Mixed Reality Technologies, LLC
• Inventors: French, Christian James
• Application Number:
  US15/686,975
• Publication Number:
  US 20180063205A1
• Time in Patent Office: Days
• CPC Class Codes: G06F 3/04815 Interaction with a metaphor-based environment or interaction object displayed as three-dimensional, e.g. changing the user viewpoint with respect to the environment or object
• CPC Class Codes: G06F 3/04842 Selection of displayed objects or displayed text elements G06F3/0482 takes precedence
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: G06T 2219/024 Multi-user, collaborative environment
• CPC Class Codes: H04L 65/1069 Session establishment or de-establishment
• CPC Class Codes: H04L 65/1073 Registration or de-registration
• CPC Class Codes: H04L 65/1089 by adding media; by removing media
• CPC Class Codes: H04L 65/1093 by adding participants; by removing participants
• CPC Class Codes: H04L 65/403 Arrangements for multi-party communication, e.g. for conferences data switching systems for conference H04L12/18; arrangements for connecting several subscribers to a common circuit, i.e. affording conference facilities H04M3/56; television conferencing systems H04N7/15
• CPC Class Codes: H04L 65/4038 with floor control

Abstract

Mixed reality collaboration applications and a mixed reality collaboration platform providing mixed reality collaboration are described. The platform can include a data resource with supported device information, registered user information, and session data stored on the data resource. Two or more devices (e.g., a first user device and a second user device) with different operating systems can register with the platform. The platform can store registration information, such as user device information and user information, received from the two or more devices in the data resource as part of the registered user information. The platform can receive, from the second user device, session data in a format compatible with the second user device operating system. The platform can then access the supported device information and communicate the session data to the first user device according to the API calls for the first user device.

Claims

1. A system for performing mixed reality communication between multiple users, comprising:

a processing system;

a storage system operatively coupled with the processing system;

a data resource operatively coupled with the processing system;

supported device information stored on the data resource, the supported device information indicating devices and operating systems that the system can support and their corresponding application programming interface (API) calls;

registered user information stored on the data resource, the registered user information including user identifiers and device information; and

instructions for performing mixed reality communication between multiple users, stored on the storage system, that when executed by the processing system, direct the processing system to at least;

in response to receiving registration information from a first user device, store the registration information from the first user device in the data resource as part of the registered user information, wherein the registration information includes at least first user device information and first user information;

in response to receiving registration information from a second user device, store the registration information from the second user device in the data resource as part of the registered user information, wherein the registration information includes at least second user device information and second user information, and wherein the first user device and the second user device have different operating systems;

receive, from the second user device, session data in a format compatible with the second user device operating system; and

access the supported device information and communicate the session data to the first user device according to the API calls for the first user device.

2. The system of claim 1, wherein the instructions for performing mixed reality communication between multiple users further direct the processing system to:

create a collaboration session for both the first user device and the second user device; and

link both the first user device and the second user device to the collaboration session.

3. The system of claim 2, wherein the instructions for performing mixed reality communication between multiple users further direct the processing system to:

send an application library manifest to the first user device;

receive, from the first user device, a user selection of an application of the application library, the application being associated with the user of the second user device; and

initiate the link with the second user device, wherein initiating the link comprises sending a request to establish a connection to the second user device.

4. The system of claim 1, wherein the instructions that direct the processing system to access the supported device information and communicate the session data, direct the processing system to at least:

determine if the format of the session data is compatible with the first user device operating system;

if the format of the session data is not compatible with the first user device operating system, convert the session data to a format that is compatible with the first user device operating system; and

send the session data to the first user device.

5. The system of claim 1, wherein the session data is three-dimensional map data, wherein the three-dimensional map data defines a virtual environment associated with a user of the second user device.

6. The system of claim 1, wherein the session data is first manipulation data.

7. The system of claim 1, wherein the instructions for performing mixed reality communication between multiple users further direct the processing system to:

receive, from the first user device, second manipulation data in a format compatible with the first user device operating system; and

access the supported device information and communicate the second manipulation data to the second user device according to the API calls for the second user device.

8. The system of claim 7, wherein the instructions that direct the processing system to access the supported device information and communicate the second manipulation data, direct the processing system to at least:

determine if the format of the second manipulation data is compatible with the second user device operating system;

if the format of the second manipulation data is not compatible with the second user device operating system, convert the second manipulation data to a format that is compatible with the second user device operating system; and

send the second manipulation data to the second user device.

9. The system of claim 1, wherein the session data includes one or more of three-dimensional map data, environment data, geographic location data, sound data, video data, asset data, manipulation data, connection status data, time data, progress data, and preference data.

10. The system of claim 1, wherein the instructions for performing mixed reality communication between multiple users further direct the processing system to:

receive, from a third user device, third manipulation data in a format compatible with the third user device operating system; and

access the supported device information and communicate the third manipulation data to the first user device according to the API calls for the first user device and the second user device according to the API calls for the second user device.

11. A method for performing mixed reality communication between multiple users, the method comprising:

in response to receiving registration information from a first user device, storing the registration information from the first user device in a data resource as part of registered user information, wherein the registration information includes at least first user device information and first user information, and wherein the data resource comprises supported device information and the registered user information, the supported device information indicating devices and operating systems that a system for performing mixed reality communication between multiple users can support and their corresponding application programming interface (API) calls, and the registered user information including user identifiers and device information;

in response to receiving registration information from a second user device, storing the registration information from the second user device in the data resource as part of the registered user information, wherein the registration information includes at least second user device information and second user information, and wherein the first user device and the second user device have different operating systems;

receiving, from the second user device, session data in a format compatible with the second user device operating system; and

accessing the supported device information and communicating the session data to the first user device according to the API calls for the first user device.

12. The method of claim 11, further comprising:

creating a collaboration session for both the first user device and the second user device; and

linking both the first user device and the second user device to the collaboration session.

13. The method of claim 11, further comprising:

sending an application library manifest to the first user device;

receiving, from the first user device, a user selection of an application of the application library, the application being associated with the user of the second user device; and

initiating the link with the second user device, wherein initiating the link comprises sending a request to establish a connection to the second user device.

14. The method of claim 11, wherein the accessing the supported device information and communicating the session data further comprises:

determining if the format of the session data is compatible with the first user device operating system;

if the format of the session data is not compatible with the first user device operating system, converting the session data to a format that is compatible with the first user device operating system; and

sending the session data to the first user device.

15. The method of claim 11, wherein the session data is three-dimensional map data, wherein the three-dimensional map data defines a virtual environment associated with a user of the second user device.

16. The method of claim 11, wherein the session data is first manipulation data.

17. The method of claim 11, further comprising:

receiving, from the first user device, second manipulation data in a format compatible with the first user device operating system; and

accessing the supported device information and communicating the second manipulation data to the second user device according to the API calls for the second user device.

18. The method of claim 17, wherein the accessing the supported device information and communicating the second manipulation data further comprises:

determining if the format of the second manipulation data is compatible with the second user device operating system;

if the format of the second manipulation data is not compatible with the second user device operating system, converting the second manipulation data to a format that is compatible with the second user device operating system; and

sending the second manipulation data to the second user device.

19. The system of claim 11, wherein the session data includes one or more of three-dimensional map data, environment data, geographic location data, sound data, video data, asset data, manipulation data, connection status data, time data, progress data, and preference data.

20. The system of claim 11, further comprising tracking the session data.