Distributed simultaneous localization and mapping (SLAM) in virtual, augmented, and mixed reality (xR) applications

• US 10,482,677 B1
• Filed: 11/20/2018
• Issued: 11/19/2019
• Est. Priority Date: 11/20/2018
• Status: Active Grant

• Current Assignee: Dell Products LP (Dell Technologies Inc.)
• Original Assignee: Dell Products LP (Dell Technologies Inc.)
• Inventors: Iyer, Vivek Viswanathan, Gatson, Michael S.
• Primary Examiner(s): Xiao, Ke
• Assistant Examiner(s): Imperial, Jed-Justin
• Application Number:
  US16/197,064
• Time in Patent Office: 364 Days
• CPC Class Codes: G02B 27/017 Head mounted
• CPC Class Codes: G02B 27/0172 characterised by optical features
• CPC Class Codes: G06F 17/16 Matrix or vector computation , e.g. matrix-matrix or matrix-vector multiplication, matrix factorization matrix transposition G06F7/78
• CPC Class Codes: G06F 3/012 Head tracking input arrangements
• CPC Class Codes: G06F 3/04815 Interaction with a metaphor-based environment or interaction object displayed as three-dimensional, e.g. changing the user viewpoint with respect to the environment or object
• CPC Class Codes: G06T 15/20 Perspective computation
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: G06T 2219/024 Multi-user, collaborative environment

Abstract

Embodiments of systems and methods for distributed Simultaneous Localization and Mapping (SLAM) in virtual, augmented, and mixed reality (xR) applications are described. An Information Handling System (IHS) may include a processor and a memory coupled to the processor, the memory having program instructions that, upon execution by the processor, cause the IHS to: receive first SLAM data obtained by a first HMD worn by a first user during execution of an xR application; receive second SLAM data obtained by a second HMD worn by a second user co-located with the first user; apply a transform matrix to the second SLAM data to produce transformed second SLAM data in a common coordinate system with respect to the first SLAM data; and render at least a partial map for display by the first HMD using a combination of landmarks identified in the first SLAM data and in the transformed second SLAM data.

Claims

1. An Information Handling System (IHS), comprising:

a processor; and

a memory coupled to the processor, the memory having program instructions stored thereon that, upon execution by the processor, cause the IHS to;

receive first Simultaneous Localization and Mapping (SLAM) data obtained by a first Head-Mounted Device (HMD) worn by a first user during execution of a virtual, augmented, or mixed reality (xR) application;

receive second SLAM data obtained by a second HMD worn by a second user co-located with the first user;

apply a transform matrix to the second SLAM data to produce transformed second SLAM data in a common coordinate system with respect to the first SLAM data;

render at least a partial map for display by the first HMD using a combination of landmarks identified in the first SLAM data and in the transformed second SLAM data; and

in response to a determination that the processor'"'"'s utilization is above a threshold level, reduce an amount of second SLAM data relative to an amount of first SLAM data processed.

2. The IHS of claim 1, wherein the first SLAM data comprises landmark data found in a Region of Interest (ROI) within an infrared (IR) or near-IR (NIR) frame captured via a first camera mounted on the first HMD, and wherein the second SLAM data comprises landmark data found in another ROI within another IR or NIR frame captured via a second camera mounted on the second HMD.

3. The IHS of claim 1, wherein reducing the second amount of SLAM data comprises reducing a size of an ROI, per frame, received from the second HMD.

4. The IHS of claim 1, wherein reducing the second amount of SLAM data comprises reducing a number of frames, per unit time, received from the second HMD.

5. The IHS of claim 1, wherein using the combination of landmarks further comprises resolving two or more landmarks that overlap in space between a first ROI in the first SLAM data and a second ROI in the transformed second SLAM data.

6. The IHS of claim 1, wherein the program instructions, upon execution, further cause the IHS to select the second HMD among a plurality of other HMDs as being closest in space to the first HMD, at the given location.

7. The IHS of claim 1, wherein the program instructions, upon execution, further cause the IHS to select the second HMD among a plurality of other HMDs as being closest the first HMD'"'"'s predicted trajectory.

8. The IHS of claim 1, wherein the program instructions, upon execution, further cause the IHS to request that another IHS coupled to the IHS:

receive at least a portion of the first SLAM data;

apply a transform matrix to the first SLAM data to produce transformed first SLAM data in a common coordinate system with respect to the second SLAM data; and

render at least another partial map for display by the second HMD using a combination of landmarks identified in the first transformed SLAM data and in the second SLAM data.

9. The IHS of claim 1, wherein the program instructions, upon execution by the processor, further cause the IHS to send a request to the second IHS in response to a determination that the second IHS has joined a mesh network.

10. The IHS of claim 9, wherein the second HMD is in direct communications with the second IHS and not with the first IHS, and wherein the first IHS is in direct communications with the second IHS.

11. A method, comprising:

receiving, at an Information Handling System (IHS) coupled to a Head-Mounted Device (HMD) worn by a first user during execution of a virtual, augmented, or mixed reality (xR) application, first Simultaneous Localization and Mapping (SLAM) data obtained via a first camera mounted on the first HMD;

receiving second SLAM data obtained by a second HMD worn by a second user co-located with the first user;

applying a transform matrix to the second SLAM data to produce transformed second SLAM data in a common coordinate system with respect to the first SLAM data;

rendering at least a partial map for display by the first HMD using a combination of landmarks identified in the first SLAM data and in the transformed second SLAM data; and

in response to a determination that a processor'"'"'s utilization is above a threshold level, reducing an amount of second SLAM data relative to an amount of first SLAM data processed.

12. The method of claim 11, wherein the first SLAM data comprises landmarks found within a first Region of Interest (ROI) observable by the first user, and wherein the second SLAM data comprises other landmarks found within a second ROI observable by another user.

13. The method of claim 12, wherein using the combination of landmarks further comprises resolving two or more landmarks that overlap in space between the first ROI and the second ROI.

14. The method of claim 13, wherein resolving the two or more landmarks further comprises comparing Kalman Gains of each of the two or more landmarks separated by a threshold distance, and selecting a landmark with highest Kalman Gain.

15. A hardware memory of a Head-Mounted Device (HMD) wearable by a user during execution of a virtual, augmented, or mixed reality (xR) application, the hardware memory having program instructions stored thereon that, upon execution by a processor, cause the HMD to:

collect first Simultaneous Localization and Mapping (SLAM) data;

transmit the first SLAM data to an Information Handling System (IHS) coupled to the HMD;

display a rendering received from IHS, wherein the rendering is created, at least in part, using a combination of landmarks identified in the first SLAM data and in second SLAM data obtained by the IHS from a second HMD; and

in response to a determination that the processor'"'"'s utilization is above a threshold level, reduce an amount of second SLAM data relative to an amount of first SLAM data processed.

16. The hardware memory of claim 15, wherein the second HMD is in direct communication with the first IHS.

17. The hardware memory of claim 15, wherein the second HMD is in direct communications with a second IHS and not with the first IHS, and wherein the second IHS is in direct communication with the first IHS.