SHARING NEIGHBORING MAP DATA ACROSS DEVICES

• US 20180285052A1
• Filed: 05/11/2017
• Published: 10/04/2018
• Est. Priority Date: 03/30/2017
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: EADE, Ethan, VANTURENNOUT, Jeroen, LYONS, Jonathan, FIELDS, David, LAZAROW, Gavin Dean, BHATNAGAR, Tushar Cyril
• Granted Patent: US 10,466,953 B2
• Time in Patent Office: Days
• CPC Class Codes: B65G 1/0492 with cars adapted to travel in storage aisles
• CPC Class Codes: G02B 2027/0138 comprising image capture systems, e.g. camera
• CPC Class Codes: G02B 2027/0141 characterised by the informative content of the display
• CPC Class Codes: G02B 27/017 Head mounted
• CPC Class Codes: G03H 1/0005 Adaptation of holography to specific applications holographic optical element G02B5/32; holographic scanner G02B26/106; recognition using holographic mask G06V10/88; holographic memories G11B7/0065, G11C13/042
• CPC Class Codes: G03H 1/0808 Methods of numerical synthesis, e.g. coherent ray tracing [CRT], diffraction specific
• CPC Class Codes: G03H 1/2249 Holobject properties
• CPC Class Codes: G03H 2001/0088 for video-holography, i.e. integrating hologram acquisition, transmission and display
• CPC Class Codes: G03H 2001/2252 Location of the holobject
• CPC Class Codes: G03H 2226/02 Computing or processing means, e.g. digital signal processor [DSP]
• CPC Class Codes: G03H 2226/04 Transmission or communication means, e.g. internet protocol
• CPC Class Codes: G03H 2226/05 Means for tracking the observer
• CPC Class Codes: G05B 2219/40126 Virtual landmarks, reference points for operator
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/012 Head tracking input arrangements
• CPC Class Codes: G06F 3/04815 Interaction with a metaphor-based environment or interaction object displayed as three-dimensional, e.g. changing the user viewpoint with respect to the environment or object
• CPC Class Codes: G06F 3/1423 controlling a plurality of local displays, e.g. CRT and flat panel display
• CPC Class Codes: G06F 3/1454 involving copying of the display data of a local workstation or window to a remote workstation or window so that an actual copy of the data is displayed simultaneously on two or more displays, e.g. teledisplay
• CPC Class Codes: G06F 3/147 using display panels
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: G09G 3/002 : to project the image of a two-dimensional display, such as an array of light emitting or modulating elements or a CRT
• CPC Class Codes: G09G 3/003 : to produce spatial visual effects
• CPC Class Codes: Y10S 901/01 : Mobile robot

Abstract

A computing device and method are provided for transmitting a relevant subset of map data, called a neighborhood, to enable mutual spatial understanding by multiple display devices around a target virtual location to display a shared hologram in the same exact location in the physical environment at the same moment in time. The computing device may comprise a processor, a memory operatively coupled to the processor, and an anchor transfer program stored in the memory and executed by the processor.

Claims

1. A server computing device, comprising:

a processor;

a non-volatile storage device operatively coupled to the processor; and

an anchor transfer program stored in the non-volatile storage device and executed by the processor of the computing device, wherein

the anchor transfer program is configured to;

receive a request to transfer anchor data of a target virtual place-located anchor at a target virtual location from a first display device operated by a first user to a second display device operated by a second user;

retrieve and transmit to the second display device neighboring map data corresponding to the target virtual location, the neighboring map data being map data of a neighborhood in a vicinity around the target virtual location;

cause the second display device to incorporate the neighboring map data into existing map data of the second display device; and

cause the second display device to display one or more holograms at the target virtual place-located anchor at the target virtual location from the second user'"'"'s vantage point based on the incorporated map data and the existing map data of the second display device.

2. The server computing device of claim 1, further comprising:

an anchor program stored in the memory and executed by the processor of the computing device, wherein

the anchor program is configured to;

receive an instruction from the first display device to generate the target virtual place-located anchor at the target virtual location;

generate the target virtual place-located anchor at the target virtual location; and

send the target virtual place-located anchor to the first display device.

3. The server computing device of claim 1, wherein

the anchor transfer program is configured to transmit to the first display device neighboring map data in a serialized format;

and

the second display device receives the neighboring map data in a deserialized format.

4. The server computing device of claim 1, wherein

the target virtual location is world-locked to a position that is fixed in a three-dimensional coordinate space overlaid upon a real world three-dimensional environment.

5. The server computing device of claim 1, wherein

the target virtual location is world-locked to a position relative to an object in a real world three-dimensional environment.

6. The server computing device of claim 1, wherein

the neighboring map data comprises keyframes and at least a portion of a pose-graph describing rotational and translational motion of the display devices through a real world three-dimensional environment.

7. The server computing device of claim 6, further comprising:

visual sensors and/or inertial measurement sensors, wherein

the visual sensors and/or inertial measurement sensors track the rotational and translational motion of the display devices for the keyframes and pose-graphs.

8. The server computing device of claim 6, wherein

the keyframes comprise at least one of a fingerprint of a Wi-Fi beacon, gravity data, temperature data, global positioning data, and calibration data.

9. A method, comprising:

receiving a request to transfer anchor data of a target virtual place-located anchor at a target virtual location from a first display device operated by a first user to a second display device operated by a second user;

retrieving and transmitting map data to the first display device neighboring map data corresponding to the target virtual location; and

subsequent to the first display device transferring anchor data and the neighboring map data to the second display device, causing the second display device to incorporate the neighboring map data into existing map data of the second display device, and causing the second display device to display one or more holograms to the second user at the target virtual place-located anchor at the target virtual location from the second user'"'"'s vantage point based on the incorporated map data and the existing map data of the second display device,

wherein the neighboring map data is map data of a neighborhood around the target virtual location.

10. The method of claim 9, further comprising:

receiving an instruction from the first display device to generate the target virtual place-located anchor at the target virtual location;

generating the target virtual place-located anchor at the target virtual location; and

sending the target virtual place-located anchor to the first display device.

11. The method of claim 9, wherein

the neighboring map data is transmitted to the first display device in a serialized format, and

the second display device receives the neighboring map data in a deserialized format.

12. The method of claim 9, wherein

the target virtual location is world-locked to a position that is fixed in a three-dimensional coordinate space overlaid upon a real world three-dimensional environment.

13. The method of claim 9, wherein

the target virtual location is world-locked to a position relative to an object in a real world three-dimensional environment.

14. The method of claim 9, wherein

the neighboring map data comprises keyframes and pose-graphs recording rotational and translational motion of the display devices through a real world three-dimensional environment.

15. The method of claim 14, wherein

visual sensors and/or inertial measurement sensors track the rotational and translational motion of the display devices for the keyframes and pose-graphs.

16. The method of claim 14, wherein

the keyframes comprise at least one of a fingerprint of a Wi-Fi beacon, gravity data, temperature data, global positioning data, and calibration data.

17. A first computing device operated by a first user, networked with a second computing device and a third computing device, the first computing device comprising:

a processor;

a memory operatively coupled to the processor and storing local map data of the first computing device;

a first display operatively coupled to the memory and the processor; and

an anchor transfer program stored in the memory and executed by the processor to be configured to receive first anchor data causing the first display to display one or more holograms to the first user at a first virtual place-located anchor at a first target virtual location from the first user'"'"'s vantage point, and further configured to execute an export anchor mode, wherein

in the export anchor mode, the anchor transfer program is configured to;

receive a first neighboring map data of a neighborhood around the first target virtual location; and

send the first anchor data and the first neighboring map data to the second computing device operated by a second user, thereby causing a second display of the second computing device to display the one or more holograms at the first virtual place-located anchor at the first target virtual location from the second user'"'"'s vantage point based on the first neighboring map data and the local map data of the second computing device.

18. The computing device of claim 17, wherein

the anchor transfer program is further configured to execute an import anchor mode;

and

in the import anchor mode, the anchor transfer program is configured to;

receive, from the third computing device, second neighboring map data of a neighborhood around a second virtual place-located anchor at a second target virtual location;

integrate the second neighboring map data into the local map data of the first computing device to create an integrated map data; and

cause the first display to display the one or more holograms at the second virtual place-located anchor at the second target virtual location from the first user'"'"'s vantage point based on the integrated map data.

19. A server computing device, comprising:

a processor;

a non-volatile storage device operatively coupled to the processor; and

an anchor transfer program stored in the non-volatile storage device and executed by the processor of the computing device, wherein

the anchor transfer program is configured to;

receive a request to transfer anchor data of a target virtual place-located anchor at a target virtual location from a first display device to a second display device;

retrieve and transmit to the second display device neighboring map data corresponding to the target virtual location, the neighboring map data being map data of a neighborhood in a vicinity around the target virtual location;

cause the second display device to incorporate the neighboring map data into existing map data of the second display device; and

cause the second display device to align movements in aligned coordinate space with the first display device based on the incorporated map data and the existing map data of the second display device,

wherein the first and second display devices are configured to be autonomous robots.

20. The server computing device of claim 19, wherein

the anchor transfer program is further configured to:

cause the second display device to render and interact with one or more virtual objects and/or virtual spaces at the target virtual place-located anchor at the target virtual location from the vantage point of the second display device based on the incorporated map data and the existing map data of the second display device.