Generation and Sharing Coordinate System Between Users on Mobile

• US 20140267234A1
• Filed: 03/15/2013
• Published: 09/18/2014
• Est. Priority Date: 03/15/2013
• Status: Abandoned Application

• Current Assignee: Dekko, Inc.
• Original Assignee: Dekko, Inc.
• Inventors: Maes, Anthony, Gardeya, Marc, Naimark, Leonid, Hook, Anselm, Fite-Georgel, Pierre, Meisnieks, Matt
• Application Number:
  US13/835,822
• Publication Number:
  US 20140267234A1
• Time in Patent Office: Days
• US Class Current: 345/419
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: H04W 4/02 Services making use of location information
• CPC Class Codes: H04W 4/027 using movement velocity, acceleration information
• CPC Class Codes: H04W 4/029 Location-based management or tracking services
• CPC Class Codes: H04W 4/80 Services using short range communication, e.g. near-field communication [NFC], radio-frequency identification [RFID] or low energy communication

Abstract

A multi-device system for mobile devices to acquire and share 3D maps of an environment. The mobile devices determine features of the environment and construct a local map and coordinate system for the features identified by the mobile device. The mobile devices may create a joint map by joining the local map of another mobile device or by merging the local maps created by the mobile devices. To merge maps, the coordinate system of each system may be constrained in degrees of freedom using information from sensors on the devices to determine the global position and orientation of each device. When the devices operate on a joint map, the device share information about new features to extend the range of features on the map and share information about augmented reality objects manipulated by users of each device.

Claims

1. A computer-implemented method for a shared augmented reality experience, comprising:

accessing a first local feature map of an environment in which a mobile device is located, the first local feature map associated with a first local feature map coordinate system;

receiving, a second local feature map of the environment associated with a second mobile device, the second local feature map associated with a second local feature map coordinate system;

merging the first local feature map and the second local feature map to generate a joint feature map of the environment;

determining the position of the mobile device relative to the joint feature map.

2. The computer-implemented method of claim 1, wherein merging the local feature map comprises determining reduced degrees of freedom of the first local feature map and the second local feature map.

3. The computer-implemented method of claim 2, wherein the reduced degrees of freedom are based on sensor data associated with the mobile device and the second mobile device.

4. The computer-implemented method of claim 3, wherein the sensor data comprises at least one of:

accelerometer data, gyroscope data, magnetometer data, rangefinder data, global positioning satellite receiver data, cellular tower data, range differential data, altitude data, photo resister data, or a clock.

5. The computer-implemented method of claim 1, wherein merging the local feature map comprises eliminating duplicative features of the first local feature map and second local feature map from the joint feature map.

6. The computer-implemented method of claim 1, wherein merging the first local feature map and the second local feature map comprises determining a coordinate system translation between the first local feature map and the second local feature map.

7. A computer-implemented method of managing communication of a joint feature map in an augmented reality system, comprising:

accessing a feature map associated with a joint feature map maintained with respect to a global coordinate system;

receiving a video feed of an environment;

determining environment features from frames of the video feed of the camera;

determining a pose of a mobile device relative to the joint feature map;

identifying at least one feature from the environment features not included in the feature map associated with the joint feature map;

adding the at least one feature not included in the feature map to the feature map;

broadcasting the added at least one feature to a second mobile device.

8. The computer-implemented method of claim 7, wherein prior to broadcasting the at least one feature, the at least one feature is reduced using data sparcification.

9. The computer-implemented method of claim 7, wherein prior to broadcasting the at least one feature, the at least one feature is reduced using data compression.

10. The computer-implemented method of claim 7, wherein the joint feature map is stored on a server, and further comprising receiving, by the server, the added at least one feature and merging the at least one feature to the joint feature map.

11. A computer-implemented method for sharing a joint feature map, comprising

generating, on a first mobile device, a feature map of an environment based on features identified from a first video feed;

providing, by the first mobile device, the feature map of the environment to a second mobile device;

receiving, by the second mobile device, the feature map of the environment;

determining, by the second mobile device, second features of the environment identified by a video feed from a camera on the second mobile device;

using, by the second mobile device, the feature map received from the first device as a local feature map; and

identifying a position of the second mobile device relative to the local feature map by comparing the features in the local feature map to the second features.

12. The computer-implemented method of claim 11, further comprising:

generating, on the first mobile device, augmented reality content associated with a location in the feature map;

transmitting, by the first mobile device, the augmented reality content to the second mobile device;

generating a display on the second mobile device including at least a portion of the augmented reality content responsive to the video feed including a portion of the environment including the location;

providing the display to a user of the second user device.