Systems and Methods for Unifying Coordinate Systems in Augmented Reality Applications

• US 20100287485A1
• Filed: 05/06/2009
• Published: 11/11/2010
• Est. Priority Date: 05/06/2009
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: Mann, Samuel A., Bronder, Matthew L., Dougherty, Michael A., Bertolami, Joseph, Lee, Matthew W., Craig, Robert M.
• Granted Patent: US 8,839,121 B2
• Time in Patent Office: Days
• US Class Current: 715/764
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00

Abstract

Systems and methods for unifying coordinate systems in an augmented reality application or system are disclosed. User devices capture an image of a scene, and determine a location based on the scene image. The scene image may be compared to cartography data or images to determine the location. User devices may propose an origin and orientation or transformation data for a common coordinate system and exchange proposed coordinate system data to agree on a common coordinate system. User devices may also transmit location information to an augmented reality system that then determines an a common coordinate system and transmits coordinate system data such as transformation matrices to the user devices. Images presented to users may be adjusted based on user device locations relative to the coordinate system.

Claims

1. A method of unifying coordinate systems, comprising:

detecting on a first device a first scene image;

determining on the first device a precise location based on the first scene image;

determining on the first device first coordinate system negotiation data;

transmitting the first coordinate system negotiation data and the location to a second device; and

receiving a response from the second device comprising second coordinate system negotiation data.

2. The method of claim 1, wherein the second coordinate system negotiation data comprises transformation data, the method further comprising:

determining virtual element locations based on the transformation data, and

presenting images comprising virtual elements on a display of the first device based on the virtual element locations.

3. The method of claim 1, further comprising determining on the first device an estimated location and determining a first set of cartography data based on the estimated location.

4. The method of claim 3, wherein determining on the first device the precise location based on the first scene image comprises comparing at least a portion of the first scene image to the first set of cartography data.

5. The method of claim 4, wherein transmitting the first coordinate system negotiation data and the location to the second device comprises transmitting identification data of the first set of cartography data to the second device.

6. The method of claim 1, wherein determining on the first device the precise location based on the first scene image comprises determining whether at least a portion of the first scene image contains a fiducial tag represented in a first set of cartography data.

7. The method of claim 1, further comprising generating cartography data based on the first scene image.

8. The method of claim 1, further comprising receiving cartography data from the second device.

9. A computer-readable storage medium comprising computer-readable instructions for determining a common coordinate system in an augmented reality system, the computer-readable instructions comprising instructions for:

receiving first location data from a first device;

receiving second location data from a second device;

determining an origin and orientation based on the first location data and the second location data;

determining coordinate system data based on the origin and orientation, and

transmitting the coordinate system data to the first device and the second device.

10. The computer-readable storage medium of claim 9, wherein determining coordinate system data based on the origin comprises determining a first transformation matrix based on the first location and the origin and orientation and determining a second transformation matrix based on the second location and the origin and orientation.

11. The computer-readable storage medium of claim 9, further comprising:

receiving first cartography data for a first area from the first device;

receiving second cartography data for a second area from the second device;

determining congruence data for the first area and the second area based on the first cartography data and the second cartography data;

determining a first matrix transformation for the first device and a second matrix transformation for the second device based on the congruence data; and

transmitting the first matrix transformation to the first device and the second matrix transformation to the second device.

12. The computer-readable storage medium of claim 11, further comprising:

generating third cartography data for a third area, wherein the third area comprises the first area and the second area; and

transmitting the third cartography data to the first device and the second device.

13. The computer-readable storage medium of claim 9, further comprising:

detecting a resynchronization trigger;

transmitting a request for location data to the first device and the second device;

receiving updated first location data from the first device;

receiving updated second location data from the second device;

determining an updated origin based on the updated first location data and the updated second location data;

determining updated coordinate system data based on the updated origin; and

transmitting the updated coordinate system data to the first device and the second device.

14. The computer-readable storage medium of claim 9, wherein determining the coordinate system data based on the origin comprises determining cartography data based on the first location and the second location, and wherein transmitting the coordinate system data to the first device and the second device comprises transmitting the cartography data to the first device and the second device.

15. A system for presenting images to a user based on a common coordinate system, comprising:

a detection component configured to capture a first scene image;

a processor configured to determine a precise position and orientation based on the scene image;

a transmitter configured to transmit the precise position and orientation to a remote device;

a receiver configured to receive coordinate system data from the remote device;

the processor further configured to generate a first presentation image based, at least in part, on the coordinate system data; and

a display configured to render the first presentation image.

16. The system of claim 15, wherein the coordinate system data comprises a transformation matrix.

17. The system of claim 16, wherein the processor is further configured to generate the first presentation image by determining a transformed precise position and orientation using the transformation matrix, determining a virtual element location relative to the transformed precise position and orientation, and rendering the first presentation image comprising a virtual element based on the virtual element location.

18. The system of claim 15, wherein the detection component is configured to capture a second scene image, and wherein the processor is further configured to generate the first presentation image by compositing the second scene image with an image of a virtual element.

19. The system of claim 15, wherein the processor configured to determine the precise position and orientation based on the scene image comprises the processor configured to compare a subset of the scene image to at least one element template.

20. The system of claim 19, further comprising a position and orientation component configured to determine a coarse position and orientation, wherein the transmitter is further configured to transmit a request for element template data comprising the coarse position and orientation to the remote device, and wherein the receiver is further configured to receive the element template data from the remote device.