Systems and methods for unifying coordinate systems in augmented reality applications

• US 8,839,121 B2
• Filed: 05/06/2009
• Issued: 09/16/2014
• Est. Priority Date: 05/06/2009
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: Bertolami, Joseph, Mann, Samuel A., Bronder, Matthew L., Dougherty, Michael A., Craig, Robert M., Lee, Matthew W.
• Primary Examiner(s): ULRICH, NICHOLAS S
• Application Number:
  US12/436,420
• Publication Number:
  US 20100287485A1
• Time in Patent Office: 1,959 Days
• Field of Search: 345/633, 715/757, 708/442
• US Class Current: 715/757
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00

Abstract

Systems and methods for unifying coordinate systems in an augmented reality application or system are disclosed. User devices capture an image of a scene, and determine a location based on the scene image. The scene image may be compared to cartography data or images to determine the location. User devices may propose an origin and orientation or transformation data for a common coordinate system and exchange proposed coordinate system data to agree on a common coordinate system. User devices may also transmit location information to an augmented reality system that then determines an a common coordinate system and transmits coordinate system data such as transformation matrices to the user devices. Images presented to users may be adjusted based on user device locations relative to the coordinate system.

Claims

1. A method comprising:

capturing on a first device a first scene image;

identifying on the first device a landmark shown in the first scene image;

determining on the first device a precise location of the first device based on the landmark;

determining on the first device a precise location of the landmark;

determining on the first device first coordinate system negotiation data comprising the precise location of the landmark as a proposed origin for a first coordinate system;

transmitting the first coordinate system negotiation data and the precise location of the first device to a second device; and

receiving a response from the second device comprising second coordinate system negotiation data comprising an acceptance of the first coordinate system negotiation data.

2. The method of claim 1, wherein the second coordinate system negotiation data comprises transformation data, the method further comprising:

determining virtual element locations based on the transformation data, and presenting images comprising virtual elements on a display of the first device based on the virtual element locations.

3. The method of claim 1, further comprising determining on the first device an estimated location of the first device and determining a first set of cartography data based on the estimated location of the first device.

4. The method of claim 3, wherein identifying on the first device the landmark comprises selecting the landmark from a plurality of landmarks associated with the estimated location of the first device.

5. The method of claim 3, wherein transmitting the first coordinate system negotiation data and the precise location of the first device to the second device comprises transmitting identification data of the first set of cartography data to the second device.

6. The method of claim 1, wherein determining on the first device the precise location of the first device based on the landmark comprises determining that the landmark comprises a fiducial tag represented in a first set of cartography data.

7. The method of claim 1, further comprising generating cartography data based on the first scene image.

8. The method of claim 1, further comprising receiving cartography data from the second device.

9. A computer-readable storage medium comprising computer-readable instructions that when executed by a computing device cause the computing device to perform the instructions comprising:

receiving first location data from a first device comprising a precise location of a first landmark detected by the first device;

receiving second location data from a second device comprising a precise location of a second landmark detected by the second device;

selecting the precise location of the first landmark as a proposed origin and determining a first orientation based on the first location data and the second location data;

transmitting the proposed origin and the first orientation to the first device and the second device;

receiving a response from at least one of the first device and the second device;

responsive to receiving the response, selecting the precise location of the second landmark as the proposed origin and determining a second orientation based on the first location data and the second location data;

determining coordinate system data based on the proposed origin and the second orientation; and

transmitting the coordinate system data to the first device and the second device.

10. The computer-readable storage medium of claim 9, wherein determining coordinate system data based on the proposed origin and the second orientation comprises determining a first transformation matrix based on the first location data and the proposed origin and the second orientation and determining a second transformation matrix based on the second location data and the proposed origin and the second orientation.

11. The computer-readable storage medium of claim 9, further comprising:

receiving first cartography data for a first area from the first device;

receiving second cartography data for a second area from the second device;

determining congruence data for the first area and the second area based on the first cartography data and the second cartography data;

determining a first matrix transformation for the first device and a second matrix transformation for the second device based on the congruence data; and

transmitting the first matrix transformation to the first device and the second matrix transformation to the second device.

12. The computer-readable storage medium of claim 11, further comprising:

generating third cartography data for a third area, wherein the third area comprises the first area and the second area; and

transmitting the third cartography data to the first device and the second device.

13. The computer-readable storage medium of claim 9, further comprising:

detecting a resynchronization trigger;

transmitting a request for location data to the first device and the second device;

receiving updated first location data from the first device;

receiving updated second location data from the second device;

determining an updated origin based on the updated first location data and the updated second location data;

determining updated coordinate system data based on the updated origin; and

transmitting the updated coordinate system data to the first device and the second device.

14. The computer-readable storage medium of claim 9, wherein determining the coordinate system data based on the proposed origin and the second orientation comprises determining cartography data based on the first location data and the second location data, and wherein transmitting the coordinate system data to the first device and the second device comprises transmitting the cartography data to the first device and the second device.

15. A system comprising:

a detection component configured to capture a first scene image;

a processor configured to;

identify a landmark shown in the first scene image,

determine a precise position and orientation of the system based on the landmark,

and

determine a precise location of the landmark;

a transmitter configured to transmit the precise position and orientation of the system and the precise location of the landmark as a proposed origin for a first coordinate system to a remote device;

a receiver configured to receive, from the remote device, coordinate system data comprising an indication that the precise location of the landmark is an origin for the first coordinate system;

the processor further configured to generate a first presentation image based, at least in part, on the coordinate system data; and

a display configured to render the first presentation image.

16. The system of claim 15, wherein the coordinate system data comprises a transformation matrix.

17. The system of claim 16, wherein the processor is further configured to generate the first presentation image by determining a transformed precise position and orientation of the system using the transformation matrix, determining a virtual element location relative to the transformed precise position and orientation of the system, and rendering the first presentation image comprising a virtual element based on the virtual element location.

18. The system of claim 15, wherein the detection component is configured to capture a second scene image, and wherein the processor is further configured to generate the first presentation image by compositing the second scene image with an image of a virtual element.

19. The system of claim 15, wherein the processor is further configured to determine the precise position and orientation of the system based on the landmark comprises the processor configured to compare the landmark to at least one element template.

20. The system of claim 19, further comprising a position and orientation component configured to determine a coarse position and orientation of the system, wherein the transmitter is further configured to transmit a request for element template data comprising the coarse position and orientation of the system to the remote device, and wherein the receiver is further configured to receive the element template data from the remote device.