ACCURATE POSITIONING OF AUGMENTED REALITY CONTENT

• US 20180005450A1
• Filed: 05/10/2017
• Published: 01/04/2018
• Est. Priority Date: 11/11/2014
• Status: Active Grant

• Current Assignee: Youar Inc.
• Original Assignee: Bent Image Lab, LLC
• Inventors: DANIELS, Oliver Clayton, DANIELS, David Morris, DI CARLO, Raymond Victor, CALICA, Carlo J., HARTWIG, Luke Timothy
• Granted Patent: US 10,311,643 B2
• Time in Patent Office: Days
• CPC Class Codes: G01S 19/01 Satellite radio beacon positioning systems transmitting time-stamped messages, e.g. GPS [Global Positioning System], GLONASS [Global Orbiting Navigation Satellite System] or GALILEO
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: G06T 2215/16 Using real world measurements to influence rendering
• CPC Class Codes: G06V 20/20 in augmented reality scenes

Abstract

A system for accurately positioning augmented reality (AR) content within a coordinate system such as the World Geodetic System (WGS) may include AR content tethered to trackable physical features. As the system is used by mobile computing devices, each mobile device may calculate and compare relative positioning data between the trackable features. The system may connect and group the trackable features hierarchically, as measurements are obtained. As additional measurements are made of the trackable features in a group, the relative position data may be improved, e.g., using statistical methods.

Claims

1. A computer-implemented method for accurately locating augmented reality (AR) content, the method comprising:

measuring, using one or more sensors of a computing device, a first six-degree-of-freedom (DOF) vector between a first vantage point of the computing device and a first trackable feature in a first sensor range of the computing device;

measuring, using the one or more sensors of the computing device, a second six-DOF vector between a second vantage point of the computing device and a second trackable feature in a second sensor range of the computing device;

measuring, using the one or more sensors of the computing device, a third six-DOF vector between the first vantage point and the second vantage point;

estimating, based on the first vector, the second vector, and the third vector, using a processor of the computing device, a fourth six-DOF vector between the first trackable feature and the second trackable feature;

storing the estimated fourth vector in a data store in association with the first trackable feature and the second trackable feature, such that the estimated fourth vector represents a spatial relationship between the first trackable feature and the second trackable feature;

determining, within a common spatial coordinate system, a first set of spatial coordinates for the first trackable feature and a second set of spatial coordinates for the second trackable feature; and

storing, in the data store, the first set of spatial coordinates associated with the first trackable feature and the second set of spatial coordinates associated with the second trackable feature, such that the first and second sets of spatial coordinates represent respective locations of the first and second trackable features within the common coordinate system;

wherein the data store includes a fifth six-DOF vector representing a spatial relationship and an orientation of an AR content item relative to the first trackable feature.

2. The method of claim 1, further comprising:

measuring, using the one or more sensors of the computing device, a sixth six-DOF vector between a current vantage point of the computing device and the second trackable feature when the second trackable feature is in a current sensor range of the computing device;

determining, using the processor of the computing device and based on the fourth vector, the fifth vector, and the sixth vector, a six-DOF current rendering vector for the current vantage point of the computing device, the current rendering vector including an orientation component; and

presenting, at a graphical display of the computing device, an AR representation including the AR content item rendered based on the current rendering vector within a live view of a real-world environment for a current field of view (FOV) of the computing device.

3. The method of claim 2, wherein the first trackable feature is outside the current sensor range of the computing device.

4. The method of claim 1, further comprising assigning an uncertainty value to the estimated fourth vector in the data store, based on respective uncertainties in the measurements of the first vector, the second vector, and the third vector.

5. The method of claim 1, wherein the computing device is a first computing device, the method further comprising:

updating at least one component of the fourth vector based on a measurement by a second computing device.

6. The method of claim 5, wherein updating the at least one component of the fourth vector comprises updating an uncertainty value assigned to the fourth vector.

7. The method of claim 1, further comprising:

associating the first trackable feature and the second trackable feature with a first group of trackable features in the data store, each trackable feature in the first group having a position and orientation defined with respect to a first coordinate system;

measuring, using the computing device, a seventh six-DOF vector between the first trackable feature and a third trackable feature associated with a second group of trackable features in the data store, each trackable feature in the second group having a position and orientation defined with respect to a second coordinate system; and

calculating a positional offset and an orientation offset between the first group and the second group based on the seventh six-DOF vector.

8. The method of claim 7, further comprising:

generating a third group comprising the first, second, and third trackable features, and defining a respective position and orientation of each of the first, second, and third trackable features with respect to a third coordinate system.

9. The method of claim 7, wherein the computing device is a first computing device, the method further comprising:

updating at least one component of the fourth vector based on a measurement by a second computing device, wherein updating the estimated fourth vector is performed independently with respect to the seventh vector.

10. The method of claim 7, further comprising:

adjusting the positional offset and the orientation offset between the first group and the second group based on at least one additional measurement between a selected trackable feature of the first group and a selected trackable feature of the second group.

11. The method of claim 7, further comprising associating an uncertainty value to the positional offset, the uncertainty value being based on an uncertainty of the seventh six-DOF vector;

and

adjusting the uncertainty value of the positional offset based on at least one additional measurement between a selected trackable feature of the first group and a selected trackable feature of the second group.

12. The method of claim 1, wherein the common spatial coordinate system comprises the World Geodetic System.

13. The method of claim 1, wherein the computing device is a first computing device, the method further comprising:

measuring, using one or more sensors of a second computing device, an eighth six-degree-of-freedom (DOF) vector between a vantage point of the second computing device and the first trackable feature; and

updating the first set of spatial coordinates based on the eighth vector.

14. The method of claim 13, wherein updating the first set of spatial coordinates comprises updating an uncertainty associated with the first set of spatial coordinates.

15. The method of claim 1, wherein the data store is at least partially disposed on a communications network, and the computing device is in communication with the data store via the communications network.

16. A computer-implemented method, comprising:

maintaining, in a data store, a set of trackable points, each trackable point representing a trackable physical feature to which an augmented reality (AR) content item is tetherable;

for each trackable point in the set of trackable points, maintaining a first set of estimated spatial values and a second set of estimated spatial values in the data store, the first set of estimated spatial values representing a position of the respective trackable point in a respective group coordinate system, and, by extension, one or more spatial relationships between the respective trackable point and all other trackable points of a respective group of trackable points positioned within the respective group coordinate system, and the second set of estimated spatial values representing a position of the respective trackable point in a respective global coordinate system;

obtaining first measurement data using at least one first mobile computing device, the first measurement data comprising a plurality of first measurements relating to a plurality of onsite trackable physical features within a selected real-world environment, the at least one first mobile computing device being onsite with respect to the selected real-world environment;

for each first set of spatial values corresponding to the plurality of onsite trackable physical features, maintaining a respective uncertainty value corresponding to the first measurement data;

obtaining second measurement data using at least one onsite second computing device, the second measurement data comprising at least one second measurement relating to the plurality of onsite trackable physical features within the selected real-world environment; and

updating the first and second estimated spatial values of at least one trackable point in the set of trackable points, based on the second measurement data.

17. The method of claim 16, wherein updating the first and second estimated spatial values of the at least one trackable point in the set of trackable points includes reducing the uncertainty value for the first set of spatial values.

18. The method of claim 16, wherein at least one of the respective group coordinate systems has an origin and an orientation.

19. The method of claim 16, further comprising:

forming a parent group comprising at least two of the respective groups of trackable points, wherein each of the at least two groups is a respective child group having a third set of spatial values representing a position and orientation of the respective group coordinate system with respect to the parent group.

20. The method of claim 19, further comprising:

for the trackable points in the at least two child groups, calculating and assigning the first set of spatial values in relation to the parent group coordinate system; and

removing the at least two child groups from the system.

21. The method of claim 16, further comprising:

presenting, at a graphical display of the at least one second computing device, an AR representation including an AR content item tethered to one of the trackable physical features within a live view of the selected real-world environment.

22. The method of claim 21, wherein the trackable physical feature to which the AR content item is tethered is untracked by the first computing device.

23. The method of claim 22, wherein the trackable physical feature to which the AR content item is tethered is out of sensing range of the second computing device.

24. The method of claim 16, further comprising

generating a new trackable point in the set of trackable points;

and

adding the new trackable point to one of the existing groups of trackable points, based on third measurement data taken in the selected real-world environment, such that a respective spatial relationship is defined between the new trackable point and at least one of the other trackable points.

25. The method of claim 24, further comprising:

maintaining the first set of estimated spatial values and the second set of estimated spatial values for the new trackable point in the data store, the first set of estimated spatial values representing the spatial relationship between the new trackable point and at least one of the other trackable points, the second set of spatial values representing the position of the new trackable point in the global coordinate system.