Multi-node poster location

• US 9,105,210 B2
• Filed: 06/29/2012
• Issued: 08/11/2015
• Est. Priority Date: 06/29/2012
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: Lamb, Mathew J., Sugden, Ben J., Crocco, Robert L. Jr., Keane, Brian E., Miles, Christopher E., Perez, Kathryn Stone, Massey, Laura K., Kipman, Alex Aben-Athar
• Primary Examiner(s): Amini, Javid A
• Application Number:
  US13/538,829
• Publication Number:
  US 20140002495A1
• Time in Patent Office: 1,138 Days
• Field of Search: 345/633
• US Class Current: 1/1
• CPC Class Codes: G02B 2027/0138 comprising image capture systems, e.g. camera
• CPC Class Codes: G02B 2027/014 comprising information/image processing systems
• CPC Class Codes: G02B 2027/0178 Eyeglass type eyeglass details G02C
• CPC Class Codes: G02B 27/017 Head mounted
• CPC Class Codes: G06F 3/1431 using a single graphics controller
• CPC Class Codes: G09G 2370/022 Centralised management of display operation, e.g. in a server instead of locally
• CPC Class Codes: G09G 3/003 to produce spatial visual effects
• CPC Class Codes: H04N 2201/3245 of image modifying data, e.g. handwritten addenda, highlights or augmented reality information
• CPC Class Codes: H04N 2201/3269 of machine readable codes or marks, e.g. bar codes or glyphs

Abstract

A system for identifying an AR tag and determining a location for a virtual object within an augmented reality environment corresponding with the AR tag is described. In some environments, including those with viewing obstructions, the identity of the AR tag and the location of a corresponding virtual object may be determined by aggregating individual identity and location determinations from a plurality of head-mounted display devices (HMDs). The virtual object may comprise a shared virtual object that is viewable from each of the plurality of HMDs as existing at a shared location within the augmented reality environment. The shared location may comprise a weighted average of individual location determinations from each of the plurality of HMDs. By aggregating and analyzing individual identity and location determinations, a particular HMD of the plurality of HMDs may display a virtual object without having to identify a corresponding AR tag directly.

Claims

1. A method for generating and displaying one or more virtual objects, comprising:

identifying a particular tag within an environment, the identifying is performed by a first mobile device;

determining a first location associated with the particular tag in response to the identifying a particular tag, the determining a first location is performed by the first mobile device, the determining a first location includes determining the first location relative to a 3D map of the environment;

acquiring a second location associated with the particular tag from a second mobile device different from the first mobile device, the second location is relative to the 3D map of the environment, the second location is different from the first location;

determining a shared location associated with the particular tag based on the first location and the second location, the determining a shared location includes determining a weighted average of the first location and the second location;

determining a first confidence value associated with the first location, the first confidence value corresponds with a first distance between the first mobile device and the particular tag;

acquiring a second confidence value associated with the second location from the second mobile device, the second confidence value corresponds with a second distance between the second mobile device and the particular tag, the determining a shared location includes determining the weighted average of the first location and the second location based on the first confidence value and the second confidence value;

acquiring a virtual object associated with the particular tag; and

displaying at the first mobile device the virtual object such that the virtual object is perceived to exist at a point in space corresponding with the shared location.

2. The method of claim 1, further comprising:

the first confidence value is determined based on a first number of feature points of the particular tag that have been identified by the first mobile device; and

the second confidence value is determined based on a second number of feature points of the particular tag that have been identified by the second mobile device.

3. The method of claim 1, wherein:

the virtual object comprises a shared virtual object, the shared virtual object is viewable from both the first mobile device and the second mobile device as existing at the shared location within the environment.

4. The method of claim 1, further comprising:

acquiring the 3D map of the environment; and

capturing a first set of images of the environment, the determining a first location includes determining the first location relative to the 3D map based on the first set of images.

5. The method of claim 4, wherein:

the determining a shared location includes determining the shared location relative to the 3D map based on the first location and the second location.

6. The method of claim 1, further comprising:

receiving a second shared location from the second mobile device; and

determining whether to issue an alert to the second mobile device based on the shared location and the second shared location.

7. The method of claim 1, wherein:

the particular tag is associated with a particular page of a reading object; and

the virtual object comprises a holographic visual aid associated with the particular page.

8. The method of claim 7, wherein:

the reading object comprises an augmentable book.

9. One or more storage devices containing processor readable code for programming one or more processors to perform a method for generating and displaying one or more virtual objects comprising:

capturing a first set of images of an environment at a first mobile device;

acquiring a 3D map of the environment at the first mobile device;

identifying a particular tag within the environment based on the first set of images;

determining that the first mobile device can determine a pose relative to the 3D map based on the first set of images;

determining a first location associated with the particular tag relative to the 3D map of the environment in response to the determining that the first mobile device can determine a pose relative to the 3D map;

acquiring a second location associated with the particular tag from a second mobile device different from the first mobile device, the second location is relative to the 3D map of the environment, the second location is different from the first location;

determining a shared location associated with the particular tag based on the first location and the second location, the determining a shared location includes determining a weighted average of the first location and the second location, the determining a shared location includes determining a first confidence value associated with the first location, the first confidence value corresponds with a first distance between the first mobile device and the particular tag, the determining a shared location includes determining a second confidence value associated with the second location, the second confidence value corresponds with a second distance between the second mobile device and the particular tag, the determining a shared location includes determining the weighted average of the first location and the second location based on the first confidence value and the second confidence value;

acquiring a virtual object associated with the particular tag; and

displaying at the first mobile device the virtual object such that the virtual object is perceived to exist at a point in space corresponding with the shared location.

10. The one or more storage devices of claim 9, wherein:

the virtual object comprises a shared virtual object, the shared virtual object is viewable from both the first mobile device and the second mobile device as existing at the shared location within the environment.

11. The one or more storage devices of claim 9, further comprising:

the first confidence value is determined based on a first number of feature points of the particular tag that have been identified by the first mobile device; and

the second confidence value is determined based on a second number of feature points of the particular tag that have been identified by the second mobile device.

12. The one or more storage devices of claim 9, further comprising:

receiving a second shared location from the second mobile device; and

determining whether to issue an alert to the second mobile device based on the shared location and the second shared location.

13. The one or more storage devices of claim 9, wherein:

the particular tag is associated with a particular page of a reading object; and

the virtual object comprises a holographic visual aid associated with the particular page.

14. The one or more storage devices of claim 13, wherein:

the reading object comprises an augmentable book.

15. An electronic device for generating and displaying one or more virtual objects, comprising:

a memory, the memory stores a 3D map of an environment;

one or more processors in communication with the memory, the one or more processors identify a particular tag within the environment and determine a first location associated with the particular tag relative to the 3D map, the one or more processors acquire a second location associated with the particular tag from a second mobile device different from the electronic device, the second location is relative to the 3D map, the second location is different from the first location, the one or more processors determine a shared location based on a weighted average of the first location and the second location, the one or more processors detect a first triggering event associated with a virtual object, the one or more processors generate one or more images corresponding with the virtual object such that the virtual object is perceived to exist within the environment at a point in space corresponding with the shared location, the one or more processors determine a first confidence value associated with the first location, the first confidence value corresponds with a first distance between the first mobile device and the particular tag, the one or more processors acquire a second confidence value associated with the second location, the second confidence value corresponds with a second distance between the second mobile device and the particular tag, the one or more processors determine the shared location by determining the weighted average of the first location and the second location based on the first confidence value and the second confidence value; and

a see-through display in communication with the one or more processors, the see-through display displays the one or more images.

16. The electronic device of claim 15, wherein:

the particular tag is associated with a particular page of a reading object; and

the one or more processors detect the first triggering event by detecting an end user of the electronic device gazing at the reading object.

17. The electronic device of claim 15, wherein:

the particular tag corresponds with a field tag within the environment; and

the one or more processors detect the first triggering event by detecting an end user of the electronic device performing a particular gesture.