CONSTRAINT BASED INFORMATION INFERENCE

• US 20140002496A1
• Filed: 06/29/2012
• Published: 01/02/2014
• Est. Priority Date: 06/29/2012
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Robert L. Crocco Jr
• Inventors: Lamb, Mathew J., Sugden, Ben J., Crocco, Robert L. Jr., Keane, Brian E., Miles, Christopher E., Perez, Kathryn Stone, Massey, Laura K., Kipman, Alex Aben-Athar
• Granted Patent: US 9,035,970 B2
• Time in Patent Office: Days
• US Class Current: 345/633
• CPC Class Codes: G02B 2027/0134 of stereoscopic type
• CPC Class Codes: G02B 2027/0138 comprising image capture systems, e.g. camera
• CPC Class Codes: G02B 2027/0178 Eyeglass type eyeglass details G02C
• CPC Class Codes: G02B 2027/0187 slaved to motion of at least a part of the body of the user, e.g. head, eye
• CPC Class Codes: G02B 27/017 Head mounted
• CPC Class Codes: G06F 3/14 Digital output to display device ; Cooperation and interconnection of the display device with other functional units
• CPC Class Codes: G06F 3/147 using display panels
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: G09G 2370/022 Centralised management of display operation, e.g. in a server instead of locally
• CPC Class Codes: G09G 2380/14 Electronic books and readers
• CPC Class Codes: G09G 3/003 to produce spatial visual effects
• CPC Class Codes: G09G 5/00 Control arrangements or circuits for visual indicators common to cathode-ray tube indicators and other visual indicators image data processing or generation, in general G06T

Abstract

A system for identifying an AR tag and determining a location for a virtual object within an augmented reality environment corresponding with the AR tag is described. In some environments, the location of a virtual object corresponding with a particular AR tag may be determined by identifying a predefined object, determining an orientation and a scale of the predefined object relative to a head-mounted display device (HMD) based on a model of the predefined object, and inferring the location of the virtual object based on the orientation and the scale of the predefined object. In some cases, an identification of the particular AR tag corresponding with the virtual object may be acquired by aggregating and analyzing individual identity determinations from a plurality of HMDs within an augmented reality environment.

Claims

1. A method for generating and displaying one or more virtual objects, comprising:

identifying a particular object within an environment, the identifying is performed by a first mobile device;

acquiring a 3D model of the particular object, the particular object is associated with one or more tags;

acquiring an identification of a first tag of the one or more tags from a second mobile device different from the first mobile device;

determining a virtual object corresponding with the first tag based on the identification of the first tag;

determining a first location associated with the virtual object relative to the 3D model; and

displaying at the first mobile device the virtual object such that the virtual object is perceived to exist at a point in space corresponding with the first location.

2. The method of claim 1, wherein:

the acquiring an identification of a first tag includes acquiring a poster index corresponding with the first tag from the second mobile device.

3. The method of claim 1, wherein:

the acquiring an identification of a first tag includes aggregating a plurality of individual identity determinations from a plurality of mobile devices within the environment, the plurality of mobile devices includes the second mobile device, an individual identity determination corresponding with the second mobile device comprises the identification of the first tag.

4. The method of claim 1, wherein:

the particular object comprises a reading object; and

the determining a first location associated with the virtual object includes determining a location of a cover tag associated with the reading object.

5. The method of claim 1, further comprising:

determining an orientation and a scale of the particular object relative to the first mobile device based on the 3D model, the determining an orientation and a scale of the particular object includes determining the orientation of the particular object within the environment relative to the first mobile device and determining a size of the particular object within the environment.

6. The method of claim 1, further comprising:

capturing a first set of images associated with the environment, the capturing is performed by the first mobile device, the identifying a particular object includes identifying the particular object based on the first set of images.

7. The method of claim 1, wherein:

the particular object comprises a reading object; and

the virtual object comprises a holographic visual aid associated with the reading object.

8. The method of claim 1, wherein:

the particular object comprises a reading object;

the first tag is associated with a particular page of the reading object; and

the virtual object comprises a holographic visual aid associated with the particular page.

9. The method of claim 1, wherein:

the particular object comprises a reading object; and

the 3D model includes a plurality of plates connected to a single axis hinge.

10. The method of claim 9, wherein:

the reading object comprises an augmentable book.

11. One or more storage devices containing processor readable code for programming one or more processors to perform a method for generating and displaying one or more virtual objects comprising the steps of:

identifying a predefined object within an environment, the predefined object is associated with an AR tag, the identifying is performed by a first mobile device;

acquiring a 3D model of the predefined object;

determining an orientation and a scale of the predefined object based on the 3D model;

acquiring an identification of the AR tag, the acquiring an identification includes aggregating a plurality of individual identity determinations from a plurality of mobile devices within the environment, the plurality of mobile devices includes a second mobile device different from the first mobile device, an individual identity determination corresponding with the second mobile device comprises the identification of the AR tag;

determining a virtual object corresponding with the AR tag based on the identification of the AR tag;

determining a first location associated with the virtual object based on the orientation and the scale of the predefined object; and

displaying at the first mobile device the virtual object such that the virtual object is perceived to exist at a point in space corresponding with the first location.

12. The one or more storage devices of claim 11, wherein:

the predefined object comprises a reading object; and

the determining a first location associated with the virtual object includes determining a location of a cover tag associated with the reading object.

13. The one or more storage devices of claim 11, wherein:

the determining an orientation and a scale of the predefined object includes determining the orientation of the predefined object within the environment relative to the first mobile device and determining a size of the predefined object within the environment.

14. The one or more storage devices of claim 11, wherein:

the predefined object comprises a reading object; and

the virtual object comprises a holographic visual aid associated with the reading object.

15. The one or more storage devices of claim 11, wherein:

the predefined object comprises a reading object;

the AR tag is associated with a particular page of the reading object; and

the virtual object comprises a holographic visual aid associated with the particular page.

16. The one or more storage devices of claim 11, wherein:

the predefined object comprises a reading object; and

the 3D model includes a plurality of plates connected to a single axis hinge.

17. The one or more storage devices of claim 16, wherein:

the reading object comprises an augmentable book.

18. An electronic device for generating and displaying one or more virtual objects, comprising:

a memory, the memory stores a 3D model of a particular object, the particular object is associated with one or more tags;

one or more processors in communication with the memory, the one or more processors identify the particular object within an environment, the one or more processors acquire an identification of a first tag of the one or more tags from a second mobile device different from the electronic device, the one or more processors determine a virtual object corresponding with the first tag based on the identification of the first tag, the one or more processors determine a first location associated with the virtual object relative to the 3D model; and

a see-through display in communication with the one or more processors, the see-through display displays one or more images of the virtual object such that the virtual object is perceived to exist at a point in space corresponding with the first location.

19. The electronic device of claim 18, wherein:

the one or more processors acquire the identification of the first tag by aggregating a plurality of individual identity determinations from a plurality of mobile devices within the environment, the plurality of mobile devices includes the second mobile device, an individual identity determination corresponding with the second mobile device comprises the identification of the first tag.

20. The electronic device of claim 19, wherein:

the particular object comprises a reading object;

the one or more processors determine a first location by determining a location of a cover tag associated with the reading object; and

the virtual object comprises a holographic visual aid associated with the reading object.