MECHANISM TO GIVE HOLOGRAPHIC OBJECTS SALIENCY IN MULTIPLE SPACES

• US 20140002442A1
• Filed: 06/29/2012
• Published: 01/02/2014
• Est. Priority Date: 06/29/2012
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Robert L. Crocco Jr
• Inventors: Lamb, Mathew J., Sugden, Ben J., Crocco, Robert L. Jr., Keane, Brian E., Miles, Christopher E., Perez, Kathryn Stone, Massey, Laura K., Kipman, Alex Aben-Athar
• Granted Patent: US 9,317,971 B2
• Time in Patent Office: Days
• US Class Current: 345/419
• CPC Class Codes: G02B 2027/014 comprising information/image processing systems
• CPC Class Codes: G02B 2027/0174 holographic
• CPC Class Codes: G02B 27/0172 characterised by optical features
• CPC Class Codes: G06F 3/013 Eye tracking input arrangements G06F3/015 takes precedence
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/1431 using a single graphics controller
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: G09G 2370/022 Centralised management of display operation, e.g. in a server instead of locally
• CPC Class Codes: G09G 2380/14 Electronic books and readers
• CPC Class Codes: G09G 3/003 to produce spatial visual effects
• CPC Class Codes: H04N 2213/008 Aspects relating to glasses for viewing stereoscopic images

Abstract

A system for allowing a virtual object to interact with other virtual objects across different spaces within an augmented reality (AR) environment and to transition between the different spaces is described. An AR environment may include a plurality of spaces, each comprising a bounded area or volume within the AR environment. In one example, an AR environment may be associated with a three-dimensional world space and a two-dimensional object space corresponding with a page of a book within the AR environment. A virtual object within the AR environment may be assigned to the object space and transition from the two-dimensional object space to the three-dimensional world space upon the detection of a space transition event. In some cases, a dual representation of the virtual object may be used to detect interactions between the virtual object and other virtual objects in both the world space and the object space.

Claims

1. A method for generating and displaying one or more virtual objects, comprising:

identifying one or more real objects within an environment, the identifying is performed by a mobile device, the environment is associated with a world space, the one or more real objects include a first real object associated with a first space different from the world space;

acquiring a virtual object associated with the environment;

assigning the virtual object to the first space;

detecting a space transition event for the virtual object;

assigning the virtual object to the world space in response to the detecting a space transition event;

determining a location for the virtual object relative to the world space; and

displaying at the mobile device the virtual object such that the virtual object is perceived to exist at a point in space corresponding with the location.

2. The method of claim 1, wherein:

the first space comprises a two-dimensional space; and

the world space comprises a three-dimensional space.

3. The method of claim 1, further comprising:

detecting an interaction with the virtual object, the detecting an interaction includes generating a dual representation for the virtual object.

4. The method of claim 1, wherein:

the detecting a space transition event includes detecting that the virtual object has reached a boundary of the first space.

5. The method of claim 1, wherein:

the detecting a space transition event includes detecting that a space triggering event associated with the virtual object has occurred.

6. The method of claim 5, wherein:

the detecting that a space triggering event associated with the virtual object has occurred includes detecting that an end user of the mobile device has gazed at the virtual object for at least a particular period of time.

7. The method of claim 5, wherein:

the detecting that a space triggering event associated with the virtual object has occurred includes detecting that an end user of the mobile device has performed a particular gesture.

8. The method of claim 1, wherein:

the acquiring a virtual object includes acquiring the virtual object from a virtual object server; and

the detecting a space transition event is performed by the mobile device, the mobile device comprises a see-through HMD.

9. The method of claim 1, wherein:

the first real object comprises a reading object; and

the virtual object comprises a holographic visual aid associated with the reading object.

10. The method of claim 1, wherein:

the assigning the virtual object to the first space include determining the first space based on an AR tag associated with the first real object.

11. One or more storage devices containing processor readable code for programming one or more processors to perform a method for generating and displaying one or more virtual objects comprising the steps of:

identifying an environment associated with a mobile device, the environment corresponds with a world space;

detecting a first AR tag associated with a first real object within the environment, the first real object corresponds with a first space;

acquiring a virtual object associated with the first AR tag, the virtual object comprises a first portion of the virtual object and a second portion of the virtual object;

assigning the first portion of the virtual object to the first space;

assigning the second portion of the virtual object to the first space;

detecting a space transition event for the second portion of the virtual object;

assigning the second portion of the virtual object to the world space in response to the detecting a space transition event; and

displaying at the mobile device one or more images corresponding with the virtual object such that the first portion of the virtual object is perceived to be locked to the first space and the second portion of the virtual object is perceived to be locked to the world space.

12. The one or more storage devices of claim 11, wherein:

the first space comprises a two-dimensional space; and

the world space comprises a three-dimensional space.

13. The one or more storage devices of claim 11, wherein:

the detecting a space transition event includes detecting that the second portion of the virtual object has reached a boundary of the first space.

14. The one or more storage devices of claim 11, wherein:

the detecting a space transition event includes detecting that a space triggering event associated with the second portion of the virtual object has occurred.

15. The one or more storage devices of claim 14, wherein:

the detecting that a space triggering event associated with the second portion of the virtual object has occurred includes detecting that an end user of the mobile device has gazed at the second portion of the virtual object for at least a particular period of time.

16. The one or more storage devices of claim 14, wherein:

the detecting that a space triggering event associated with the second portion of the virtual object has occurred includes detecting that an end user of the mobile device has performed a particular gesture.

17. The one or more storage devices of claim 11, wherein:

the first real object comprises a reading object; and

the virtual object comprises a holographic visual aid associated with the reading object.

18. An electronic device for generating and displaying one or more virtual objects, comprising:

a memory, the memory stores a virtual object associated with an environment;

one or more processors in communication with the memory, the one or more processors identify one or more real objects within the environment, the environment is associated with a world space, the one or more real objects include a first real object associated with a first space different from the world space, the one or more processors assign the virtual object to the first space, the one or more processors detect a space transition event for the virtual object, the one or more processors assign the virtual object to the world space in response to detecting the space transition event, the one or more processors determine a location for the virtual object relative to the world space, the one or more processors generate one or more images of the virtual object corresponding with the location; and

a see-through display in communication with the one or more processors, the see-through display displays the one or more images of the virtual object such that the virtual object is perceived to exist at a point in space corresponding with the location.

19. The electronic device of claim 18, wherein:

the one or more processors detect an interaction with the virtual object by generating a dual representation for the virtual object;

the first space comprises a two-dimensional space; and

the world space comprises a three-dimensional space.

20. The electronic device of claim 19, wherein:

the one or more processors detect the space transition event by detecting that the virtual object has reached a boundary of the first space.