Management of content in a 3D holographic environment

• US 9,508,195 B2
• Filed: 09/03/2014
• Issued: 11/29/2016
• Est. Priority Date: 09/03/2014
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: Heuvel, Joseph Van Den, Ouellette, Ken, Sugden, Ben, Omotani, Todd, Tanner, Marcus, Alt, Greg, Hale, Michael, Sutherland, Jeff
• Primary Examiner(s): Faulk, Devona
• Assistant Examiner(s): BEARD, CHARLES LLOYD
• Application Number:
  US14/476,642
• Publication Number:
  US 20160063762A1
• Time in Patent Office: 818 Days
• US Class Current: 1/1
• CPC Class Codes: G02B 2027/0138 comprising image capture systems, e.g. camera
• CPC Class Codes: G02B 2027/014 comprising information/image processing systems
• CPC Class Codes: G02B 2027/0178 Eyeglass type eyeglass details G02C
• CPC Class Codes: G02B 27/0093 with means for monitoring data relating to the user, e.g. head-tracking, eye-tracking
• CPC Class Codes: G02B 27/0172 characterised by optical features
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/147 using display panels
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: G06T 7/70 Determining position or orientation of objects or cameras camera calibration G06T7/80
• CPC Class Codes: G09G 5/14 Display of multiple viewports

Abstract

Methods for managing content within an interactive augmented reality environment are described. An augmented reality environment may be provided to an end user of a head-mounted display device (HMD) in which content (e.g., webpages) may be displayed to the end user using one or more curved slates that are positioned on a virtual cylinder that appears body-locked to the end user. The virtual cylinder may be located around the end user with the end user positioned in the middle of the virtual cylinder such that the one or more curved slates appear to be displayed at the same distance from the end user. The position and size of each of the one or more curved slates may be controlled by the end user using head gestures and a virtual pointer projected onto the virtual cylinder.

Claims

1. An electronic device that provides an interactive augmented reality environment, comprising:

a see-through display; and

one or more processors configured to acquire a first set of content and a second set of content, the one or more processors configured to determine a first GPS location of the electronic device at a first point in time, the one or more processors configured to determine a first radius for a first virtual cylinder that surrounds an end user of the electronic device based on the first GPS location and determine a second radius for a second virtual cylinder that surrounds the end user of the electronic device based on the first GPS location, wherein the first radius is greater than the second radius, the one or more processors configured to adjust the first radius for the first virtual cylinder and the second radius for the second virtual cylinder based on a second GPS location of the electronic device in response to determining the second GPS location of the electronic device at a second point in time subsequent to the first point in time, wherein the second radius is greater than the first radius, the see-through display configured to display a first set of content within a portion of the first virtual cylinder at the adjusted first radius such that the first set of content appears to be body-locked to the end user of the electronic device and display a second set of content within a portion of the second virtual cylinder at the adjusted second radius such that the second set of content appears to be body-locked to the end user of the electronic device.

2. The electronic device of claim 1, wherein:

the one or more processors configured to determine the first radius for the first virtual cylinder based on a number of curved slates that are displayed within the first virtual cylinder.

3. The electronic device of claim 1, wherein:

the one or more processors configured to generate one or more other images associated with a second curved slate using the second set of content, the one or more processors configured to cause the one or more other images to be displayed using the see-through display such that the second curved slate appears to be body-locked to the end user of the electronic device.

4. The electronic device of claim 1, wherein:

the one or more processors configured to dynamically adjust the first radius of the first virtual cylinder and the second radius of the second virtual cylinder based on a GPS location of the electronic device.

5. The electronic device of claim 1, wherein:

the one or more processors configured to detect a repositioning of a first curved slate by the end user of the electronic device and automatically reposition a second curved slate displayed to the end user of the electronic device in response to detecting the repositioning of the first curved slate.

6. The electronic device of claim 5, wherein:

the one or more processors configured to assign the second curved slate to the second virtual cylinder in response to detecting the repositioning of the first curved slate.

7. The electronic device of claim 1, wherein:

the one or more processors configured to determine an axis for the first virtual cylinder using a gravity vector generated by a second mobile device that is attached to a torso of the end user; and

the electronic device comprise a head-mounted display device.

8. A method for managing content within an interactive augmented reality environment using a mobile device, comprising:

determining a first set of content and a second set of content to be displayed using the mobile device;

determining a first GPS location of the mobile device at a first point in time;

determining a first radius for a first virtual cylinder that surrounds an end user of the mobile device based on the first GPS location of the mobile device and determining a second radius for a second virtual cylinder that surrounds the end user of the mobile device based on the first GPS location of the mobile device, wherein the first radius is greater than the second radius;

adjusting the first radius for the first virtual cylinder and the second radius for the second virtual cylinder based on a second GPS location of the mobile device in response to determining the second GPS location of the mobile device at a second point in time subsequent to the first point in time, wherein the second radius is greater than the first radius; and

displaying the first set of content within a portion of the first virtual cylinder at the adjusted first radius such that the first set of content appears to be body-locked to the end user of the mobile device and displaying the second set of content within a portion of the second virtual cylinder at the adjusted second radius such that the second set of content appears to be body-locked to the end user of the mobile device.

9. The method of claim 8, wherein:

the determining the first radius for the first virtual cylinder includes determining the first radius for the first virtual cylinder based on a number of curved slates that are displayed within the first virtual cylinder.

10. The method of claim 8, wherein:

the determining the first set of content includes determining the first set of content based on the first GPS location.

11. The method of claim 8, further comprising:

generating one or more other images associated with a second curved slate using the second set of content; and

displaying the one or more other images using the mobile device such that the second curved slate appears to be located within the second portion of the second virtual cylinder and the second curved slate appears to be body-locked to the end user of the mobile device.

12. The method of claim 8, further comprising:

dynamically adjusting the first radius of the first virtual cylinder based on a number of slate overlaps of curved slates that are displayed within the first virtual cylinder.

13. The method of claim 8, further comprising:

generating one or more images associated with a first curved slate using the first set of content;

displaying the one or more images using the mobile device such that the first curved slate appears to be located within a portion of the first virtual cylinder and the first curved slate appears to be body-locked to the end user of the mobile device;

detecting a repositioning of the first curved slate by the end user of the mobile device; and

automatically repositioning a second curved slate displayed to the end user of the mobile device in response to detecting the repositioning of the first curved slate.

14. The method of claim 13, wherein:

the automatically repositioning a second curved slate includes assigning the second curved slate from the first virtual cylinder to the second virtual cylinder.

15. The method of claim 13, wherein:

the detecting a repositioning of the first curved slate includes detecting a head gesture performed by the end user of the mobile device.

16. The method of claim 8, further comprising:

determining an axis for the first virtual cylinder based on a gravity vector generated by a second mobile device that is attached to a torso of the end user.

17. The method of claim 8, further comprising:

determining an axis for the first virtual cylinder, the mobile device includes a body facing camera, the determining the axis for the first virtual cylinder includes determining the axis based on images of a torso of the end user captured using the body facing camera.

18. The method of claim 8, wherein:

the mobile device comprises a head-mounted display device.

19. One or more storage devices containing processor readable code for programming one or more processors to perform a method for managing content within an interactive augmented reality environment using a mobile device comprising the steps of:

determining a first GPS location of the mobile device at a first point in time;

determining a first radius for a first virtual cylinder that surrounds an end user of the mobile device based on the first GPS location of the mobile device and determining a second radius for a second virtual cylinder that surrounds the end user of the mobile device based on the first GPS location of the mobile device, wherein the first radius is greater than the second radius;

adjusting the first radius for the first virtual cylinder and the second radius for the second virtual cylinder based on a second GPS location of the mobile device in response to determining the second GPS location of the mobile device at a second point in time subsequent to the first point in time, wherein the second radius is greater than the first radius; and

displaying a first set of content within a portion of the first virtual cylinder at the adjusted first radius such that the first set of content appears to be body-locked to the end user of the mobile device and displaying a second set of content within a portion of the second virtual cylinder at the adjusted second radius such that the second set of content appears to be body-locked to the end user of the mobile device.