Interactions of virtual objects with surfaces

• US 9,245,388 B2
• Filed: 05/13/2013
• Issued: 01/26/2016
• Est. Priority Date: 05/13/2013
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: Keibler, Evan Michael, Tomlin, Arthur, Brown, Cameron, McCulloch, Daniel, Mount, Brian, Kroymann, Dan, Alt, Gregory Lowell, Poulos, Adam G
• Primary Examiner(s): Yang, Ryan R
• Application Number:
  US13/893,146
• Publication Number:
  US 20140333666A1
• Time in Patent Office: 988 Days
• Field of Search: None
• US Class Current: 1/1
• CPC Class Codes: G02B 2027/0112 comprising device for genereting colour display
• CPC Class Codes: G02B 2027/0138 comprising image capture systems, e.g. camera
• CPC Class Codes: G02B 2027/014 comprising information/image processing systems
• CPC Class Codes: G02B 2027/0178 Eyeglass type eyeglass details G02C
• CPC Class Codes: G02B 2027/0187 slaved to motion of at least a part of the body of the user, e.g. head, eye
• CPC Class Codes: G02B 27/017 Head mounted
• CPC Class Codes: G02B 27/0172 characterised by optical features
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/013 Eye tracking input arrangements G06F3/015 takes precedence
• CPC Class Codes: G06F 3/147 using display panels
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: G09G 2340/0464 Positioning

Abstract

Embodiments are disclosed that relate to operating a user interface on an augmented reality computing device comprising a display system. For example, one disclosed embodiment includes displaying a virtual object via the display system as free-floating, detecting a trigger to display the object as attached to a surface, and, in response to the trigger, displaying the virtual object as attached to the surface via the display system. The method may further include detecting a trigger to detach the virtual object from the surface and, in response to the trigger to detach the virtual object from the surface, detaching the virtual object from the surface and displaying the virtual object as free-floating.

Claims

1. A method for operating a user interface of an augmented reality computing device comprising a display system, the method comprising:

displaying a virtual object via the display system as free-floating;

detecting a trigger to display the object as attached to a surface, wherein the trigger comprises a threshold distance between the virtual object and the surface; and

in response to the trigger, displaying the virtual object as attached to the surface via the display system.

2. The method of claim 1, wherein the surface comprises a real-world surface.

3. The method of claim 1, further comprising updating displaying of the virtual object on the display system based on a change in a location of a gaze of a user.

4. The method of claim 1, further comprising obtaining one or more policies associated with the virtual object, where the one or more policies dictate one or more of which surfaces the virtual object may be attached to and under what conditions the virtual object may be attached to a surface, and displaying the virtual object on the display system based on the one or more policies associated with the virtual object.

5. The method of claim 1, further comprising:

detecting a trigger to detach the virtual object from the surface; and

in response to the trigger to detach the virtual object from the surface, detaching the virtual object from the surface and displaying the virtual object as free-floating.

6. The method of claim 1, further comprising:

detecting a location of user focus on the surface; and

displaying the virtual object on the surface at a location based upon the location of user focus.

7. The method of claim 6, wherein the surface is a first surface, and further comprising after displaying the virtual object as attached to the first surface via the display system, detecting a change in the location of user focus on the first surface and maintaining the virtual object attached at a fixed location on the first surface.

8. The method of claim 7, further comprising detecting a discontinuity on the first surface at a location of user focus, and detaching the virtual object from the first surface at the discontinuity and then reattaching the virtual object at a location past the discontinuity.

9. The method of claim 7, further comprising detecting a discontinuity between the first surface and a second surface, and displaying a first portion of the virtual object on the first surface adjacent to the discontinuity and a second portion of the virtual object on the second surface adjacent to the discontinuity.

10. The method of claim 7, further comprising detecting a discontinuity between the first surface and a second surface, and in response to an angle between a direction of user focus and the second surface meeting a threshold condition, displaying the virtual object on the second surface and not displaying the virtual object on the first surface.

11. The method of claim 1, wherein the virtual object comprises a holographic television display.

12. The method of claim 1, wherein the virtual object comprises a holographic robotic assistant.

13. The method of claim 1, wherein the virtual object comprises a holographic control interface configured to change one or more properties of a local environment.

14. On an augmented reality computing device comprising a see-through display system, a method for operating a user interface, the method comprising:

detecting a location of user focus on a first real-world surface;

displaying on the see-through display system a virtual object at the location of user focus on the first real-world surface;

detecting a change in location of user focus to a new location on the first real-world surface;

displaying on the see-through display system the virtual object at the new location of user focus on the first real-world surface;

detecting a trigger to attach the virtual object to the first real-world surface, wherein the trigger comprises a threshold distance between the virtual object and the real-world surface; and

in response to the trigger, displaying on the see-through display system the virtual object attached at a fixed location on the first real-world surface that does not change based upon changes in the location of user focus.

15. The method of claim 14, further comprising obtaining one or more policies associated with the virtual object, where the one or more policies dictate one or more of which real-world surfaces the virtual object may be attached to and under what conditions the virtual object may be attached to a real-world surface, and displaying the virtual object on the see-through display system based on the one or more policies associated with the virtual object.

16. The method of claim 14, further comprising, after displaying the virtual object attached at the fixed location on the first real-world surface on the see-through display system, detecting a second change in location of user focus on the first real-world surface and maintaining the virtual object attached at the fixed location on the first real-world surface.

17. The method of claim 14, further comprising:

detecting a trigger to detach the virtual object from the first real-world surface; and

in response to the trigger to detach the virtual object from the first real-world surface, detaching the virtual object from the first real-world surface and displaying the virtual object as free-floating.

18. A wearable augmented reality computing device, comprising:

a see-through display device;

a sensor system including one or more sensors;

a logic subsystem; and

a storage subsystem comprising instructions stored thereon that are executable by the logic subsystem to

detect a direction of user focus;

display on the see-through display device a free-floating virtual object based upon the direction of user focus;

detect a change in direction of user focus;

display on the see-through display device the free-floating virtual object in a different location based upon the change in direction of user focus;

detect a trigger to attach the free-floating object to a real-world surface;

in response to the trigger, display on the see-through display device the virtual object attached to a real-world surface at a location of user focus on the real-world surfaces;

detect a discontinuity on the real-world surface at the location of user focus; and

change the display of the free-floating virtual object according to the discontinuity.

19. The device of claim 18, wherein the real-world surface is a first real-world surface, and wherein

changing the display of the virtual object includes detaching the virtual object from the first real-world surface at the discontinuity and then reattaching the virtual object to the real-world surface at a location past the discontinuity.

20. The device of claim 18, wherein the real-world surface is a first real-world surface, and wherein

changing the display of the virtual object includes displaying a first portion of the virtual object as attached to the first real-world surface and a second portion of the virtual object as attached to the second real-world surface adjacent to the discontinuity.