Interactions of virtual objects with surfaces

• US 9,530,252 B2
• Filed: 01/25/2016
• Issued: 12/27/2016
• Est. Priority Date: 05/13/2013
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: Poulos, Adam G., Keibler, Evan Michael, Tomlin, Arthur, Brown, Cameron, McCulloch, Daniel, Mount, Brian, Kroymann, Dan, Alt, Gregory Lowell
• Primary Examiner(s): Yang, Ryan R
• Application Number:
  US15/005,530
• Publication Number:
  US 20160155270A1
• Time in Patent Office: 337 Days
• Field of Search: None
• US Class Current: 1/1
• CPC Class Codes: G02B 2027/0112 comprising device for genereting colour display
• CPC Class Codes: G02B 2027/0138 comprising image capture systems, e.g. camera
• CPC Class Codes: G02B 2027/014 comprising information/image processing systems
• CPC Class Codes: G02B 2027/0178 Eyeglass type eyeglass details G02C
• CPC Class Codes: G02B 2027/0187 slaved to motion of at least a part of the body of the user, e.g. head, eye
• CPC Class Codes: G02B 27/017 Head mounted
• CPC Class Codes: G02B 27/0172 characterised by optical features
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/013 Eye tracking input arrangements G06F3/015 takes precedence
• CPC Class Codes: G06F 3/147 using display panels
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: G09G 2340/0464 Positioning

Abstract

Embodiments are disclosed that relate to operating a user interface on an augmented reality computing device comprising a display system. For example, one disclosed embodiment includes displaying a virtual object via the display system as free-floating, detecting a trigger to display the object as attached to a surface, and, in response to the trigger, displaying the virtual object as attached to the surface via the display system. The method may further include detecting a trigger to detach the virtual object from the surface and, in response to the trigger to detach the virtual object from the surface, detaching the virtual object from the surface and displaying the virtual object as free-floating.

Claims

1. A method for operating a user interface of an augmented reality computing device comprising a display system, the method comprising:

identifying a surface via image data acquired with the augmented reality computing device;

identifying a discontinuity of the surface via the image data acquired with the augmented reality computing device;

comparing the discontinuity of the surface to a policy associated with a virtual object; and

based at least on comparing the discontinuity of the surface to the policy associated with the virtual object, displaying the virtual object entirely on one side of the discontinuity or displaying a different portion of the virtual object on each side of the discontinuity.

2. The method of claim 1, wherein the surface is a first surface, further comprising identifying a second surface via the image data, comparing the second surface to the policy associated with the virtual object, and not displaying the virtual object as attached to the second surface based upon comparing the second surface to the policy associated with the virtual object.

3. The method of claim 1, wherein the surface comprises a real-world surface.

4. The method of claim 1, further comprising updating displaying of the virtual object on the display system based on a change in a location of a gaze of a user.

5. The method of claim 1, further comprising:

detecting a trigger to detach the virtual object from the surface; and

in response to the trigger to detach the virtual object from the surface, detaching the virtual object from the surface and displaying the virtual object as free-floating.

6. The method of claim 1, further comprising:

detecting a location of user focus on the surface; and

displaying the virtual object on the surface at a location based upon the location of user focus.

7. The method of claim 6, further comprising detecting a change in the location of user focus and maintaining the virtual object attached at a fixed location.

8. The method of claim 6, further comprising detecting a change in the location of user focus, and in response detaching the virtual object from the surface at the discontinuity and then reattaching the virtual object at a location past the discontinuity.

9. The method of claim 6, further comprising detecting a change in the location of user focus, and displaying the virtual object on one side of the discontinuity or another side of the discontinuity based at least upon an angle between a direction of user focus after the change in the location and the surface meeting a threshold condition.

10. The method of claim 1, wherein the policy specifies one or more of a color and a texture of a surface to which the virtual object can attach.

11. The method of claim 1, wherein the policy specifies one or more of size, a location, and an orientation of a surface to which the virtual object can attach.

12. A method for operating a user interface of an augmented reality computing device comprising a see-through display system, the method comprising:

identifying a surface of a first orientation and a surface of a second, different orientation via image data acquired with the augmented reality computing device;

based at least on comparing the surface to a policy associated with the virtual object, displaying the virtual object as attached to the surface of the first orientation when the virtual object is closer than a threshold distance from the surface of the first orientation, and not displaying the virtual object as attached to the surface of the second orientation when the virtual object is closer than the threshold distance from the surface of the second orientation.

13. The method of claim 12, wherein the policy specifies one or more of which real-world surfaces the virtual object may be attached to and under what conditions the virtual object may be attached to a real-world surface, and displaying the virtual object on the see-through display system based on the one or more policies associated with the virtual object.

14. The method of claim 12, further comprising, after displaying the virtual object attached at the fixed location on the first real-world surface on the see-through display system, detecting a second change in location of user focus on the first real-world surface and maintaining the virtual object attached at the fixed location on the first real-world surface.

15. The method of claim 12, further comprising:

detecting a trigger to detach the virtual object from the first real-world surface; and

in response to the trigger to detach the virtual object from the first real-world surface, detaching the virtual object from the first real-world surface and displaying the virtual object as free-floating.

16. A wearable augmented reality computing device, comprising:

a see-through display device;

a sensor system including one or more sensors;

a logic subsystem; and

a storage subsystem comprising instructions stored thereon that are executable by the logic subsystem to

identify a surface via image data acquired with the augmented reality computing device;

identify a discontinuity of the surface via the image data acquired with the augmented reality computing device

compare the discontinuity of the surface to a policy associated with a virtual object; and

based at least on comparing the discontinuity of the surface to the policy associated with the virtual object, display the virtual object entirely on one side of the discontinuity or display a different portion of the virtual object on each side of the discontinuity.

17. The device of claim 16, wherein the policy specifies one or more of a color and a texture of a surface to which the virtual object can attach.

18. The device of claim 16, wherein the policy specifies one or more of size, a location, and an orientation of a surface to which the virtual object can attach.