INDICATING OUT-OF-VIEW AUGMENTED REALITY IMAGES

• US 20140375683A1
• Filed: 06/25/2013
• Published: 12/25/2014
• Est. Priority Date: 06/25/2013
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Robert Crocco Jr
• Inventors: Sugden, Ben, Deptford, Daniel, Crocco, Robert Jr., Kipman, Alex, Kinnebrew, Peter Tobias, Kamuda, Nicholas, Scavezze, Michael, Hastings, Ryan, Brown, Cameron, Ambrus, Tony, Scott, Jason, Bevis, John, Fajt, Nicholas Gervase, Markovic, Relja, Latta, Stephen, McCulloch, Daniel, Salter, Thomas George, Keane, Brian, Massey, Laura, Quarles, Zachary, Kirschenbaum, Jamie B., Klucher, Michael
• Granted Patent: US 9,129,430 B2
• Time in Patent Office: Days
• US Class Current: 345/633
• CPC Class Codes: G02B 2027/0178 Eyeglass type eyeglass details G02C
• CPC Class Codes: G02B 27/0172 characterised by optical features
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/012 Head tracking input arrangements
• CPC Class Codes: G06F 3/013 Eye tracking input arrangements G06F3/015 takes precedence
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/04815 Interaction with a metaphor-based environment or interaction object displayed as three-dimensional, e.g. changing the user viewpoint with respect to the environment or object
• CPC Class Codes: G06F 3/167 Audio in a user interface, e.g. using voice commands for navigating, audio feedback
• CPC Class Codes: G06T 13/20 3D [Three Dimensional] animation
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00

Abstract

Embodiments are disclosed that relate to operating a user interface on an augmented reality computing device comprising a see-through display system. For example, one disclosed embodiment includes identifying one or more objects located outside a field of view of a user, and for each object of the one or more objects, providing to the user an indication of positional information associated with the object.

Claims

1. On an augmented reality computing device comprising a display system, a method for operating a user interface, the method comprising:

identifying one or more virtual objects located outside a field of view available for a display of augmented reality information; and

for each object of the one or more virtual objects, providing to the user an indication of positional information associated with the virtual object.

2. The method of claim 1, wherein the indication of positional information associated with the virtual object includes an indication of a position of the virtual object relative to a position of the user.

3. The method of claim 2, wherein the indication of positional information associated with the virtual object is displayed on a see-through display system within the field of view of a user adjacent to a periphery of the field of view.

4. The method of claim 2, wherein the indication of positional information associated with the virtual object is displayed on a see-through display system within the field of view of a user as a marker indicating a presence of the virtual object and a direction to turn to view the virtual object.

5. The method of claim 4, wherein the marker comprises a tendril extending from a location within the field of view to the virtual object.

6. The method of claim 4, further comprising varying a display of the marker based on one or more of a property of the virtual object, a distance from the virtual object to the user, and an orientation of the virtual object relative to the user.

7. The method of claim 6, wherein varying a display of the marker comprises one or more of varying a color of the marker, varying an opacity of the marker, varying a size of the marker, varying a shape of the marker, and varying a movement of the marker.

8. The method of claim 1, wherein the display system comprises a see-through display system, and wherein the indication of positional information associated with the virtual object is displayed on the see-through display system within the field of view.

9. The method of claim 1, wherein the indication of positional information associated with the virtual object comprises one or more sounds emitted from speakers indicating a presence of the virtual object.

10. The method of claim 9, further comprising varying the one or more sounds emitted from speakers based on one or more of a property of the virtual object, a density of objects, a distance from the virtual object to the user, and an orientation of the virtual object relative to the user.

11. The method of claim 10, wherein varying the one or more sounds emitted from speakers comprises one or more of varying a volume of the one or more sounds, varying one or more frequencies of the one or more sounds, varying a position of the one or more sounds, and varying a tactile response.

12. The method of claim 1, wherein the display system comprises a see-through display system, and wherein the indication of positional information associated with the virtual object is displayed on the see-through display system as a virtual map.

13. The method of claim 1, wherein the indication of positional information associated with the virtual object is provided in response to user input.

14. On an augmented reality computing device comprising a see-through display system, a method for operating a user interface, the method comprising:

identifying an object located outside a field of view of the see-through display system available for a display of augmented reality information; and

displaying via the see-through display system a tendril that leads from a location within the field of view toward the object, the tendril indicating a visual path to follow to locate the object.

15. The method of claim 14, wherein the tendril extends to the object.

16. The method of claim 14, further comprising animating an appearance of the tendril based upon user interaction with the object.

17. The method of claim 14, further comprising emitting a sound from one or more speakers to indicate creation of the tendril.

18. The method of claim 14, further comprising ceasing to display the tendril when the object enters the field of view.

19. The method of claim 18, further comprising varying a display of the tendril based on one or more of a property of the object, a distance from the object to the user, and an orientation of the object relative to the user, wherein varying a display of the tendril comprises one or more of varying a color of the tendril, varying an opacity of the tendril, varying a size of the tendril, varying a shape of the tendril, and varying a movement of the tendril.

20. A wearable augmented reality computing device, comprising:

a see-through display device;

a graphical user interface displayed via the see-through display device;

a sensor system including one or more sensors;

a logic subsystem; and

a storage subsystem comprising instructions stored thereon that are executable by the logic subsystem to;

detect one or more objects located outside a field of view of the see-through display device available for the display of augmented reality imagery; and

for each object of the one or more objects, providing indications to the user of presence and direction of the object located outside the field of view; and

vary the indications provided to the user based on one or more of a change in a location of the object relative to the see-through display device, a distance of the object from the see-through display device, and a property of the object.