AR glasses with event and sensor triggered AR eyepiece interface to external devices

• US 9,759,917 B2
• Filed: 01/03/2012
• Issued: 09/12/2017
• Est. Priority Date: 02/28/2010
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: Osterhout, Ralph F., Haddick, John D., Lohse, Robert Michael, Cella, Charles, Nortrup, Robert J., Nortrup, Edward H.
• Primary Examiner(s): Gaffin, Jeffrey A
• Assistant Examiner(s): HUERTAS TORRES, AYESHA E
• Application Number:
  US13/342,968
• Publication Number:
  US 20120206323A1
• Time in Patent Office: 2,079 Days
• Field of Search: 348 8
• CPC Class Codes: G02B 2027/0138 comprising image capture systems, e.g. camera
• CPC Class Codes: G02B 2027/014 comprising information/image processing systems
• CPC Class Codes: G02B 2027/0178 Eyeglass type eyeglass details G02C
• CPC Class Codes: G02B 2027/0187 slaved to motion of at least a part of the body of the user, e.g. head, eye
• CPC Class Codes: G02B 27/0093 with means for monitoring data relating to the user, e.g. head-tracking, eye-tracking
• CPC Class Codes: G02B 27/017 Head mounted
• CPC Class Codes: G06F 1/163 Wearable computers, e.g. on a belt
• CPC Class Codes: G06F 3/005 Input arrangements through a video camera
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/013 Eye tracking input arrangements G06F3/015 takes precedence
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06Q 30/02 Marketing; Price estimation or determination; Fundraising

Abstract

This disclosure concerns an interactive head-mounted eyepiece with an integrated processor for handling content for display and an integrated image source for introducing the content to an optical assembly through which the user views a surrounding environment and the displayed content, wherein the eyepiece includes event and sensor triggered interface to external devices.

Claims

1. A system comprising:

an interactive head-mounted device including an optical assembly configured to display virtual content and to enable viewing of at least a portion of a surrounding environment, an integrated processor for processing the virtual content for display, and an integrated image source for introducing the virtual content to the optical assembly; and

a sensor configured to detect a physical quantity of items as input, the physical quantity of items comprising a number of items to be transferred in an inventory system, wherein the interactive head-mounted device is configured to present, via the optical assembly, an augmented reality image comprising inventory tracking information regarding an identification of one or more items of the physical quantity of items when the physical quantity of items is detected via the sensor, and

wherein the augmented reality image comprises information regarding a location where the items of the physical quantity of items are to be transferred.

2. The system of claim 1 wherein the augmented reality image comprises an instruction for the physical quantity of items.

3. The system of claim 1 wherein the sensor comprises a facial actuation sensor.

4. The system of claim 1 wherein the sensor comprises a hand-mounted sensor.

5. The system of claim 1 wherein the sensor comprises an acoustic sensor, and wherein the physical quantity of items is detected via the acoustic sensor.

6. The system of claim 1 wherein the sensor comprises an image sensor, and wherein the physical quantity of items is detected via the image sensor.

7. The system of claim 1, wherein the interactive head-mounted device is further configured to present the augmented reality image comprising inventory tracking information regarding the physical quantity of items when the sensor detects one or more items of the physical quantity of items within a threshold distance of the sensor.

8. The system of claim 1, wherein the interactive head-mounted display device further comprises an audio output device configured to provide audio information regarding the physical quantity of items.

9. In a system having an interactive head-mounted device comprising an optical assembly, an integrated image source, an integrated processor, and a sensor, a method comprising:

introducing virtual content to the optical assembly via the integrated image source;

processing the virtual content for display via an integrated processor;

displaying via the optical assembly the virtual content while enabling viewing of at least a portion of a surrounding environment;

detecting via the sensor a physical quantity of items as input, the physical quantity of items comprising a number of items to be transferred in an inventory system; and

in response to detection of the physical quantity of items as input, presenting other virtual content via the integrated image source in the form of an augmented reality image comprising inventory tracking information regarding an identification of one or more items of the physical quantity of items,

wherein the augmented reality image comprises information regarding a location where the items of the physical quantity of items are to be transferred.

10. The method of claim 9, wherein the physical quantity of items is detected via one or more of an input of audio and an input of video.

11. The method of claim 9, wherein presenting other virtual content comprises presenting an instruction regarding disposing one or more items of the physical quantity of items in an inventory system.

12. The method of claim 9, wherein presenting other virtual content comprises presenting information regarding an identification of one or more items of the physical quantity of items.

13. The method of claim 9, wherein presenting other virtual content comprises presenting an instruction regarding transferring one or more items of the physical quantity of items in an inventory system.

14. The method of claim 13, wherein presenting an instruction regarding transferring the one or more items comprises presenting an instruction regarding a location where the one or more items are to be transferred.