3D POSITIONING OF AUGMENTED REALITY INFORMATION

• US 20130278631A1
• Filed: 08/21/2012
• Published: 10/24/2013
• Est. Priority Date: 02/28/2010
• Status: Abandoned Application

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Osterhout Design Group
• Inventors: Border, John N., Haddick, John D.
• Application Number:
  US13/591,139
• Publication Number:
  US 20130278631A1
• Time in Patent Office: Days
• US Class Current: 345/633
• CPC Class Codes: G02B 2027/0123 comprising devices increasing the field of view
• CPC Class Codes: G02B 2027/0127 comprising devices increasing the depth of field
• CPC Class Codes: G02B 2027/0138 comprising image capture systems, e.g. camera
• CPC Class Codes: G02B 2027/014 comprising information/image processing systems
• CPC Class Codes: G02B 2027/0178 Eyeglass type eyeglass details G02C
• CPC Class Codes: G02B 27/0093 with means for monitoring data relating to the user, e.g. head-tracking, eye-tracking
• CPC Class Codes: G02B 27/017 Head mounted
• CPC Class Codes: G02C 5/143 having special ear pieces
• CPC Class Codes: G06F 1/163 Wearable computers, e.g. on a belt
• CPC Class Codes: G06F 2203/0331 Finger worn pointing device
• CPC Class Codes: G06F 3/005 Input arrangements through a video camera
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/013 Eye tracking input arrangements G06F3/015 takes precedence
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/04842 Selection of displayed objects or displayed text elements G06F3/0482 takes precedence
• CPC Class Codes: G06F 3/14 Digital output to display device ; Cooperation and interconnection of the display device with other functional units
• CPC Class Codes: G06Q 30/02 Marketing; Price estimation or determination; Fundraising
• CPC Class Codes: G09G 2360/144 the light being ambient light
• CPC Class Codes: G09G 3/001 using specific devices not provided for in groups G09G3/02 - G09G3/36, e.g. using an intermediate record carrier such as a film slide; Projection systems; Display of non-alphanumerical information, solely or in combination with alphanumerical information, e.g. digital display on projected diapositive as background slide projectors per se G03B23/00 = 42 HP
• CPC Class Codes: G09G 3/342 using several illumination sources separately controlled corresponding to different display panel areas, e.g. along one dimension such as lines
• CPC Class Codes: G09G 3/346 : based on modulation of the reflection angle, e.g. micromirrors micromirrors devices per se G02B26/0833
• CPC Class Codes: H04N 9/3194 : including sensor feedback

Abstract

A system and method for providing informational labels with perceived depth in the field of view of a user of a head mounted display device. In one embodiment, the method includes determining a physical location of the user and the head mounted display device, and identifying and determining a distance from the user to one or more objects of interest in the user'"'"'s field of view. Using the distance from the user for each object, one can calculate a disparity value for viewing each object. The processor of the head mounted device may gather information concerning each of the objects in which the user is interested. The head mounted display device then provides a label for each of the objects and for each eye of the user, and, using the disparity values, places the labels within the field of view of the user.

Claims

1. A method for providing informational labels with perceived depth in the field of view of a user of a head mounted display device comprising the steps of:

providing a head mounted display device having a right see-through display for exhibiting displayed content to the user'"'"'s right eye and a left see-through display for exhibiting displayed content to the user'"'"'s left eye;

determining the physical location of the head mounted display device;

identifying one or more objects of interest relative to the user'"'"'s field of view by referencing the determined location;

determining the distance from the determined location to each of the one or more objects of interest;

determining a disparity value for each of the one or more objects of interest by referencing the determined distance the particular object of interest is from the determined location;

determining labeling information for at least one of the one or more objects of interest; and

exhibiting a first copy of the labeling information on the right see-through display and a second copy of the labeling information on the left see-through display, wherein the locations on the right and left see-through displays that the first and second copies are respectively exhibited are determined by referencing the disparity value for the object of interest to which the labeling information corresponds so as to provide the user with a perceived depth of view for the labeled information that corresponds to the determined distance the object of interest is from the determined location.

2. The method of claim 1, wherein the head mounted display device includes electronics and wherein the step of identifying one or more of the objects of interest is performed at least in part by the electronics.

3. The method of claim 1, wherein the step of identifying one or more objects of interest is performed at least in part by a device that is in communication with the head mounted display device.

4. The method of claim 1, wherein the step of determining labeling information is performed at least in part by a device that is in communication with the head mounted display device.

5. The method of claim 1, wherein the step of determining the distance from the determined location to each of the one or more objects of interest is performed at least in part by a device that is in communication with the head mounted display device.

6. A method for providing informational labels with perceived depth in the field of view of a user of a head mounted display device comprising the steps of:

providing a head mounted display device having a right see-through display for exhibiting displayed content to the user'"'"'s right eye and a left see-through display for exhibiting displayed content to the user'"'"'s left eye;

determining the physical location of the head mounted display device;

determining a view direction for the head mounted display device;

identifying one or more objects of interest relative to the user'"'"'s field of view by referencing the determined location;

determining the distance from the determined location to each of the one or more objects of interest;

determining a disparity value for each of the one or more objects of interest by referencing the determined distance the particular object of interest is from the determined location;

determining labeling information for at least one of the one or more objects of interest; and

exhibiting a first copy of the labeling information on the right see-through display and a second copy of the labeling information on the left see-through display, wherein the locations on the right and left see-through displays that the first and second copies are respectively exhibited are determined by referencing the determined view direction and the disparity value for the object of interest to which the labeling information corresponds so as to provide the user with a perceived depth of view for the labeled information that corresponds to the determined distance the object of interest is from the determined location.

7. The method of claim 6, wherein the head mounted display device includes electronics and wherein the step of identifying one or more of the objects of interest is performed at least in part by the electronics.

8. The method of claim 6, wherein the step of identifying one or more objects of interest is performed at least in part by a device that is in communication with the head mounted display device.

9. The method of claim 6, wherein the step of determining the view direction is performed at least in part by a device that is in communication with the head mounted display device.

10. The method of claim 6, wherein the step of determining labeling information is performed at least in part by a device that is in communication with the head mounted display device.

11. The method of claim 6, wherein the step of determining the distance from the determined location to each of the one or more objects of interest is performed at least in part by a device that is in communication with the head mounted display device.

12. A method for providing informational labels with perceived depth in the field of view of a user of a head mounted display device comprising the steps of:

providing a head mounted display device having a right see-through display for exhibiting displayed content to the user'"'"'s right eye and a left see-through display for exhibiting displayed content to the user'"'"'s left eye;

capturing one or more images of the user'"'"'s surrounding environment;

analyzing at least one of the captured images to identify one or more objects of interest;

determining the distance from the head mounted display device to each of the one or more objects of interest;

determining a disparity value for each of the one or more objects of interest by referencing the determined distance the particular object of interest is from the head mounted display device;

determining labeling information for at least one of the one or more objects of interest; and

exhibiting a first copy of the labeling information on the right see-through display and a second copy of the labeling information on the left see-through display, wherein the locations on the right and left see-through displays that the first and second copies are respectively exhibited are determined by referencing the disparity value for the object of interest to which the labeling information corresponds so as to provide the user with a perceived depth of view for the labeled information that corresponds to the determined distance the object of interest is from the head mounted display device.

13. The method of claim 12, wherein the step of determining the distance from the head mounted display device to each of the one or more objects of interest is performed at least in part by using a rangefinder device.

14. The method of claim 12, wherein the head mounted display device includes electronics and wherein the step of analyzing at least one of the captured images to identify one or more objects of interest is performed at least in part by the electronics.

15. The method of claim 12, wherein the step of analyzing at least one of the captured images to identify one or more objects of interest is performed at least in part by a device that is in communication with the head mounted display device.

16. The method of claim 12, wherein at least one of the images that are captured is one selected from the group of a visible light image, an ultraviolet light image, an infrared light image, and a hyperspectral light image.

17. A method for providing informational labels with perceived depth in the field of view of a user of a head mounted display device comprising the steps of:

providing a head mounted display device having a right see-through display for exhibiting displayed content to the user'"'"'s right eye and a left see-through display for exhibiting displayed content to the user'"'"'s left eye;

capturing one or more stereo image sets of the user'"'"'s surrounding environment;

analyzing at least one of the captured stereo image sets to identify one or more objects of interest;

determining the distance from the head mounted display device to each of the one or more objects of interest;

determining a disparity value for each of the one or more objects of interest by comparing to one another the images of at least one of the captured stereo image sets;

determining labeling information for at least one of the one or more objects of interest; and

exhibiting a first copy of the labeling information on the right see-through display and a second copy of the labeling information on the left see-through display, wherein the locations on the right and left see-through displays that the first and second copies are respectively exhibited are determined by referencing the disparity value for the object of interest to which the labeling information corresponds so as to provide the user with a perceived depth of view for the labeled information that corresponds to the determined distance the object of interest is from the head mounted display device.

18. The method of claim 17, wherein the step of determining labeling information is performed at least in part by a device that is in communication with the head mounted display device.

19. The method of claim 17, wherein the head mounted display device includes electronics and wherein the step of analyzing at least one of the captured stereo images to identify one or more objects of interest is performed at least in part by the electronics.

20. The method of claim 17, wherein the step of analyzing at least one of the captured stereo images to identify one or more objects of interest is performed at least in part by a device that is in communication with the head mounted display device.

21. The method of claim 17, wherein the images of at least one of the captured stereo image sets is one selected from the group of a visible light image, an ultraviolet light image, an infrared light image, and a hyperspectral light image.