Wearable computer with nearby object response

• US 9,081,177 B2
• Filed: 10/07/2011
• Issued: 07/14/2015
• Est. Priority Date: 10/07/2011
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Wong, Adrian, Miao, Xiaoyu
• Primary Examiner(s): Mengistu, Amare
• Assistant Examiner(s): MATHEWS, CRYSTAL
• Application Number:
  US13/267,963
• Publication Number:
  US 20130335301A1
• Time in Patent Office: 1,376 Days
• US Class Current: 1/1
• CPC Class Codes: G02B 2027/0138 comprising image capture systems, e.g. camera
• CPC Class Codes: G02B 2027/014 comprising information/image processing systems
• CPC Class Codes: G02B 2027/015 involving arrangement aiming to get less bulky devices
• CPC Class Codes: G02B 2027/0178 Eyeglass type eyeglass details G02C
• CPC Class Codes: G02B 27/0093 with means for monitoring data relating to the user, e.g. head-tracking, eye-tracking
• CPC Class Codes: G02B 27/017 Head mounted
• CPC Class Codes: G02B 27/0172 characterised by optical features
• CPC Class Codes: G02B 27/0176 characterised by mechanical features
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/013 Eye tracking input arrangements G06F3/015 takes precedence
• CPC Class Codes: G06F 3/04815 Interaction with a metaphor-based environment or interaction object displayed as three-dimensional, e.g. changing the user viewpoint with respect to the environment or object
• CPC Class Codes: G06F 3/0482 Interaction with lists of selectable items, e.g. menus
• CPC Class Codes: G06F 3/167 Audio in a user interface, e.g. using voice commands for navigating, audio feedback
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: G06T 2200/24 involving graphical user interfaces [GUIs]
• CPC Class Codes: G06V 20/10 Terrestrial scenes scenes under surveillance with static cameras G06V20/52; scenes perceived from the exterior of a vehicle G06V20/56; scenes perceived from the interior of a vehicle G06V20/59

Abstract

Exemplary methods and systems relate to detecting physical objects near a substantially transparent head-mounted display (HMD) system and activating a collision-avoidance action to alert a user of the detected objects. Detection techniques may include receiving data from distance and/or relative movement sensors and using this data as a basis for determining an appropriate collision-avoidance action. Exemplary collision-avoidance actions may include de-emphasizing virtual objects displayed on the HMD to provide a less cluttered view of the physical objects through the substantially transparent display and/or presenting new virtual objects.

Claims

1. A computer-implemented method comprising:

causing a head-mountable device (HMD) to display a user-interface mapped to a static sphere, wherein the user-interface comprises;

a view region; and

at least one content region located above the view region, wherein the at least one content region comprises a set of selectable content objects that are arranged along an at least partial ring above the view region, wherein the at least partial ring is substantially centered above the HMD, wherein the view region and the at least one content region are mapped to the sphere, and wherein a field of view provided in a see-through display of the HMD is moveable with respect to the sphere;

associating a forward-looking head position with a first location of the field of view within the sphere, wherein (a) the view region is substantially within the first location of the field of view provided in the see-through display of the HMD, and (b) each of the set of selectable content objects is above and substantially outside the first location of the field of view, wherein the field of view at the first location, which is associated with the forward-looking head position, is in a direction that is generally parallel to the ground;

displaying one or more virtual objects in the view region;

using data from one or more first sensors as a basis for determining a distance between the HMD and a physical object; and

using the determined distance between the HMD and the physical object as basis for initiating a collision-avoidance action, wherein the collision-avoidance action comprises de-emphasizing at least one of the displayed virtual objects so as to provide a less-obstructed view of the physical object through the see-through display of the HMD.

2. The method of claim 1, wherein using the determined distance as basis for initiating the collision-avoidance action comprises:

determining that the physical object is less than a threshold distance from the HMD; and

responsively initiating the collision-avoidance action.

3. The method of claim 1, further comprising:

using data from one or more second sensors to determine a relative movement of the physical object with respect to the HMD; and

using the relative movement as a further basis for initiating the collision-avoidance action.

4. The method of claim 3, wherein using the relative movement as a further basis for initiating the collision-avoidance action comprises:

using the relative movement as a basis for determining the threshold distance between the physical object and the HMD; and

determining that the physical object is less than the threshold distance from the HMD and responsively initiating the collision-avoidance action.

5. The method of claim 3, wherein using the relative movement as a further basis for initiating the collision-avoidance action comprises:

using (a) the distance between the physical object and the HMD and (b) the relative movement to determine a time period until collision; and

determining that the time period until collision is less than a threshold time period and responsively initiating the collision-avoidance action.

6. The method of claim 3, further comprising, subsequent to initiating the collision-avoidance action:

using data from the one or more second sensors as a basis for determining a subsequent relative movement of the physical object with respect to the HMD;

determining that a direction of the subsequent relative movement of the physical object is directed sufficiently away from the HMD such that collision with the physical object is unlikely; and

in response to determining that the subsequent relative movement of the physical object is directed sufficiently away from the HMD, re-displaying in the view region the at least one virtual object that was de-emphasized in the collision-avoidance action.

7. The method of claim 1, further comprising:

using data from one or more third sensors to determine that the HMD is in motion; and

using the determination that the HMD is in motion as a further basis for initiating a collision-avoidance action.

8. The method of claim 1, further comprising:

determining that the view region is sufficiently cluttered; and

using the determination that the view region is sufficiently cluttered as a further basis for initiating the collision-avoidance action.

9. The method of claim 1, wherein de-emphasizing at least one of the virtual objects comprises removing at least one of the virtual objects from the view region.

10. The method of claim 9, wherein the at least one removed object is moved to either:

(a) one of the content regions that is not fully visible in the field of view or (b) a history region that is located outside of the view region.

11. The method of claim 9, wherein all of the one or more objects are removed from the view region.

12. The method of claim 1, wherein the moving the at least one of said virtual objects comprises efficiently portraying a potential hazard by moving the at least one of said virtual objects quickly from a center point of the view region toward outer edges of the view region.

13. The method of claim 12, further comprising:

determining a portion of the view region that overlays the physical object;

wherein the at least one of said virtual objects is moved quickly away from the portion of the view region that overlays the physical object.

14. The method of claim 1, wherein the physical object is outside of the field of view provided in the see-through display, and wherein the collision-avoidance action further comprises displaying a visual indication of a direction of the physical object.

15. The method of claim 1, wherein the collision-avoidance action further comprises presenting an audio alert.

16. The method of claim 1, wherein the collision-avoidance action further comprises displaying a visual indication that highlights the physical object.

17. The method of claim 1, further comprising:

receiving user-input data that indicates to re-display the at least one virtual object that was de-emphasized in the collision-avoidance action; and

in response to at least the receiving the user-input data, re-displaying in the view region the at least one virtual object that was de-emphasized in the collision-avoidance action.

18. The method of claim 17, wherein the user-input data comprises head-movement data corresponding to a predetermined movement pattern.

19. A wearable computer comprising:

at least one processor; and

data storage comprising logic executable by the at least one processor to;

cause a head-mountable device (HMD) to display a user-interface mapped to a sphere, wherein the user-interface comprises a view region and at least one content region located above the view region, wherein the at least one content region comprises a set of selectable content objects that are arranged along an at least partial ring above the view region, wherein the at least partial ring is substantially centered above the HMD, wherein the view region and the at least one content region are mapped to the sphere, and wherein a field of view provided in a see-through display of the HMD is moveable with respect to the sphere;

associate a forward-looking head position with a first location of the field of view within the sphere, wherein (a) the view region is substantially within the first location of the field of view provided in the see-through display of the HMD, and (b) each of the set of selectable content objects is above and substantially outside the first location of the field of view, wherein the field of view at the first location, which is associated with the forward-looking head position, is in a direction that is generally parallel to the ground;

display one or more virtual objects in the view region;

use data from one or more first sensors as a basis for determining a distance between the HMD and a physical object; and

use the determined distance between the HMD and the physical object as basis for initiating a collision-avoidance action, wherein the collision-avoidance action comprises de-emphasizing at least one of the displayed virtual objects so as to provide a less-obstructed view of the physical object through the see-through display of the HMD.

20. The wearable computer of claim 19, wherein the logic is further executable by the at least one processor to:

receive user-input data that indicates to re-display the at least one virtual object that was de-emphasized in the collision-avoidance action; and

in response to at least the receiving the user-input data, re-display in the view region the at least one virtual object that was de-emphasized in the collision-avoidance action.

21. The wearable computer of claim 20, wherein the user-input comprises detection by sensors that the head-mounted display has moved in specified movement pattern.

22. A non-transitory computer-readable medium having stored therein instructions executable by a computing device to cause the computing device to perform functions comprising:

causing a head-mountable device (HMD) to display a user-interface mapped to a sphere, wherein the user-interface comprises;

a view region; and

at least one content region located above the view region, wherein the at least one content region comprises a set of selectable content objects that are arranged along an at least partial ring above the view region, wherein the at least partial ring is substantially centered above the HMD, wherein the view region and the at least one content region are mapped to the sphere, and wherein a field of view provided in a see-through display of the HMD is moveable with respect to the sphere;

associating a forward-looking head position with a first location of the field of view within the sphere, wherein (a) the view region is substantially within the first location of the field of view provided in the see-through display of the HMD, and (b) each of the set of selectable content objects is above and substantially outside the first location of the field of view, wherein the field of view at the first location, which is associated with the forward-looking head position, is in a direction that is generally parallel to the ground;

displaying one or more virtual objects in view region;

using data from one or more first sensors as a basis for determining a distance between the HMD and a physical object; and

using the determined distance between the HMD and the physical object as basis for initiating a collision-avoidance action, wherein the collision-avoidance action comprises de-emphasizing at least one of the displayed virtual objects so as to provide a less-obstructed view of the physical object through the see-through display of the HMD.

23. The non-transitory computer-readable medium of claim 22, wherein the functions further comprise:

using data from one or more second sensors to determine a relative movement of the physical object with respect to the HMD; and

using the relative movement as a further basis for initiating the collision-avoidance action.

24. The non-transitory computer-readable medium of claim 23, wherein the functions further comprise subsequent to initiating the collision-avoidance action:

using data from the one or more second sensors as a basis for determining a subsequent relative movement of the physical object with respect to the HMD;

determining that a direction of the subsequent relative movement of the physical object is directed sufficiently away from the HMD such that collision with the physical object is unlikely; and

in response to determining that the subsequent relative movement of the physical object is directed sufficiently away from the HMD, re-displaying in the view region the at least one virtual object that was de-emphasized in the collision-avoidance action.

25. The non-transitory computer-readable medium of claim 22, wherein the moving the at least one of said virtual objects comprises efficiently portraying a potential hazard by moving the at least one of said virtual objects quickly from a center point of the view region toward outer edges of the view region.