Wearable computer with nearby object response

• US 9,552,676 B2
• Filed: 04/19/2016
• Issued: 01/24/2017
• Est. Priority Date: 10/07/2011
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Wong, Adrian, Miao, Xiaoyu
• Primary Examiner(s): Mengistu, Amare
• Assistant Examiner(s): MATHEWS, CRYSTAL
• Application Number:
  US15/132,852
• Publication Number:
  US 20160232717A1
• Time in Patent Office: 280 Days
• US Class Current: 1/1
• CPC Class Codes: G02B 2027/0138 comprising image capture systems, e.g. camera
• CPC Class Codes: G02B 2027/014 comprising information/image processing systems
• CPC Class Codes: G02B 2027/015 involving arrangement aiming to get less bulky devices
• CPC Class Codes: G02B 2027/0178 Eyeglass type eyeglass details G02C
• CPC Class Codes: G02B 27/0093 with means for monitoring data relating to the user, e.g. head-tracking, eye-tracking
• CPC Class Codes: G02B 27/017 Head mounted
• CPC Class Codes: G02B 27/0172 characterised by optical features
• CPC Class Codes: G02B 27/0176 characterised by mechanical features
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/013 Eye tracking input arrangements G06F3/015 takes precedence
• CPC Class Codes: G06F 3/04815 Interaction with a metaphor-based environment or interaction object displayed as three-dimensional, e.g. changing the user viewpoint with respect to the environment or object
• CPC Class Codes: G06F 3/0482 Interaction with lists of selectable items, e.g. menus
• CPC Class Codes: G06F 3/167 Audio in a user interface, e.g. using voice commands for navigating, audio feedback
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: G06T 2200/24 involving graphical user interfaces [GUIs]
• CPC Class Codes: G06V 20/10 Terrestrial scenes scenes under surveillance with static cameras G06V20/52; scenes perceived from the exterior of a vehicle G06V20/56; scenes perceived from the interior of a vehicle G06V20/59

Abstract

Exemplary methods and systems relate to detecting physical objects near a substantially transparent head-mounted display (HMD) system and activating a collision-avoidance action to alert a user of the detected objects. Detection techniques may include receiving data from distance and/or relative movement sensors and using this data as a basis for determining an appropriate collision-avoidance action. Exemplary collision-avoidance actions may include de-emphasizing virtual objects displayed on the HMD to provide a less cluttered view of the physical objects through the substantially transparent display and/or presenting new virtual objects.

Claims

1. A computer-implemented method comprising:

causing a head-mountable device (HMD) to display a field of view to a user-interface mapped to a sphere, wherein the field of view is moveable with respect to the user-interface, and wherein the user-interface comprises;

(i) a view region, and (ii) a set of selectable content objects arranged in at least a partial ring;

associating a forward-looking head position with a forward-looking location of the field of view on the sphere, wherein a default arrangement of the spherical user-interface both;

(i) locates the view region within the forward-looking field of view, and (ii) locates the set of selectable content objects substantially outside the forward-looking field of view, wherein the forward-looking field of view is in a direction that is generally parallel to the ground;

displaying one or more virtual objects in the view region;

using data from one or more first sensors as a basis for determining a distance between the HMD and a physical object; and

using the determined distance between the HMD and the physical object as basis for initiating a collision-avoidance action, wherein the collision-avoidance action de-emphasizes one or more of the displayed virtual objects to provide a less-obstructed view of the physical object.

2. The method of claim 1, wherein using the determined distance as basis for initiating the collision-avoidance action comprises:

determining that the physical object is less than a threshold distance from the HMD; and

responsively initiating the collision-avoidance action.

3. The method of claim 1, further comprising:

using data from one or more second sensors to determine a relative movement of the physical object with respect to the HMD; and

using the relative movement as a further basis for initiating the collision-avoidance action.

4. The method of claim 3, wherein using the relative movement as a further basis for initiating the collision-avoidance action comprises:

using the relative movement as a basis for determining the threshold distance between the physical object and the HMD; and

determining that the physical object is less than the threshold distance from the HMD and responsively initiating the collision-avoidance action.

5. The method of claim 3, wherein using the relative movement as a further basis for initiating the collision-avoidance action comprises:

using (a) the distance between the physical object and the HMD and (b) the relative movement to determine a time period until collision; and

determining that the time period until collision is less than a threshold time period and responsively initiating the collision-avoidance action.

6. The method of claim 3, further comprising, subsequent to initiating the collision-avoidance action:

using data from the one or more second sensors as a basis for determining a subsequent relative movement of the physical object with respect to the HMD;

determining that a direction of the subsequent relative movement of the physical object is directed sufficiently away from the HMD such that collision with the physical object is unlikely; and

in response to determining that the subsequent relative movement of the physical object is directed sufficiently away from the HMD, re-displaying in the view region the at least one virtual object that was de-emphasized in the collision-avoidance action.

7. The method of claim 1, further comprising:

using data from one or more third sensors to determine that the HMD is in motion; and

using the determination that the HMD is in motion as a further basis for initiating a collision-avoidance action.

8. The method of claim 1, further comprising:

determining that the view region is sufficiently cluttered; and

using the determination that the view region is sufficiently cluttered as a further basis for initiating the collision-avoidance action.

9. The method of claim 1, wherein de-emphasizing at least one of the virtual objects comprises removing at least one of the virtual objects from the view region.

10. The method of claim 1, wherein the moving the at least one of said virtual objects comprises efficiently portraying a potential hazard by moving the at least one of said virtual objects quickly from a center point of the view region toward outer edges of the view region.

11. The method of claim 10, further comprising:

determining a portion of the view region that overlays the physical object;

wherein the at least one of said virtual objects is moved quickly away from the portion of the view region that overlays the physical object.

12. The method of claim 1, wherein the physical object is outside of the field of view provided in the see-through display, and wherein the collision-avoidance action further comprises displaying a visual indication of a direction of the physical object.

13. The method of claim 1, wherein the collision-avoidance action further comprises presenting an audio alert.

14. The method of claim 1, wherein the collision-avoidance action further comprises displaying a visual indication that highlights the physical object.

15. A computing system comprising:

at least one processor;

one or more first sensors operable to generate data indicative of distances to physical objects in an environment of a head-mountable device (HMD);

data storage comprising logic executable by the at least one processor to;

cause a display of the HMD to display a field of view to a user-interface mapped to a sphere, wherein the field of view is moveable with respect to the user-interface, and wherein the user-interface comprises;

(i) a view region, and (ii) a set of selectable content objects arranged in at least a partial ring;

associate a forward-looking head position with a forward-looking location of the field of view on the sphere, wherein a default arrangement of the spherical user-interface both;

(i) locates the view region within the forward-looking field of view, and (ii) locates the set of selectable content objects substantially outside the forward-looking field of view, wherein the forward-looking field of view is in a direction that is generally parallel to the ground;

display one or more virtual objects in the view region;

use data from one or more first sensors as a basis for determining a distance between the HMD and a physical object; and

use the determined distance between the HMD and the physical object as basis to initiate a collision-avoidance action, wherein the collision-avoidance action de-emphasizes one or more of the displayed virtual objects to provide a less-obstructed view of the physical object.

16. The computing system of claim 15, wherein the user-input comprises detection by sensors that the head-mounted display has moved in specified movement pattern.

17. A non-transitory computer-readable medium having stored therein instructions executable by a computing device to cause the computing device to perform functions comprising:

causing a head-mountable device (HMD) to display a field of view to a user-interface mapped to a sphere, wherein the field of view is moveable with respect to the user-interface, and wherein the user-interface comprises;

(i) a view region, and (ii) a set of selectable content objects arranged in at least a partial ring;

associating a forward-looking head position with a forward-looking location of the field of view on the sphere, wherein a default arrangement of the spherical user-interface both;

(i) locates the view region within the forward-looking field of view, and (ii) locates the set of selectable content objects substantially outside the forward-looking field of view, wherein the forward-looking field of view is in a direction that is generally parallel to the ground;

displaying one or more virtual objects in the view region;

using data from one or more first sensors as a basis for determining a distance between the HMD and a physical object; and

using the determined distance between the HMD and the physical object as basis for initiating a collision-avoidance action, wherein the collision-avoidance action de-emphasizes one or more of the displayed virtual objects to provide a less-obstructed view of the physical object.

18. The non-transitory computer-readable medium of claim 17, wherein the functions further comprise:

using data from one or more second sensors to determine a relative movement of the physical object with respect to the HMD; and

using the relative movement as a further basis for initiating the collision-avoidance action.

19. The non-transitory computer-readable medium of claim 18, wherein the functions further comprise subsequent to initiating the collision-avoidance action:

using data from the one or more second sensors as a basis for determining a subsequent relative movement of the physical object with respect to the HMD;

determining that a direction of the subsequent relative movement of the physical object is directed sufficiently away from the HMD such that collision with the physical object is unlikely; and

in response to determining that the subsequent relative movement of the physical object is directed sufficiently away from the HMD, re-displaying in the view region the at least one virtual object that was de-emphasized in the collision-avoidance action.

20. The non-transitory computer-readable medium of claim 17, wherein the moving the at least one of said virtual objects comprises efficiently portraying a potential hazard by moving the at least one of said virtual objects quickly from a center point of the view region toward outer edges of the view region.