Context detection and context-based user-interface population

• US 8,947,322 B1
• Filed: 03/19/2012
• Issued: 02/03/2015
• Est. Priority Date: 03/19/2012
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Chi, Liang-Yu (Tom), Ryskamp, Robert Allen, Wheeler, Aaron Joseph, Prada Gomez, Luis Ricardo
• Primary Examiner(s): Sasinowski, Andrew
• Application Number:
  US13/423,392
• Time in Patent Office: 1,051 Days
• Field of Search: 345/1.1, 345 7- 9, 345/629, 348/46, 715/744, 715/747, 715/850
• US Class Current: 345/8
• CPC Class Codes: G02B 2027/0138 comprising image capture systems, e.g. camera
• CPC Class Codes: G02B 2027/014 comprising information/image processing systems
• CPC Class Codes: G02B 2027/0178 Eyeglass type eyeglass details G02C
• CPC Class Codes: G02B 2027/0187 slaved to motion of at least a part of the body of the user, e.g. head, eye
• CPC Class Codes: G02B 27/017 Head mounted
• CPC Class Codes: G06F 1/163 Wearable computers, e.g. on a belt
• CPC Class Codes: G06F 3/012 Head tracking input arrangements
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0482 Interaction with lists of selectable items, e.g. menus
• CPC Class Codes: G06F 3/04842 Selection of displayed objects or displayed text elements G06F3/0482 takes precedence

Abstract

Exemplary methods and systems relate to a wearable computing device determining a user-context and dynamically changing the content of a user-interface based on the determined user-context. The device may determine a user-context based on digital context; such as a text document a user is reading or a current website the device is accessing. User-context may also be based on physical context; such as the device'"'"'s location or the air temperature around a user. Once a user-context is determined, a device may identify content that is related to the user-context and add objects representing this related content to a user-interface.

Claims

1. A method implemented by a head-mountable device comprising:

receiving, by the head-mountable computing device, input data that is indicative of head position;

providing, by the head-mountable computing device, a user-interface that comprises a content region containing a set of selectable content objects that are horizontally arranged, wherein the content region is located at a fixed height within the user-interface;

defining, by the head-mountable computing device, a view region that is movable within the user-interface, wherein the view region is smaller than the user-interface;

associating, by the head-mountable computing device, a forward-looking head position with a first location of the view region within the user-interface, wherein the fixed height of the content region is such that at least a portion of the content region is located above the view region, when the view region is at the first location associated with forward-looking head position;

based on head movement data, moving the view region within the user-interface, wherein the head movement data is determined based on the input data that is indicative of head position;

as the view region moves within the user-interface, displaying a portion of the user-interface corresponding to the view region in the see-through display;

determining a first user-context associated with the head-mountable device; and

dynamically changing the set of selectable content objects contained in the content region based on the determined first user-context.

2. The method of claim 1, wherein the user-interface is configured such that the content region is substantially arranged along an at least partial ring above the view region.

3. The method of claim 1, wherein the content region is initially not displayed at all in the first portion of the user-interface.

4. The method of claim 1, wherein determining the first user-context associated with the head-mountable device comprises:

transmitting one or more user-context signals to a remote server system, wherein the one or more user-context signals are associated with the head-mountable device; and

receiving from the remote server system an indication of a first user-context associated with the head-mountable device.

5. The method of claim 1, wherein the first user-context is determined based on both physical context signals and digital context signals.

6. The method of claim 1, wherein the user-context is determined based on one or more user-context signals, wherein the one or more user-context signals comprise one or more of the following:

(a) a current time, (b) a current date, (c) a current day of the week, (d) a current month, (e) a current season, (f) a time of a future event or future context, (g) a date of a future event or future context, (h) a day of the week of a future event or future context, (i) a month of a future event or future user-context, (j) a season of a future event or future context, (k) a time of a past event or past context, (l) a date of a past event or past context, (m) a day of the week of a past event or past context, (n) a month of a past event or past context, (o) a season of a past event or past context, (p) ambient temperature, (q) a current, future, or past weather forecast at a current location, (r) a current, future, or past weather forecast at a location of a planned event, (s) a current, future, or past weather forecast at or near a location of a previous event, (t) information on a calendar associated with a user-profile, such as information regarding events or statuses of a user or a user'"'"'s friends, (t) information accessible via a user'"'"'s social networking account, such as information relating a user'"'"'s status, statuses of a user'"'"'s friends in a social network group, and/or communications between the user and the user'"'"'s friends, (u) ambient noise level (v) any recognizable sounds detected around the head-mountable device, (w) items that are currently detected around the head-mountable device, (x) items that have been detected around the head-mountable device, (y) items with which other devices associated with a user of the head-mountable device are currently in communication, (z) items with which other devices associated with a user of the head-mountable device have been in communication (aa) information derived from cross-referencing any two or more of;

information on a user'"'"'s calendar, information available via a user'"'"'s social networking account, and other context signals and sources of context information, (bb) health statistics and characterizations of a user'"'"'s current health, (cc) a user'"'"'s recent context as determined from sensors on or near the user and/or other sources of context information, (dd) a current location, (ee) a past location, (ff) a future location, (gg) computer-readable media currently accessed on the wearable computing device, (hh) computer-readable media previously accessed on the head-mountable device, (ii) computer-readable media currently accessed on other devices associated with a user of the head-mountable device, and (jj) computer-readable media previously accessed on other devices associated with a user of the head-mountable device.

7. The method of claim 1, wherein dynamically changing the set of selectable content objects contained in the content region comprises (i) identifying content related to the determined user-context and (ii) adding, to the set of selectable content objects contained in the content region, a selectable content object that references the identified content.

8. The method of claim 1, wherein dynamically changing the set of selectable content objects contained in the content region comprises (i) identifying at least one selectable content object that is contained in the content region and is no longer associated with a current user-context;

and (ii) removing, from the set of selectable content objects contained in the content region, the identified at least one selectable content object.

9. The method of claim 1, wherein determining the first user-context associated with the head-mountable device is performed repeatedly, and wherein dynamically changing the set of selectable content objects contained in the content region is also performed repeatedly.

10. The method of claim 9, further comprising:

storing data representing the first user-context;

determining a second user-context associated with the head-mountable device;

using the stored data representing the first user-context to determine that the second user-context differs sufficiently from the first user-context; and

in response to determining that the second user-context differs sufficiently from the first user-context, dynamically changing the set of selectable content objects contained in the content region based on the determined second user-context.

11. The method of claim 1, further comprising:

receiving input selecting one or more content objects from the set of selectable content objects contained in the content region; and

displaying in the view region content associated with the selected one or more content objects.

12. A head-mountable device (HMD) comprising:

a see-through display;

at least one processor; and

data storage comprising logic executable by the at least one processor to;

receive input data that is indicative of head position;

define a user-interface that comprises a content region containing a set of selectable content objects that are horizontally arranged, wherein the content region is located at a fixed height within the user-interface;

define a view region that is movable within the user-interface, wherein the view region is smaller than the user-interface;

associate a forward-looking head position with a first location of the view region within the user-interface, wherein the fixed height of the content region is such that at least a portion of the content region is located above the view region, when the view region is at the first location associated with forward-looking head position;

based on head movement data, move the view region within the user-interface, wherein the head movement data is determined based on the input data that is indicative of head position;

as the view region moves within the user-interface, display a portion of the user-interface corresponding to the view region in the see-through display;

determine a first user-context associated with the head-mountable device; and

dynamically change the set of selectable content objects contained in the content region based on the determined first user-context.

13. The head-mountable device of claim 12, wherein determining the first user-context associated with the wearable computing device comprises:

transmitting one or more user-context signals to a remote server system, wherein the one or more user-context signals are associated with the head-mountable device; and

receiving from the remote server system an indication of a first user-context associated with the head-mountable device.

14. The head-mountable device of claim 12, wherein the logic is further executable by the at least one processor to:

receive input selecting one or more content objects from the set of selectable content objects contained in the content region; and

display in the view region content associated with the selected one or more content objects.

15. The wearable computing device of claim 12, further comprising:

one or more communicative links to other digital devices, wherein the other digital devices are associated with a user of the head-mountable device; and

wherein the user-context is determined based on one or more user-context signals received from the other digital devices through the one or more communicative links.

16. The HMD of claim 12, wherein a first subset from the set of selectable content objects is located at a fixed horizontal position within the user-interface, and wherein a second subset from the set of selectable content objects is horizontally moveable with respect to the user-interface based on input data indicative of horizontal head movement.

17. The HMD of claim 16, wherein the second subset comprises one or more tools.

18. A non-transitory computer-readable medium having stored thereon instructions executable by a computing device to cause the computing device to perform functions comprising:

receiving, by the head-mountable computing device, input data that is indicative of head position;

providing, by the head-mountable computing device, a user-interface that comprises a content region containing a set of selectable content objects that are horizontally arranged, wherein the content region is located at a fixed height within the user-interface;

defining, by the head-mountable computing device, a view region that is movable within the user-interface, wherein the view region is smaller than the user-interface;

associating, by the head-mountable computing device, a forward-looking head position with a first location of the view region within the user-interface, wherein the fixed height of the content region is such that at least a portion of the content region is located above the view region, when the view region is at the first location associated with forward-looking head position;

based on head movement data, moving the view region within the user-interface, wherein the head movement data is determined based on the input data that is indicative of head position;

as the view region moves within the user-interface, displaying a portion of the user-interface corresponding to the view region in the see-through display;

determining a first user-context associated with the head-mountable device; and

dynamically changing the set of selectable content objects contained in the content region based on the determined first user-context.

19. The non-transitory computer-readable medium of claim 18, wherein determining the user-context associated with the wearable computing device comprises:

transmitting one or more user-context signals to a remote server system, wherein the one or more user-context signals are associated with the head-mountable device; and

receiving from the remote server system an indication of a user-context associated with the head-mountable device.