User interface adaptation based on detected user location

• US 10,372,193 B2
• Filed: 08/15/2017
• Issued: 08/06/2019
• Est. Priority Date: 07/31/2014
• Status: Active Grant

• Current Assignee: Google Technology Holdings LLC (Alphabet Inc.)
• Original Assignee: Google Technology Holdings LLC (Alphabet Inc.)
• Inventors: Ady, Roger W., Alameh, Rachid, Steuer, Paul R.
• Primary Examiner(s): Amini, Javid A
• Application Number:
  US15/677,274
• Publication Number:
  US 20170344094A1
• Time in Patent Office: 721 Days
• Field of Search: 345619
• CPC Class Codes: G06F 1/3231 Monitoring the presence, absence or movement of users
• CPC Class Codes: G06F 1/3265 Power saving in display device
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: Y02D 10/00 Energy efficient computing, e.g. low power processors, power management or thermal management

Abstract

A device (102) detects when a user is in close physical proximity to the device, such as within a few feet of the device. While the user is in close physical proximity to the device, the device also detects the location of the user relative to the device, and optionally a speed at which the user is moving. The user interface presented by the device is adapted to the location of the user relative to the device and optionally the speed at which the user is moving, but without needing physical manipulation of the device by the user. The adaptation can include, for example, orienting content presented on a display of the device based on the location of the user relative to the device so that the content appears “upright” or “right side up” for the user to view given their location.

Claims

1. A computing device comprising:

a display that presents content;

two or more microphones, wherein a first microphone of the two or microphones detects a sound that includes a voice; and

a user interface adaptation system configured to;

determine, based on the sound, whether a user is in close physical proximity to the computing device; and

responsive to determining that the user is in close physical proximity to the computing device;

activate at least a second microphone from the two or more microphones, wherein, after the second microphone is activated, the first and second microphones detect speech of the user;

determine, based on the speech of the user as detected by each of the first and second microphones, a direction of the user relative to the computing device; and

change, based on the direction of the user relative to the computing device, an orientation of content presented to the user.

2. The computing device of claim 1, the user interface adaptation system being further configured to change, based on one or more of a distance between the user and the computing device or a speed at which the user is moving relative to the computing device, the content presented to the user by at least changing one or more of a size a portion of the content, a brightness of the display, or one or more colors of the content.

3. The computing device of claim 1, wherein the user interface adaptation system is further configured to change the orientation of the content presented to the user by at least being configured to change the orientation of the content presented to the user in the absence of physical manipulation of the computing device by the user.

4. The computing device of claim 1, wherein the user interface adaptation system is further configured to repeat determining the direction of the user relative to the computing device and changing the orientation of the content presented to the user of the computing device as the direction of the user relative to the computing device changes.

5. The computing device of claim 1, wherein the user interface adaptation system is further configured to:

determine, based on the sound detected by the first microphone, that multiple users are in close physical proximity to the computing device; and

select one of the multiple users,

wherein the direction of the user relative to the computing device is the direction of the selected user relative to the computing device, and

wherein the orientation of the content presented to the user of the computing device is changed based on the direction of the selected user relative to the computing device.

6. A method comprising:

detecting, by a microphone of a computing device, sound that includes a voice;

determining, by the computing device and based on the sound, whether a user is in close physical proximity to the computing device; and

responsive to determining that the user is in close physical proximity to the computing device;

activating at least one additional microphone of the computing device;

detecting, by two or more microphones of the computing device, speech of the user;

determining, by the computing device, based on the speech of the user as detected by each of the two or more microphones, a direction of the user relative to the computing device; and

changing, based on the direction of the user relative to the computing device, an orientation of content presented to the user.

7. The method of claim 6, further comprising:

determining, based on the speech of the user as detected by each of the two or more microphones, a speed at which the user is moving relative to the computing device; and

changing, based on the speed at which the user is moving relative to the computing device, the content presented to the user by at least changing one or more of a size a portion of the content, a brightness of a display of the computing device, or one or more colors of the content.

8. The method of claim 6, further comprising:

determining, based on the speech of the user as detected by each of the two or more microphones, a distance between the user and the computing device;

changing, based on the distance between the user and the computing device, an appearance of the content presented to the user by at least changing one or more of a size of a portion of the content, a brightness of a display of the computing device, or one or more colors of the content.

9. The method of claim 6, wherein changing the orientation of the content presented to the user comprises changing the orientation of the content presented to the user in the absence of physical manipulation of the computing device by the user.

10. The method of claim 6, wherein changing the orientation of the content comprises changing the orientation of the content presented to the user in increments.

11. The method of claim 6, further comprising repeating determining the direction of the user relative to the computing device and changing the orientation of the content presented to the user of the computing device as the direction of the user relative to the computing device changes.

12. The method of claim 11, further comprising imposing a time limit on changing the orientation of the content presented to the user of the computing device so that the orientation of the content presented to the user is not changed at greater than a threshold frequency.

13. The method of claim 11, further comprising changing the orientation of the content presented to the user of the computing device only if movement of the user relative to the computing device exceeds an angular threshold.

14. The method of claim 6, further comprising:

determining, based on the sound detected by the microphone of the computing device, that multiple users are in close physical proximity to the computing device; and

selecting one of the multiple users,

wherein determining the direction of the user relative to the computing device comprises determining the direction of the selected user relative to the computing device, and

wherein changing the orientation of the content presented to the user of the computing device comprises changing the orientation of the content presented to the user of the computing device based on the direction of the selected user relative to the computing device.

15. The method of claim 6, wherein selecting one of the multiple users includes one or more of:

selecting the one of the multiple users who is closest to the computing device;

selecting the one of the multiple users who is moving fastest;

selecting the one of the multiple users having a face recognized by the computing device;

or

selecting the one of the multiple users who is moving towards the computing device.

16. A non-transitory computer-readable storage medium encoded with instructions that, when executed, cause one or more processors of a computing device to:

determining, by the computing device and based on sound detected by a microphone of the computing device, whether a user is in close physical proximity to the computing device; and

responsive to determining that the user is in close physical proximity to the computing device;

activate at least one additional microphone of the computing device;

determining, by the computing device, based on speech of the user as detected by each of the two or more microphones, a direction of the user relative to the computing device; and

change, based on the direction of the user relative to the computing device, an orientation of content presented to the user.

17. The non-transitory computer-readable storage medium of claim 16, wherein the instructions further cause the one or more processors to:

change, based on one or more of a distance between the user and the computing device or a speed at which the user is moving relative to the computing device, one or more of a size a portion of the content, a brightness of a display of the computing device, or one or more colors of the content presented to the user.

18. The non-transitory computer-readable storage medium of claim 16, wherein the instructions cause the one or more processors to change the orientation of the content presented to the user in the absence of physical manipulation of the computing device by the user.

19. The non-transitory computer-readable storage medium of claim 16, wherein the instructions further cause the one or more processors to repeat determining the direction of the user relative to the computing device and changing the orientation of the content presented to the user of the computing device as the direction of the user relative to the computing device changes.

20. The non-transitory computer-readable storage medium of claim 17, wherein the instructions further cause the one or more processors to:

determine, based on the sound detected by the microphone of the computing device, that multiple users are in close physical proximity to the computing device; and

select one of the multiple users,

wherein the direction of the user relative to the computing device is the direction of the selected user relative to the computing device, and

wherein the orientation of the content presented to the user of the computing device is changed based on the direction of the selected user relative to the computing device.