Adaptive screen interfaces based on viewing distance

• US 9,159,116 B2
• Filed: 02/13/2013
• Issued: 10/13/2015
• Est. Priority Date: 02/13/2013
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Plagemann, Christian, Kauffmann, Alejandro
• Primary Examiner(s): HARRISON, CHANTE E
• Application Number:
  US13/765,877
• Publication Number:
  US 20140225931A1
• Time in Patent Office: 972 Days
• Field of Search: 345/660, 345/667, 345/665, 345/168, 345/169, 345/173, 345/650, 715/863, 715/864, 715/764, 710/5, 714/27
• US Class Current: 1/1
• CPC Class Codes: G06F 2203/04806 Zoom, i.e. interaction techniques or interactors for controlling the zooming operation
• CPC Class Codes: G06F 3/005 Input arrangements through a video camera
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0488 using a touch-screen or digitiser, e.g. input of commands through traced gestures
• CPC Class Codes: G06T 3/40 Scaling of whole images or parts thereof, e.g. expanding or contracting

Abstract

Voice commands and gesture recognition are two mechanisms by which an individual may interact with content such as that on a display. In an implementation, interactivity of a user with content on a device or display may be modified based on the distance between a user and the display. An attribute such as a user profile may be used to tailor the modification of the display to an individual user. In some configurations, the commands available to the user may also be modified based on the determined distance between the user and a device or display.

Claims

1. A computer-implemented method comprising:

obtaining, by a device that includes a display and a camera, a first image and a later, second image;

obtaining, by the device, a first estimate of a distance between a user and the display based on the first image, and a later, second estimate of a distance between the user and the display based on the second image;

determining, by the device and based at least on the first estimate of the distance between the user and the display and the second estimate of the distance between the user and the display, that the user has crossed a predetermined distance threshold; and

in response to determining that the user has crossed the predetermined distance threshold, switching, by the device, between (i) a touch input mode in which touch inputs through the display are enabled, and speech or gesture inputs are disabled, and (ii) a gesture or speech input mode in which gesture or speech inputs are enabled, and touch inputs through the display are disabled.

2. The method of claim 1, comprising:

receiving a user input in a given input mode;

determining, from among multiple commands that are each associated with (I) the user input, (II) a respective distance, and (III) the given input mode, a particular command that is associated with (i) the user input, (ii) the second estimate, and (iii) the given input mode; and

providing the particular command to an application running on the device.

3. The method of claim 1, comprising:

selecting, from among multiple sets of commands that are each associated with (I) a respective distance and (II) the given input mode, a particular set of commands that are associated with (i) the second estimate and (ii) the given input mode; and

providing the user with a visual indication of the particular set of commands.

4. The method of claim 1, comprising:

determining an object estimate of a distance between the user and an object other than the display;

receiving a user input in a given input mode;

determining, from among multiple commands that are each associated with (I) the user input, (II) a respective distance, and (III) the given input mode, a particular command that is associated with (i) the user input, (ii) the object estimate, and (iii) the given input mode; and

providing the particular command to an application running on the device.

5. The method of claim 1, comprising:

selecting a particular user from among multiple users that are within a physical proximity of the display; and

wherein obtaining the first estimate of a distance between the user and the display comprises obtaining a first estimate of a distance between the particular user and the display, and the later, second of a distance between the user and the display comprises obtaining a second estimate of a distance between the particular user and the display.

6. A non-transitory computer-readable storage device having instructions stored thereon that, when executed by a computing device, cause the computing device to perform operations comprising:

obtaining, by a device that includes a display and a camera, a first image and a later, second image;

obtaining, by the device, a first estimate of a distance between a user and the display based on the first image, and a later, second estimate of a distance between the user and the display based on the second image;

determining, by the device and based at least on the first estimate of the distance between the user and the display and the second estimate of the distance between the user and the display, that the user has crossed a predetermined distance threshold; and

in response to determining that the user has crossed the predetermined distance threshold, switching, by the device, between (i) a touch input mode in which touch inputs through the display are enabled, and speech or gesture inputs are disabled, and (ii) a gesture or speech input mode in which gesture or speech inputs are enabled, and touch inputs through the display are disabled.

7. The storage device of claim 6, wherein the operations further comprise:

receiving a user input in a given input mode;

determining, from among multiple commands that are each associated with (I) the user input, (II) a respective distance, and (III) the given input mode, a particular command that is associated with (i) the user input, (ii) the second estimate, and (iii) the given input mode; and

providing the particular command to an application running on the device.

8. The storage device of claim 6, wherein the operations further comprise:

selecting, from among multiple sets of commands that are each associated with (I) a respective distance and (II) the given input mode, a particular set of commands that are associated with (i) the second estimate and (ii) the given input mode; and

providing the user with a visual indication of the particular set of commands.

9. The storage device of claim 6, wherein the operations further comprise:

determining an object estimate of a distance between the user and an object other than the display;

receiving a user input in a given input mode;

determining, from among multiple commands that are each associated with (I) the user input, (II) a respective distance, and (III) the given input mode, a particular command that is associated with (i) the user input, (ii) the object estimate, and (iii) the given input mode; and

providing the particular command to an application running on the device.

10. The storage device of claim 6, wherein the operations further comprise:

selecting a particular user from among multiple users that are within a physical proximity of the display; and

wherein obtaining the first estimate of a distance between the user and the display comprises obtaining a first estimate of a distance between the particular user and the display, and the later, second of a distance between the user and the display comprises obtaining a second estimate of a distance between the particular user and the display.

11. A system comprising:

one or more data processing apparatus; and

a computer-readable storage device having stored thereon instructions that, when executed by the one or more data processing apparatus, cause the one or more data processing apparatus to perform operations comprising;

obtaining, by a device that includes a display and a camera, a first image and a later, second image;

obtaining, by the device, a first estimate of a distance between a user and the display based on the first image, and a later, second estimate of a distance between the user and the display based on the second image;

determining, by the device and based at least on the first estimate of the distance between the user and the display and the second estimate of the distance between the user and the display, that the user has crossed a predetermined distance threshold; and

in response to determining that the user has crossed the predetermined distance threshold, switching, by the device, between (i) a touch input mode in which touch inputs through the display are enabled, and speech or gesture inputs are disabled, and (ii) a gesture or speech input mode in which gesture or speech inputs are enabled, and touch inputs through the display are disabled.

12. The system of claim 11, wherein the operations further comprise:

receiving a user input in a given input mode;

determining, from among multiple commands that are each associated with (I) the user input, (II) a respective distance, and (III) the given input mode, a particular command that is associated with (i) the user input, (ii) the second estimate, and (iii) the given input mode; and

providing the particular command to an application running on the device.

13. The system of claim 11, wherein the operations further comprise:

selecting, from among multiple sets of commands that are each associated with (I) a respective distance and (II) the given input mode, a particular set of commands that are associated with (i) the second estimate and (ii) the given input mode; and

providing the user with a visual indication of the particular set of commands.

14. The system of claim 11, wherein the operations further comprise:

determining an object estimate of a distance between the user and an object other than the display;

receiving a user input in a given input mode;

determining, from among multiple commands that are each associated with (I) the user input, (II) a respective distance, and (III) the given input mode, a particular command that is associated with (i) the user input, (ii) the object estimate, and (iii) the given input mode; and

providing the particular command to an application running on the device.