Managing sensory information of a user device

• US 9,459,731 B2
• Filed: 08/03/2015
• Issued: 10/04/2016
• Est. Priority Date: 03/13/2013
• Status: Active Grant

• Current Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Original Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Inventors: Ricks, James Edward
• Primary Examiner(s): Edun, Muhammad N
• Application Number:
  US14/817,056
• Publication Number:
  US 20150338999A1
• Time in Patent Office: 428 Days
• Field of Search: None
• US Class Current: 1/1
• CPC Class Codes: G06F 1/3231 Monitoring the presence, absence or movement of users
• CPC Class Codes: G06F 2203/04106 Multi-sensing digitiser, i.e. digitiser using at least two different sensing technologies simultaneously or alternatively, e.g. for detecting pen and finger, for saving power or for improving position detection
• CPC Class Codes: G06F 2203/04108 Touchless 2D- digitiser, i.e. digitiser detecting the X/Y position of the input means, finger or stylus, also when it does not touch, but is proximate to the digitiser's interaction surface without distance measurement in the Z direction
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/041 Digitisers, e.g. for touch screens or touch pads, characterised by the transducing means
• CPC Class Codes: G06F 3/0416 Control or interface arrangements specially adapted for digitisers
• CPC Class Codes: G06F 3/0421 by interrupting or reflecting a light beam, e.g. optical touch-screen
• CPC Class Codes: G06F 3/0425 using a single imaging device like a video camera for tracking the absolute position of a single or a plurality of objects with respect to an imaged reference surface, e.g. video camera imaging a display or a projection screen, a table or a wall surface, on which a computer generated image is displayed or projected tracking a projected light spot to determine a position on a display surface G06F3/0386
• CPC Class Codes: G06F 3/043 using propagating acoustic waves
• CPC Class Codes: G06F 3/0436 in which generating transducers and detecting transducers are attached to a single acoustic waves transmission substrate
• CPC Class Codes: G06F 3/0488 using a touch-screen or digitiser, e.g. input of commands through traced gestures
• CPC Class Codes: G06F 3/04883 for inputting data by handwriting, e.g. gesture or text
• CPC Class Codes: G06V 40/172 Classification, e.g. identification
• CPC Class Codes: Y02D 10/00 Energy efficient computing, e.g. low power processors, power management or thermal management

Abstract

External mobile device sensors may be provided that are configured to manage sensory information associated with motion of objects external to the mobile device. In some examples, the object motion may be detected independent of contact with the device. In some examples, a device may include a screen with a first sensor (e.g., a touch sensor). The device may also include at least a second sensor external to the screen. Instructions may be executed by a processor of the device to at least determine when an object is hovering over a first graphical user interface (GUI) element of the screen. Additionally, in some cases, a second GUI element may be provided on the screen such that the second GUI element is rendered on the screen adjacent to a location under the hovering object.

Claims

1. A computer-implemented method, comprising:

monitoring, by a first sensor of a mobile device, for an object external to and independent of contact with a screen of the mobile device;

receiving, from the first sensor, first information that identifies the object external to and independent of contact with the screen;

determining, based at least in part on the first information, that the object comprises at least a portion of a face of a user;

determining a plurality of potential operations of the mobile device based at least in part on the portion of the face of the user being determined from the first information;

receiving, from a second sensor, second information that further identifies the object;

determining, based at least in part on the second information, facial characteristics of the user;

determining an operation of the plurality of potential operations of the mobile device based at least in part on the facial characteristics of the user corresponding to user information; and

performing the determined operation.

2. The computer-implemented method of claim 1, wherein the first sensor is configured to be an ambient sensor and the second sensor is configured to be an effective sensor.

3. The computer-implemented method of claim 2, wherein the ambient sensor is configured with less fidelity than the effective sensor.

4. The computer-implemented method of claim 3, further comprising activating the second sensor based at least in part on the first information received from the first sensor identifying the object being the face of the user.

5. The computer-implemented method of claim 1, wherein the plurality of potential operations comprises at least one of logging the user into the mobile device, logging the user into a user profile of the mobile device, or providing access to secure content of the mobile device.

6. The computer-implemented method of claim 1, further comprising:

fetching, prior to receiving the second information, the user characteristic from local memory of the mobile device based at least in part on the portion of the face of the user being determined from the first information, wherein the operation is determined based at least in part on a match between the facial characteristics of the user and the user information according to a comparison of the second information to the user information.

7. A device, comprising:

a screen configured to render a user interface;

a first sensor configured to sense first information about an object independent of contact between the object and the screen;

a second sensor configured to sense second information about characteristics of the object;

a memory that stores computer-executable instructions; and

a processor configured to access the memory and to execute the computer-executable instructions to collectively at least;

receive, from the first sensor, the first information about the object, the first information sensed by the first sensor independent of contact with the screen;

determine that the object comprises at least part of a face based at least in part on the first information;

activate the second sensor based at least in part on the at least part of the face being determined from the first information;

receive, from the second sensor, the second information about the characteristics of the object, the second information indicating a characteristic of the face; and

perform an operation of the device based at least in part on the characteristic of the face corresponding to user information.

8. The device of claim 7, wherein the first information identifies that the part of the face is within a threshold distance of the screen.

9. The device of claim 7, wherein the characteristic of the face comprises at least one of a facial hair configuration, a haircut, an angle of view of the face, or a facial expression.

10. The device of claim 9, wherein the operation comprises providing access to a user interface of the device if at least one characteristic of the face matches the user information.

11. The device of claim 9, wherein the operation comprises providing access to a user interface of the device if at least one characteristic of the face and the facial expression match the user information.

12. The device of claim 7, wherein the first sensor or the second sensor are configured to generate a sensor grid for sensing the object independent of contact with the screen, the sensor grid generated external to the screen.

13. The device of claim 7, wherein the user information comprises facial characteristic data of a user associated with the user information.

14. The device of claim 7, wherein the operation to be performed comprises logging a user into the device, logging the user into a user profile, or providing access to secure content.

15. The device of claim 7, wherein activating the second sensor comprises activating a plurality of sensors configured with a fidelity higher than the first sensor.

16. The device of claim 7, wherein performing the operations comprises at least one of providing a request to a server, providing information to an external computing system, requesting that information be processed by a web service, or processing data associated with the operation.

17. A system, comprising:

a screen;

a first sensor configured to sense an object external to the screen;

a second sensor configured to sense a characteristic of the object;

a memory that stores computer-executable instructions; and

a processor configured to access the memory and execute the computer-executable instructions to collectively at least;

identify, based at least in part on first information obtained by the first sensor, that the object external to the screen comprises at least part of a face;

activate the second sensor based at least in part on the object being identified as comprising the at least part of the face;

determine, based at least in part on second information obtained by the second sensor, the characteristic of the object;

compare the characteristic to data stored in the memory that identifies a user of the system; and

provide access to the system when the characteristic matches the data.

18. The system of claim 17, wherein the characteristic comprises a facial expression.

19. The system of claim 18, wherein a level of the access to be provided is based at least in part on the facial expression.

20. The system of claim 19, wherein the level of access is configured by the user.