Managing sensory information of a user device

• US 9,164,609 B2
• Filed: 03/13/2013
• Issued: 10/20/2015
• Est. Priority Date: 03/13/2013
• Status: Active Grant

• Current Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Original Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Inventors: Ricks, James Edward
• Primary Examiner(s): Edun, Muhammad N
• Application Number:
  US13/799,150
• Publication Number:
  US 20140267167A1
• Time in Patent Office: 951 Days
• Field of Search: None
• US Class Current: 1/1
• CPC Class Codes: G06F 1/3231 Monitoring the presence, absence or movement of users
• CPC Class Codes: G06F 2203/04106 Multi-sensing digitiser, i.e. digitiser using at least two different sensing technologies simultaneously or alternatively, e.g. for detecting pen and finger, for saving power or for improving position detection
• CPC Class Codes: G06F 2203/04108 Touchless 2D- digitiser, i.e. digitiser detecting the X/Y position of the input means, finger or stylus, also when it does not touch, but is proximate to the digitiser's interaction surface without distance measurement in the Z direction
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/041 Digitisers, e.g. for touch screens or touch pads, characterised by the transducing means
• CPC Class Codes: G06F 3/0416 Control or interface arrangements specially adapted for digitisers
• CPC Class Codes: G06F 3/0421 by interrupting or reflecting a light beam, e.g. optical touch-screen
• CPC Class Codes: G06F 3/0425 using a single imaging device like a video camera for tracking the absolute position of a single or a plurality of objects with respect to an imaged reference surface, e.g. video camera imaging a display or a projection screen, a table or a wall surface, on which a computer generated image is displayed or projected tracking a projected light spot to determine a position on a display surface G06F3/0386
• CPC Class Codes: G06F 3/043 using propagating acoustic waves
• CPC Class Codes: G06F 3/0436 in which generating transducers and detecting transducers are attached to a single acoustic waves transmission substrate
• CPC Class Codes: G06F 3/0488 using a touch-screen or digitiser, e.g. input of commands through traced gestures
• CPC Class Codes: G06F 3/04883 for inputting data by handwriting, e.g. gesture or text
• CPC Class Codes: G06V 40/172 Classification, e.g. identification
• CPC Class Codes: Y02D 10/00 Energy efficient computing, e.g. low power processors, power management or thermal management

Abstract

External mobile device sensors may be provided that are configured to manage sensory information associated with motion of objects external to the mobile device. In some examples, the object motion may be detected independent of contact with the device. In some examples, a device may include a screen with a first sensor (e.g., a touch sensor). The device may also include at least a second sensor external to the screen. Instructions may be executed by a processor of the device to at least determine when an object is hovering over a first graphical user interface (GUI) element of the screen. Additionally, in some cases, a second GUI element may be provided on the screen such that the second GUI element is rendered on the screen adjacent to a location under the hovering object.

Claims

1. A computer-implemented method for managing device sensory information, comprising:

under control of a computer system configured with executable instructions,

monitoring, by a first sensor of a mobile device, movement of an object external to and independent of contact with a screen of the mobile device;

receiving, from the first sensor, information indicative of the movement of the object when the movement external to and independent of contact with the screen of the mobile device is identified;

determining a potential action of a user of the mobile device based at least in part on the information indicative of the movement of the object;

performing an instruction corresponding to the potential action of the user prior to receiving an indication of a request to perform the potential action;

receiving the indication of the request from a second sensor; and

providing information corresponding to the performed instruction based at least in part on the indication of the request.

2. The computer-implemented method of claim 1, wherein the first sensor is configured to be at least one of an infrared sensor or an ultrasonic sensor and the second sensor is configured to be a touch screen sensor.

3. The computer-implemented method of claim 1, wherein the first sensor is configured to be an ambient sensor and the second sensor is configured to be an effective sensor.

4. The computer-implemented method of claim 3, wherein the ambient sensor is configured with less fidelity than the effective sensor.

5. The computer-implemented method of claim 4, further comprising activating the second sensor, when the second sensor is not active, based at least in part on the information received from the first sensor identifying the movement of the object.

6. The computer-implemented method of claim 1, wherein the information indicative of the movement of the object includes at least one of a speed of movement of the object, a direction of the movement of the object, a difference in time between the movement of the object monitored by the first sensor and movement of the object identified by the second sensor, or a difference in time between detection of the object monitored by the first sensor and detection of the object identified by the second sensor.

7. The computer-implemented method of claim 1, wherein performing the instruction includes at least one of providing a request to a server, providing information to an external computing system, requesting that information be processed by a web service, or processing data associated with the one or more potential actions.

8. A computer-implemented method for managing device sensory information, comprising:

under control of a computer system configured with executable instructions,

identifying, by a first sensor of a device, motion associated with a user, the motion occurring external to a screen of the device and including an object associated with the user moving over a user interface element the user interface element rendered on the screen of the device;

determining one or more potential actions by the user based at least in part on the identified motion;

performing instructions associated with at least a subset of the one or more potential actions prior to receiving a request from the user;

receiving an indication of the request from a second sensor of the device, the second sensor configured to identify contact with the screen of the device; and

providing information to the user based at least in part on the performed instructions upon receipt of the request from the user.

9. The computer-implemented method of claim 8, wherein the object associated the user includes at least one of a hand of the user, a finger of the user, a face of the user, or an item held by the user.

10. The computer-implemented method of claim 8, wherein the one or more potential actions by the user include at least one of a hover over the user interface element or a selection of the user interface element.

11. The computer-implemented method of claim 8, wherein the information provided to the user includes a second user interface element rendered on the screen of the device.

12. The computer-implemented method of claim 11, wherein the user interface element is a first user interface element, and wherein the second user interface element is rendered on an area of the screen of the device adjacent to the first user interface element.

13. The computer-implemented method of claim 12, further comprising determining a viewable area of the screen based at least in part on at least one of a size of the object associated with the user or a location of the object associated with the user, the viewable area being large enough to render the second user interface element, and wherein the second user interface element is rendered in the viewable area.

14. The computer-implemented method of claim 11, wherein the one or more potential actions by the user include a selection of at least one of the first user interface element or the second user interface element rendered on the screen.

15. The computer-implemented method of claim 14, wherein performing the instructions associated with at least a subset of the one or more potential actions includes at least one of sending a request to a server corresponding to the selection of the first user interface element rendered on the screen or sending a request to a server corresponding to the selection of the second user interface element rendered on the screen.

16. The computer-implemented method of claim 14, wherein performing the instructions associated with at least a subset of the one or more potential actions includes at least one of processing data associated with the selection of the first user interface element rendered on the screen or processing data associated with the selection of the second user interface element rendered on the screen.

17. The computer-implemented method of claim 14, wherein performing the instructions associated with at least a subset of the one or more potential actions includes providing an indication on the screen that an object associated with the user is located over a graphical user interface element of the screen.

18. The computer-implemented method of claim 8, wherein the sensor is configured to provide a sensor grid external to the screen of the device, the sensor grid configured to sense at least one of the motion occurring external to the screen of the device and an object located external to the screen of the device that is not in contact with the device.

19. A sensory information management device, comprising:

a screen configured to render a user interface;

a first sensor configured to identify an object independent of contact between the object and the screen;

a second sensor configured to identify contact with the screen;

a memory that stores computer-executable instructions; and

a processor configured to access the memory and to execute the computer-executable instructions to collectively at least;

receive, from the first sensor, first information associated with the object identified by the first sensor independent of contact with the screen;

receive, from the first sensor, second information associated with the object identified by the sensor independent of contact with the screen;

provide a first user element in the user interface rendered on the screen prior to receipt of the first information or the second information;

determine an action to be performed by the sensory information management device based at least in part on the first information, the second information, and an indication received from the second sensor; and

provide a new user interface element in the user interface rendered on the screen based at least in part on the determined action and after providing the first user interface element.

20. The system of claim 19, wherein the first information associated with the object indicates a location of the object with respect to the first user interface element.

21. The system of claim 20, wherein the second information associated with the object indicates an amount of time the object was located at the location.

22. The system of claim 21, wherein the new user interface element includes a hover element that indicates that a user is hovering over the first user interface element, the new user interface element based at least in part on the location and the amount of time the object was located at the location.

23. The system of claim 19, wherein the sensor is configured to generate a sensor grid for identifying the object independent of contact with the screen, the sensor grid generated external to the screen.

24. The system of claim 19, wherein the action to be performed includes logging a user onto the device based at least in part on at least one of the first information or the second information corresponding to a physical characteristic of the user.

25. A system for managing device sensory information, comprising:

a screen;

a first sensor;

a second sensor configured to identify an object external to the screen;

a memory that stores computer-executable instructions; and

a processor configured to access the memory, wherein the processor is configured to execute the computer-executable instructions to collectively at least;

determine when the object is hovering over a first graphical user interface element on the screen; and

provide a second graphical user interface element on the screen such that the second graphical user interface element is rendered on the screen adjacent to a location under the object that is hovering over the first graphical user interface element.

26. The system of claim 25, wherein the first sensor is a touch sensitive sensor.

27. The system of claim 25, wherein the second sensor is configured to detect objects external to the screen that are undetectable by the first sensor.

28. The system of claim 25, wherein the processor is further configured to execute the computer-executable instructions to collectively at least:

determine, based at least in part on information obtained by the second sensor, a location of the screen that is not covered, from view of the user, by the object; and

provide the second graphical user interface element on the determined location that is not covered, from the view of the user, by the object.

29. A computer-implemented method for managing device sensory information, comprising:

identifying, by a sensor of a device, motion associated with a user, the motion occurring external to a screen of the device;

determining one or more potential actions by the user based at least in part on the identified motion;

performing instructions associated with at least a subset of the one or more potential actions prior to receiving a request from the user; and

providing information to the user based at least in part on the performed instructions after receipt of the request from the user and after the request from the user is identified used at least in part on the user making physical contact with the screen.

30. A sensory information management device, comprising:

a screen configured to render a user interface;

a first sensor configured to identify an object independent of contact with the screen;

a second sensor configured to identify contact with the screen;

a memory that stores computer-executable instructions; and

a processor configured to access the memory and to execute the computer-executable instructions to collectively at least;

receive, from the first sensor, first information associated with the object identified b the sensor independent of contact with the screen;

receive, from the first sensor, second information associated with the object identified by the sensor independent of contact with the screen;

determine an action to be performed by the sensory information management device based at least in part on the first information and the second information;

provide a new user interface element in the user interface rendered on the screen based at least in part on the determined action, the first sensor configured to generate a sensor grid for identifying the object independent of contact with the screen, and the sensor grid generated external to the screen;

perform the determined action based at least in part on third information, received from the second sensor, associated with physical contact made at a location of the new user interface element.