Motion-based character selection

• US 8,947,355 B1
• Filed: 03/25/2010
• Issued: 02/03/2015
• Est. Priority Date: 03/25/2010
• Status: Active Grant

• Current Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Original Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Inventors: Karakotsios, Kenneth M.
• Primary Examiner(s): Chow, Van
• Application Number:
  US12/732,062
• Time in Patent Office: 1,776 Days
• Field of Search: 345/166
• US Class Current: 345/158
• CPC Class Codes: G06F 1/1694 the I/O peripheral being a single or a set of motion sensors for pointer control or gesture input obtained by sensing movements of the portable computer
• CPC Class Codes: G06F 2200/1637 Sensing arrangement for detection of housing movement or orientation, e.g. for controlling scrolling or cursor movement on the display of an handheld computer
• CPC Class Codes: G06F 2203/04805 Virtual magnifying lens, i.e. window or frame movable on top of displayed information to enlarge it for better reading or selection
• CPC Class Codes: G06F 3/013 Eye tracking input arrangements G06F3/015 takes precedence
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0236 using selection techniques to select from displayed items
• CPC Class Codes: G06F 3/0304 Detection arrangements using opto-electronic means constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042
• CPC Class Codes: G06F 3/0346 with detection of the device orientation or free movement in a 3D space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors
• CPC Class Codes: G06F 3/0482 Interaction with lists of selectable items, e.g. menus
• CPC Class Codes: G06F 3/0485 Scrolling or panning
• CPC Class Codes: G06F 3/04886 by partitioning the display area of the touch-screen or the surface of the digitising tablet into independently controllable areas, e.g. virtual keyboards or menus

Abstract

Various interfaces and control schemes are provided that enable a user to enter text or select specific elements of an interface based at least in part upon relative motion. In some embodiments, a user can enter text by tilting a device to locate a specific character, and press on a button or specified area of a device to select that character. In other embodiments, a user can select a character by looking at a character and performing a selection action, a user device able to determine the character by tracking a gaze direction of the user. Any relative movement of a device and/or a user can be used as input, allowing for one-handed or even hands-free entrance of text or other selection of elements of a graphical user interface.

Claims

1. A computer-implemented method of enabling a user to provide input to an electronic device, comprising:

under control of one or more computing systems configured with executable instructions,

determining a default relative orientation of the electronic device using at least in part at least one imaging element of the electronic device, wherein the default relative orientation of the electronic device comprises a relative orientation of the electronic device with respect to an aspect of a user of the electronic device;

detecting a change in the relative orientation of the electronic device with respect to the default relative orientation, wherein the change in the relative orientation of the electronic device is with respect to the aspect of the user and is caused, at least in part, by a movement of the electronic device by the user, wherein the change in the relative orientation of the electronic device is detected using at least in part the at least one imaging element, and wherein the electronic device displays a plurality of selectable elements in a graphical user interface;

if the change in relative orientation meets or exceeds a coarse threshold, moving a selection element of the graphical user interface in a direction corresponding to a direction of the change in relative orientation at a first rate;

if the change in relative orientation meets or exceeds a fine threshold, but is less than the coarse threshold, moving the selection element in a direction corresponding to a direction of the change in relative orientation at a second rate less than the first rate;

if the change in relative orientation is less than the fine threshold but the detected relative orientation is different from the default relative orientation, maintaining a position of the selection element with respect to one of the selectable elements currently associated with the selection element; and

in response to receiving a selection action, providing the selectable element currently associated with the selection element as input to the electronic device.

2. The computer-implemented method of claim 1, wherein the plurality of selectable characters comprise alphanumeric characters arranged in a substantially circular, substantially linear, or QWERTY keyboard layout.

3. The computer-implemented method of claim 1, wherein the change in relative orientation is determined using at least one of an orientation-determining element and an image capture element.

4. The computer-implemented method of claim 1, wherein the change in relative orientation is able to meet or exceed at least one additional threshold associated with at least one additional rate of movement of the selection element.

5. A computer-implemented method of enabling a user to provide input to an electronic device, comprising:

under control of one or more computing systems configured with executable instructions,

determining a default relative orientation of the electronic device using at least in part at least one imaging element of the electronic device, wherein the default relative orientation of the electronic device comprises a relative orientation of the electronic device with respect to an aspect of a user of the electronic device;

detecting a change in the relative orientation of the electronic device with respect to the default relative orientation, wherein the change in the relative orientation of the electronic device is with respect to the aspect of the user and is caused, at least in part, by a movement of the electronic device by the user, wherein the change in the relative orientation of the electronic device is detected using at least in part the at least one imaging element, and wherein the electronic device displays a plurality of selectable elements in a graphical user interface;

moving a selection element of the graphical user interface in a direction corresponding to a direction of the change in relative orientation at a rate determined at least in part by an amount of the detected change in relative orientation, the amount being detected using at least in part the at least one imaging element, wherein a position of the selection element with respect to one of the plurality of selectable elements currently associated with the selection element is maintained in response to the detected relative orientation being different from the default relative orientation and the change in relative orientation being less than a fine threshold; and

in response to receiving a selection action, providing the selectable element currently associated with the position of the selection element as input to the electronic device.

6. The computer-implemented method of claim 5, wherein the aspect of the user of the electronic device includes a physical aspect associated with at least one of a head of the user or a face of the user.

7. The computer-implemented method of claim 5, wherein the aspect of the user comprises a gaze direction of the user with respect to the electronic device, the selection element programmed to move in a direction and at a rate corresponding to a change in the gaze direction of the user.

8. The computer-implemented method of claim 5, wherein determining a default relative orientation of the electronic device includes locating a relative position of at least one object using at least one imaging element, the amount of change in the relative orientation of the electronic device being determined at least in part by monitoring a change in position of the at least one object in at least one image captured by the at least one imaging element.

9. The computer-implemented method of claim 5, wherein the selection action comprises at least one of pressing a button or interacting with a touch-sensitive material.

10. The computer-implemented method of claim 5, wherein the plurality of selectable elements correspond to alphanumeric characters, and wherein manipulation of the selection element through changes in relative orientation enable a user to input text using one-handed operation.

11. The computer-implemented method of claim 5, wherein the plurality of selectable elements are at least temporarily displayed over content displayed on the electronic device, the selectable elements being at least partially transparent with respect to the content.

12. The computer-implemented method of claim 5, wherein the graphical user interface provides at least two modes of control over the rate of movement of the selection element.

13. The computer-implemented method of claim 12, wherein the user is able to toggle between the modes of control by at least one of adjusting the relative orientation of the device or performing an input action with respect to the device.

14. The computer-implemented method of claim 12, wherein the electronic device comprises a touch sensitive material over at least a portion of an exterior of the electronic device, the user being able to provide a variety of inputs with respect to the graphical user interface using a plurality of different motions with respect to the touch sensitive material.

15. The computer-implemented method of claim 5, wherein the selection action is performed using a pressable button on a back or side of the device.

16. The computer-implemented method of claim 5, wherein the selection action is disabled when the user is not looking substantially at the electronic device.

17. The computer-implemented method of claim 5, wherein an input mode with respect to the graphical user interface changes when the user is not looking substantially at the electronic device.

18. The computer-implemented method of claim 5, wherein the selection element is moved at a rate determined at least in part using a continuous function or step function of the amount of the detected change in relative orientation.

19. A computer-implemented method of enabling a user to provide input to an electronic device, comprising:

under control of one or more computing systems configured with executable instructions, comprising;

determining a default relative orientation of the electronic device using at least in part at least one imaging element of the electronic device, wherein the default relative orientation of the electronic device comprises a relative orientation of the electronic device with respect to an aspect of a user of the electronic device;

detecting a change in the relative orientation of the electronic device with respect to the default relative orientation, wherein the change in the relative orientation of the electronic device is with respect to the aspect of the user and is caused, at least in part, by a movement of the electronic device by the user, wherein the change in the relative orientation of the electronic device is detected using at least in part the at least one imaging element, and wherein the electronic device displays a plurality of selectable elements in a graphical user interface;

moving a selection element of the graphical user interface in a direction corresponding to a direction of the change in relative orientation, wherein a position of the selection element with respect to one of the plurality of selectable elements currently associated with the selection element is maintained in response to the detected relative orientation being different from the default relative orientation and the change in relative orientation being less than a fine threshold; and

in response to receiving a selection action, providing the selectable element currently associated with the position of the selection element as input to the electronic device.

20. A system, comprising:

a processor;

at least one imaging element;

a display element; and

a memory device including instructions that, when executed by the processor, cause the processor to;

determine a default relative orientation of the computing device using at least in part the at least one imaging element, wherein the default relative orientation of the computing device comprises a relative orientation of the computing device with respect to an aspect of a user of the computing device;

detect a change in the relative orientation of the computing device with respect to the default relative orientation, wherein the change in the relative orientation of the electronic device is with respect to the aspect of the user and is caused, at least in part, by a movement of the electronic device by the user, wherein the change in the relative orientation of the computing device is detected using at least in part the at least one imaging element, and wherein the computing device displays via the display element a plurality of selectable elements in a graphical user interface;

move a selection element of the graphical user interface in a direction corresponding to a direction of the change in relative orientation at a rate determined at least in part by an amount of the detected change in relative orientation, wherein a position of the selection element with respect to one of the plurality of selectable elements currently associated with the selection element is maintained in response to the detected relative orientation being different from the default relative orientation and the change in relative orientation being less than a fine threshold; and

in response to receiving a selection action, provide the selectable element currently associated with the position of the selection element as input to the computing device.

21. The system of claim 20, wherein the default relative orientation of the electronic device comprises a relative orientation of the electronic device with respect to an aspect of a user of the electronic device, the default relative orientation being determined at least in part using at least one imaging element of the electronic device.

22. The system of claim 21, wherein the aspect of the user comprises a gaze direction of the user with respect to the electronic device, the selection element programmed to move in a direction and at a rate corresponding to a change in the gaze direction of the user.

23. The system of claim 20, wherein the selection action comprises at least one of pressing a button or interacting with a touch-sensitive material on the computing device.

24. The system of claim 20, wherein the computing device comprises a touch sensitive material over at least a portion of an exterior of the computing device, the user being able to provide a variety of inputs with respect to the graphical user interface using a plurality of different motions with respect to the touch sensitive material.

25. A non-transitory computer-readable storage medium storing instructions for enabling a user to provide input to a computing device, the instructions when executed by a processor causing the processor to:

determine a default relative orientation of the computing device using at least in part the at least one imaging element, wherein the default relative orientation of the computing device comprises a relative orientation of the computing device with respect to an aspect of a user of the computing device;

detect a change in the relative orientation of the computing device with respect to the default relative orientation, wherein the change in the relative orientation of the electronic device is with respect to the aspect of the user and is caused, at least in part, by a movement of the electronic device by the user, wherein the change in the relative orientation of the computing device is detected using at least in part the at least one imaging element, and wherein the computing device displays a plurality of selectable elements in a graphical user interface;

move a selection element of the graphical user interface in a direction corresponding to a direction of the change in relative orientation at a rate determined at least in part by an amount of the detected change in relative orientation, wherein a position of the selection element with respect to one of the plurality of selectable elements currently associated with the selection element is maintained in response to the detected relative orientation being different from the default relative orientation and the change in relative orientation being less than a fine threshold; and

in response to receiving a selection action, provide the selectable element currently associated with the position of the selection element as input to the computing device.

26. The non-transitory computer-readable storage medium of claim 25, wherein the default relative orientation of the electronic device comprises a relative orientation of the electronic device with respect to an aspect of a user of the electronic device, the default relative orientation being determined at least in part using at least one imaging element of the electronic device.

27. The non-transitory computer-readable storage medium of claim 26, wherein the aspect of the user comprises a gaze direction of the user with respect to the electronic device, the selection element programmed to move in a direction and at a rate corresponding to a change in the gaze direction of the user.

28. The non-transitory computer-readable storage medium of claim 25, wherein the selection action comprises at least one of pressing a button or interacting with a touch-sensitive material on the computing device.

29. The non-transitory computer-readable storage medium of claim 25, wherein the computing device comprises a touch sensitive material over at least a portion of an exterior of the computing device, the user being able to provide a variety of inputs with respect to the graphical user interface using a plurality of different motions with respect to the touch sensitive material.