Motion-based character selection

• US 9,740,297 B2
• Filed: 01/30/2015
• Issued: 08/22/2017
• Est. Priority Date: 03/25/2010
• Status: Active Grant

• Current Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Original Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Inventors: Karakotsios, Kenneth M.
• Primary Examiner(s): Chow, Van
• Application Number:
  US14/610,966
• Publication Number:
  US 20150149961A1
• Time in Patent Office: 935 Days
• CPC Class Codes: G06F 1/1694 the I/O peripheral being a single or a set of motion sensors for pointer control or gesture input obtained by sensing movements of the portable computer
• CPC Class Codes: G06F 2200/1637 Sensing arrangement for detection of housing movement or orientation, e.g. for controlling scrolling or cursor movement on the display of an handheld computer
• CPC Class Codes: G06F 2203/04805 Virtual magnifying lens, i.e. window or frame movable on top of displayed information to enlarge it for better reading or selection
• CPC Class Codes: G06F 3/013 Eye tracking input arrangements G06F3/015 takes precedence
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0236 using selection techniques to select from displayed items
• CPC Class Codes: G06F 3/0304 Detection arrangements using opto-electronic means constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042
• CPC Class Codes: G06F 3/0346 with detection of the device orientation or free movement in a 3D space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors
• CPC Class Codes: G06F 3/0482 Interaction with lists of selectable items, e.g. menus
• CPC Class Codes: G06F 3/0485 Scrolling or panning
• CPC Class Codes: G06F 3/04886 by partitioning the display area of the touch-screen or the surface of the digitising tablet into independently controllable areas, e.g. virtual keyboards or menus

Abstract

Various interfaces and control schemes are provided that enable a user to enter text or select specific elements of an interface based at least in part upon relative motion. In some embodiments, a user can enter text by tilting a device to locate a specific character, and press on a button or specified area of a device to select that character. In other embodiments, a user can select a character by looking at a character and performing a selection action, a user device able to determine the character by tracking a gaze direction of the user. Any relative movement of a device and/or a user can be used as input, allowing for one-handed or even hands-free entrance of text or other selection of elements of a graphical user interface.

Claims

1. A computer-implemented method, comprising:

receiving first image data captured using one or more imaging sensors associated with an electronic device;

analyzing the first image data to determine a first relative orientation between the electronic device and at least a first portion of an object represented in the first image data;

displaying a plurality of selectable elements on a display element associated with the electronic device;

receiving second image data captured using the one or more imaging sensors;

analyzing the second image data to determine a second relative orientation between the electronic device and at least a second portion of the object represented in the second image data;

determining a first rate of a first change in orientation between the first relative orientation and the second relative orientation;

displaying a first movement of a selection element to a first selectable element of the plurality of selectable elements on the display element such that a first direction of the first movement corresponds to the first change in orientation and a second rate of the first movement corresponds to the first rate of the first change in orientation;

receiving third image data captured using the one or more imaging sensors;

analyzing the third image data to determine a third relative orientation between the electronic device and at least a third portion of the object represented in the third image data;

determining a third rate of a second change in orientation between the second relative orientation and the third relative orientation;

displaying a second movement of the selection element to a second selectable element of the plurality of selectable elements on the display element such that a second direction of the second movement corresponds to the second change in orientation and a fourth rate of the second movement corresponds to the third rate of the second change in orientation;

receiving a selection of the second selectable element; and

performing an action on the electronic device associated with the selection of the second selectable element.

2. The computer-implemented method of claim 1, further comprising:

determining a relative motion of the first portion of the object over a period of time between capture of the first image data and capture of the second image data.

3. The computer-implemented method of claim 2, further comprising:

determining the relative motion of the first portion of the object by analyzing at least one of a separation, a shape, or a size of the first portion of the object relative to one or more features represented in the second image data.

4. The computer-implemented method of claim 1, further comprising:

receiving fourth image data captured using the one or more imaging sensors;

analyzing the fourth image data to determine that the object is not facing the electronic device; and

performing no action on the electronic device in response to receiving a second selection of a third selectable element of the plurality of selectable elements.

5. The computer-implemented method of claim 1, further comprising:

determining the first relative orientation based at least in part on a movement of the electronic device or a movement of the first portion of the object.

6. The computer-implemented method of claim 1, further comprising:

displaying the plurality of selectable elements overlaying content such that the plurality of selectable elements are at least partially transparent with respect to the content.

7. The computer-implemented method of claim 1, further comprising:

providing at least two modes of control over the selection element.

8. The computer-implemented method of claim 7, further comprising:

toggling between the at least two modes of control by at least one of adjusting a relative orientation of the electronic device or performing an input action with respect to the electronic device.

9. The computer-implemented method of claim 1, further comprising:

receiving the selection as at least one of a gesture, a press of a button, or an interaction with a touch-sensitive material.

10. The computer-implemented method of claim 1, further comprising:

determining that the first change in orientation meets or exceeds a threshold amount.

11. A computing device, comprising:

a processor;

a memory device including instructions that, when executed by the processor, cause the computing device to;

receive first image data captured using one or more imaging sensors associated with the computing device;

analyze the first image data to determine a first relative orientation between the computing device and at least a first portion of an object represented in the first image data;

display a plurality of selectable elements on a display element associated with the computing device;

receive second image data captured using the one or more imaging sensors;

analyze the second image data to determine a second relative orientation between the computing device and at least a second portion of the object represented in the second image data;

determine a first rate associated with a first change in orientation between the first relative orientation and the second relative orientation;

display a first movement of a selection element to a first selectable element of the plurality of selectable elements on the display element such that a first direction of the first movement corresponds to the first change in orientation and a second rate of the first movement corresponds to the first rate of the first change in orientation;

receive third image data captured using the one or more imaging sensors;

analyze the third image data to determine a third relative orientation between the computing device and at least a third portion of the object represented in the third image data;

determine a third rate of a second change in orientation between the second relative orientation and the third relative orientation;

display a second movement of the selection element to a second selectable element of the plurality of selectable elements on the display element such that a second direction of the second movement corresponds to the second change in orientation and a fourth rate of the second movement corresponds to the third rate;

receive a selection of the second selectable element; and

perform an action on the computing device associated with the selection of the second selectable element.

12. The computing device of claim 11, wherein the instructions, when executed by the processor, further cause the computing device to:

determine a relative motion of the first portion of the object over a period of time between capture of the first image data and capture of the second image data.

13. The computing device of claim 12, wherein the instructions, when executed by the processor, cause the computing device to:

determine the relative motion of the first portion of the object by analyzing at least one of a separation, a shape or a size of the first portion of the object relative to one or more features represented in the second image data.

14. The computing device of claim 12, wherein the instructions, when executed by the processor, cause the processor to:

receive fourth image data captured using the one or more imaging sensors;

analyze the fourth image data to determine that the object is not facing the computing device; and

perform no action on the computing device in response to receiving a second selection of a third selectable element of plurality of selectable elements.

15. The computing device of claim 11, wherein the instructions, when executed by the processor, further cause the computing device to:

determine the first relative orientation based at least in part on a movement of the computing device or a movement of the first portion of the object.

16. A non-transitory computer-readable storage medium storing instructions that, when executed by a processor, cause a computing device to:

receive first image data captured using one or more imaging sensors associated with the computing device;

analyze the first image data to determine a first relative orientation between the computing device and at least a first portion of an object represented in the first image data;

display a plurality of selectable elements on a display element associated with the computing device;

receive second image data captured using the one or more imaging sensors;

analyze the second image data to determine a second relative orientation between the computing device and at least a second portion of the object represented in the second image data;

determine a first rate associated with a first change in orientation between the first relative orientation and the second relative orientation;

display a first movement of a selection element to a first selectable element of the plurality of selectable elements on the display element such that a first direction of the first movement corresponds to the first change in orientation and a second rate of the first movement corresponds to the first rate of the first change in orientation;

receive third image data captured using the one or more imaging sensors;

analyze the third image data to determine a third relative orientation between the computing device and at least a third portion of the object represented in the third image data;

determine a third rate of a second change in orientation between the second relative orientation and the third relative orientation;

display a second movement of a second selectable element of the plurality of selectable elements on the display element such that a second direction of the second movement corresponds to the second change in orientation and a fourth rate of the second movement corresponds to the third rate of the second change in orientation;

receive a selection of the second selectable element; and

perform an action on the computing device associated with the selection of the second selectable element.

17. The non-transitory computer-readable storage medium of claim 16, wherein the instructions, when executed by the processor, further cause the computing device to:

determine a relative motion of the first portion of the object over a period of time between capture of the first image data and capture of the second image data.

18. The non-transitory computer-readable storage medium of claim 17, wherein the instructions, when executed by the processor, further cause the computing device to:

determine the relative motion of the first portion of the object by analyzing at least one of a separation, a shape, or a size of the first portion of the object relative to one or more features represented in the second image data.

19. The non-transitory computer-readable storage medium of claim 16, wherein the instructions, when executed by the processor, further cause the computing device to:

receive fourth image data captured using the one or more imaging sensors;

analyze the fourth image data to determine that the object is not facing the computing device; and

perform no action on the computing device in response to receiving a second selection of a third selectable element of the plurality of selectable elements.

20. The non-transitory computer-readable storage medium of claim 16, wherein the instructions, when executed by the processor, further cause the computing device to:

determine the first relative orientation based at least in part on a movement of the computing device or a movement of the first portion of the object.