Movement recognition as input mechanism

• US 9,304,583 B2
• Filed: 06/06/2014
• Issued: 04/05/2016
• Est. Priority Date: 11/20/2008
• Status: Active Grant

• Current Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Original Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Inventors: Bezos, Jeffrey P.
• Primary Examiner(s): TRAPANESE, WILLIAM C
• Application Number:
  US14/298,577
• Publication Number:
  US 20140285435A1
• Time in Patent Office: 669 Days
• Field of Search: None
• US Class Current: 1/1
• CPC Class Codes: G01C 21/367 Details, e.g. road map scale, orientation, zooming, illumination, level of detail, scrolling of road map or positioning of current position marker
• CPC Class Codes: G06F 1/1626 with a single-body enclosure integrating a flat display, e.g. Personal Digital Assistants [PDAs]
• CPC Class Codes: G06F 1/1694 the I/O peripheral being a single or a set of motion sensors for pointer control or gesture input obtained by sensing movements of the portable computer
• CPC Class Codes: G06F 2200/1637 Sensing arrangement for detection of housing movement or orientation, e.g. for controlling scrolling or cursor movement on the display of an handheld computer
• CPC Class Codes: G06F 3/012 Head tracking input arrangements
• CPC Class Codes: G06F 3/013 Eye tracking input arrangements G06F3/015 takes precedence
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0482 Interaction with lists of selectable items, e.g. menus

Abstract

The detection of relative motion or orientation between a user and a computing device can be used to control aspects of the device. For example, the computing device can include an imaging element and software for locating positions, shapes, separations, and/or other aspects of a user'"'"'s facial features relative to the device, such that an orientation of the device relative to the user can be determined. A user then can provide input to the device by performing actions such as tilting the device, moving the user'"'"'s head, making a facial expression, or otherwise altering an orientation of at least one aspect of the user with respect to the device. Such an approach can be used in addition to, or as an alternative to, conventional input devices such as keypads and touch screens.

Claims

1. A computer-implemented method, comprising:

obtaining a first image using an imaging element of a computing device, the first image including a first representation of a user;

obtaining a second image using the imaging element, the second image including a second representation of the user;

determining a relative motion of the computing device with respect to the user based at least in part by detecting at least a first change of a first feature of the user between the first image and the second image, and a second change to a second feature of the user between the first image and the second image;

determining that the relative motion corresponds to an input command for the computing device; and

generating user interface information for display on a display element of the computing device based on the input command.

2. The computer-implemented method of claim 1, further comprising:

displaying the user interface information on the display element at least at substantially a same time as one or more of obtaining the first image or obtaining the second image.

3. The computer-implemented method of claim 1, further comprising:

determining that the relative motion of the computing device with respect to the user corresponds to a threshold amount of movement.

4. The computer-implemented method of claim 1, wherein detecting at least the first change and the second change includes:

detecting the first feature and the second feature in the first image;

detecting the first feature and the second feature in the second image;

comparing the first feature in the first image to the first feature in the second image; and

comparing the second feature in the first image to the second feature in the second image.

5. The computer-implemented method of claim 4, wherein comparing the first feature in the first image to the first feature in the second image includes:

measuring a first distance between a reference feature of the user and the first feature in the first image; and

measuring a second distance between the reference feature and the first feature in the second image.

6. The computer-implemented method of claim 5, wherein the reference feature is at least one of a head, a face, an edge of the head or the face, an eyebrow, an eye, a nose, a mouth, an arm, a leg, or a hip.

7. The computer-implemented method of claim 4, wherein comparing the first feature in the first image to the first feature in the second image includes:

determining a first position of the first feature in the first image; and

determining a second position of the first feature in the second image.

8. The computer-implemented method of claim 4, wherein comparing the first feature in the first image to the first feature in the second image includes:

measuring a dimension of the first feature; and

measuring a corresponding dimension of the first feature in the second image.

9. The computer-implemented method of claim 4, wherein comparing the first feature in the first image to the first feature in the second image includes:

determining a first shape of the first feature in the first image; and

determining a second shape of the first feature in the second image.

10. The computer-implemented method of claim 1, 9, wherein the relative motion corresponds to at least one of a tilting of the computing device in at least one of a left, right, forward, backward, or diagonal direction with respect to the user, or a movement of the computing device from a first distance to a second distance with respect to the user.

11. The computer-implemented method of claim 1, wherein the input command corresponds to at least one of a movement of a virtual cursor, navigation of a graphical user interface, a movement of a graphical element, zooming in or zooming out of the graphical element, a selection of a user interface element, or a rotation of a selection mechanism.

12. The computer-implemented method of claim 1, 9, wherein the relative motion includes a movement of at least one of the first feature or the second feature.

13. The computer-implemented method of claim 1, further comprising:

obtaining information from a motion detection element of the computing device,

wherein determining the relative motion is further based at least in part on the information from the motion detection element.

14. A computing device, comprising:

a processor;

a display element;

an imaging element; and

a memory element including instructions that, upon being executed by the processor, cause the computing device to;

obtain a first image using the imaging element, the first image including a first representation of a user;

obtain a second image using the imaging element, the second image including a second representation of the user;

determine a relative motion of the computing device with respect to the user based at least in part by detecting at least a first change of a first feature of the user between the first image and the second image, and a second change to a second feature of the user between the first image and the second image;

determine that the relative motion corresponds to an input command for the computing device; and

generate user interface information for display on the display element based on the input command.

15. The computing device of claim 14, wherein the instructions upon being executed further cause the computing device to:

display the user interface information on the display element,

wherein the user interface information includes a changed perspective of a three-dimensional graphical element.

16. The computing device of claim 14, wherein the instructions upon being executed further cause the computing device to:

display the user interface information on the display element,

wherein the user interface information includes a changed view of a multi-view graphical user interface.

17. A non-transitory computer-readable storage medium including instructions that, upon being executed by a processor of a computing device, cause the computing device to:

obtain a first image using an imaging element of the computing device, the first image including a first representation of a user;

obtain a second image using the imaging element, the second image including a second representation of the user;

determine a relative motion of the computing device with respect to the user based at least in part by detecting at least a first change of a first feature of the user between the first image and the second image, and a second change to a second feature of the user between the first image and the second image;

determine that the relative motion corresponds to an input command for the computing device; and

generate user interface information for display on a display element of the computing device based on the input command.

18. The non-transitory computer-readable storage medium of claim 17, detecting at least the first change and the second change includes causing the computing device to:

detect the first feature and the second feature in the first image;

detect the first feature and the second feature in the second image;

compare the first feature in the first image to the first feature in the second image; and

compare the second feature in the first image to the second feature in the second image.

19. The non-transitory computer-readable storage medium of claim 18, wherein the instructions upon being executed to cause the computing device to compare the first feature in the first image to the first feature in the second image includes causing the computing device to:

measure a first distance between a reference feature of the user and the first feature in the first image; and

measure a second distance between the reference feature and the first feature in the second image.

20. The non-transitory computer-readable storage medium of claim 18, wherein the instructions upon being executed to cause the computing device to compare the first feature in the first image to the first feature in the second image includes causing the computing device to:

determine a first position of the first feature in the first image; and

determine a second position of the first feature in the second image.