MOVEMENT RECOGNITION AS INPUT MECHANISM

• US 20100125816A1
• Filed: 12/10/2008
• Published: 05/20/2010
• Est. Priority Date: 11/20/2008
• Status: Active Grant

• Current Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Original Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Inventors: Bezos, Jeffrey P.
• Granted Patent: US 8,788,977 B2
• Time in Patent Office: Days
• US Class Current: 715/863
• CPC Class Codes: G01C 21/367 Details, e.g. road map scale, orientation, zooming, illumination, level of detail, scrolling of road map or positioning of current position marker
• CPC Class Codes: G06F 1/1626 with a single-body enclosure integrating a flat display, e.g. Personal Digital Assistants [PDAs]
• CPC Class Codes: G06F 1/1694 the I/O peripheral being a single or a set of motion sensors for pointer control or gesture input obtained by sensing movements of the portable computer
• CPC Class Codes: G06F 2200/1637 Sensing arrangement for detection of housing movement or orientation, e.g. for controlling scrolling or cursor movement on the display of an handheld computer
• CPC Class Codes: G06F 3/012 Head tracking input arrangements
• CPC Class Codes: G06F 3/013 Eye tracking input arrangements G06F3/015 takes precedence
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0482 Interaction with lists of selectable items, e.g. menus

Abstract

The detection of relative motion or orientation between a user and a computing device can be used to control aspects of the device. For example, the computing device can include an imaging element and software for locating positions, shapes, separations, and/or other aspects of a user'"'"'s facial features relative to the device, such that an orientation of the device relative to the user can be determined. A user then can provide input to the device by performing actions such as tilting the device, moving the user'"'"'s head, making a facial expression, or otherwise altering an orientation of at least one aspect of the user with respect to the device. Such an approach can be used in addition to, or as an alternative to, conventional input devices such as keypads and touch screens.

Claims

1. A method of interacting with a graphical user interface on a portable computing device, comprising:

obtaining a first image using an imaging element of the portable computing device, the first image including a facial feature of a user of the portable computing device;

obtaining a second image using the imaging element, the second image including the facial feature of the user of the portable computing device;

analyzing the obtained first image and second image, with the portable computing device, to determine a change in the position of the facial feature of the user with respect to the portable computing device; and

altering information displayed within the graphical user interface based at least in part on the determined change, wherein the information displayed within the graphical user interface excludes a representation of either or both of the obtained first and second images.

2. A method according to claim 1, wherein:

obtaining the second image is in response to a movement of the portable computing device.

3. A method according to claim 1, wherein:

altering the information displayed within the graphical user interface includes performing a function based at least in part on the determined change, the function controlling at least one aspect of the graphical user interface of the computing device.

4. A method according to claim 1, further comprising:

analyzing the obtained first and second images to determine a change in a set of measurable aspects corresponding to facial features of the user of the computing device; and

wherein altering the information displayed within the graphical user interface is based at least in part on the determined change of the set of measurable aspects.

5. A method according to claim 4, wherein the set of measurable aspects includes at least one of a dimension of at least one of the user'"'"'s eyes, a separation of features on the user'"'"'s face, a relative location of a feature on the user'"'"'s face, and a relative orientation between features on the user'"'"'s face.

6. A method according to claim 1, wherein:

analyzing the obtained first image and second image to determine a change in the position of the facial feature of the user with respect to the portable computing device includes determining a change in relative orientation between the user and the portable computing device.

7. A method according to claim 1, wherein altering the information displayed within the graphical user interface includes at least one of:

moving a cursor with respect to the graphical user interface in a direction corresponding to the determined change; and

adjusting a point of view used to generate a two-dimensional rendering of a three-dimensional image on the computing device.

8. A method according to claim 1, further comprising:

obtaining a third image using the imaging element, the third image including the facial feature of the user of the portable computing device;

analyzing the obtained second image and third image, with the portable computing device, to determine a second change in the position of the facial feature of the user with respect to the portable computing device, the second change based at least in part on an expression of the user adjusting at least one of a shape, position, and dimension of the facial feature of the user; and

altering the information displayed within the graphical user interface based at least in part on the second change.

9. A method according to claim 1, wherein:

altering the information displayed within the graphical user interface includes at least one of;

navigating among items associated with an electronic catalog;

navigating a file system; and

selecting an item from a rotating selection mechanism.

10. A method for selecting an alphanumeric character in a computing device, comprising:

analyzing image information of a user, the image information obtained using an imaging element of the computing device, to detect a change in an orientation of the computing device with respect to the user of the computing device;

in response to detecting the change in the orientation of the computing device, identifying the alphanumeric character available for selection; and

receiving a selection of the alphanumeric character.

11. A method according to claim 10, further comprising:

displaying a set of characters using a display element of the computing device; and

at a time when a cursor is in a position to select at least one character in the set of characters, performing the step of receiving the selection of the at least one character.

12. A method according to claim 11, further comprising:

segmenting the set of characters into a plurality of subsets of characters, wherein the selection of at least one character in the set of characters comprises receiving a selection of at least one of the subsets of characters; and

segmenting the selection of the at least one subset of characters into a second plurality of subsets of characters.

13. A method according to claim 10, further comprising:

using an orientation detection element of the computing device in conjunction with analyzing the image information to determine the change in orientation;

wherein a cursor is configured to be directed toward the at least one character to be selected according to the first change in the orientation of the computing device.

14. A method according to claim 13, wherein the orientation detection element is at least one of an accelerometer, a motion detector, an electronic compass, a global-positioning system receiver, and an electronic level.

15. A method according to claim 10, further comprising:

in response to receiving a selection of a user-selectable element, reducing the movement capability of the cursor by one degree of freedom,

wherein a cursor is configured to move in at least two degrees of freedom with respect to the display element before receiving the second selection.

16. A method according to claim 10, wherein:

the change in the orientation of the computing device corresponds to a user tilting the computing device.

17. A method according to claim 10, wherein:

identifying the alphanumeric character available for selection includes at least one of;

hovering a cursor over the alphanumeric character;

changing a color of the alphanumeric character;

changing a shape of the alphanumeric character;

changing a background color of around the alphanumeric character; and

displaying a shape around the alphanumeric character.

18. A method of interacting with a computing device, comprising:

analyzing image information obtained using an imaging element of the computing device to determine a first relative orientation of at least one feature of a user to the computing device;

analyzing device orientation information obtained using an orientation detection element of the computing device to determine a second relative orientation of the computing device with respect to a surrounding environment;

detecting a change in at least one of the first and second relative orientations; and

in response to detecting the change, performing a desired function.

19. A method according to claim 18, wherein:

analyzing image information includes capturing at least one image from the portable computing device to determine the position of the at least one feature of a user with respect to the computing device, and

detecting a change in at least one of position of the at least one feature of the user and the orientation of the computing device includes capturing at least one additional image from the portable computing device and determining a difference in position of the at least one feature.

20. A method according to claim 18, wherein:

the step of performing a desired function for the detected change includes altering the information displayed within a graphical user interface based at least in part on the detected change, wherein the graphical user interface of the device excludes a representation of the image information.

21. A method according to claim 18, further comprising:

determining whether the detected change at least meets an input threshold corresponding to the change before determining the desired function.

22. A method according to claim 18, wherein:

the orientation detection element is one of an accelerometer, a motion detector, an electronic compass, and an electronic level.

23. A method according to claim 18, wherein:

the at least one feature corresponding to the user includes at least one facial feature of the user.

24. A method according to claim 18, wherein:

analyzing the image information to determine a relative location of a user includes determining an approximate location of each eye of a user in the image information.

25. A method according to claim 18, wherein:

performing a desired function includes at least one of viewing and selecting an item.

26. A method according to claim 18, further comprising:

analyzing the image information to determine an identity of the user.

27. A method according to claim 26, further comprising:

determining a plurality of inputs corresponding to a plurality of changes for the determined identity of the user.

28. A method according to claim 26, further comprising:

capturing calibration information for each determined identity before analyzing the image information.

29. A method according to claim 18, further comprising:

prompting the user to confirm the desired function before performing the function.

30. A method according to claim 29, further comprising:

enabling the user to confirm the input by changing at least one of the first and second relative orientations.

31. A computing device, comprising:

a processor;

a display element for displaying a graphical user interface;

a memory device including instructions operable to be executed by the processor to perform a set of actions;

an imaging element operable to obtain image information corresponding to a relative viewing area of the computing device; and

the processor being operable to;

obtain a first and second image using the imaging element, the first and second images including a feature of a user of the computing device;

analyze the obtained first image and second image to determine a change in the position of the feature of the user with respect to the computing device; and

alter information displayed within the graphical user interface based at least in part on the determined change, the information displayed within the graphical user interface excluding a representation of the obtained first image, the obtained second image, or both the obtained first and second images.

32. A method of receiving input for a computing device, comprising:

analyzing image information to determine at least one relative location of at least one feature of a user with respect to the computing device, the image information being obtained using an imaging element of the computing device; and

in response to detecting a change in the at least one relative location of at least one feature of the user that at least meets an input threshold corresponding to the change, selecting an input function corresponding to the detected change and executing the selected input function,

wherein a user is able to define the input function corresponding to the detected change.

33. A method according to claim 32, wherein:

detecting a change in the at least one relative location of at least one feature of the user includes detecting a change in at least one facial feature of the user.

34. A method according to claim 33, wherein:

detecting a change in the at least one location of at least one feature of the user includes detecting a change in at least one of a plurality of relative locations for the at least one facial feature of the user, at least one facial feature of the user having a plurality of relative locations associated therewith such that a change in expression for that facial feature results in a change in at least one relative position of the facial feature.

35. A method according to claim 32, wherein:

the step of determining an input function corresponding to the detected change and executing the selected input function includes altering information displayed within a graphical user interface associated with the computing device based at least in part on the detected change, wherein the graphical user interface of the computing device excludes a representation of the image information.

36. A computing device, comprising:

a processor;

a display element;

an imaging element operable to capture image information, the imaging element being oriented such that a user viewing the display element will be at least partially captured in the image information; and

a memory device including instructions that, when executed by the processor, cause the processor to;

analyze the image information to determine a set of measurable aspects corresponding to facial features of the user of the computing device;

determine a relative orientation of the user with respect to the computing device based at least in part upon values for the set of measurable aspects;

determine a change in the relative orientation of the user with respect to the computing device, the change in orientation being based at least in part on a change in position of the computing device; and

perform a function corresponding to the determined change in the relative orientation,

wherein a user of the computing device is able to define a function to be performed in response to each of a plurality of determinable changes in the relative orientation.

37. A computing device according to claim 36 wherein the memory device further includes instructions that, when executed by the processor, cause the processor to:

determine whether the change at least meets an input threshold corresponding to the change before performing the action.

38. A computing device according to claim 36, wherein:

the step of performing the function includes altering the information displayed within a graphical user interface on a display element of the computing device based at least in part on the determined change, wherein the information displayed within the graphical user interface of the device excludes a representation of the image information.

39. A computer program product embedded in a computer-readable medium for controlling a computing device, comprising:

program code for analyzing image information to determine at least one relative location of at least one feature of a user with respect to the computing device, the image information being obtained using at least one imaging element of the computing device positioned such that a representation of the user of the computing device is at least partially included in the image information;

program code for determining, in response to detecting a change in the at least one relative location of at least one feature of the user, whether the change at least meets an input threshold corresponding to the change; and

program code for determining an input corresponding to a detected change that at least meets the input threshold, and executing an action on the computing device in response to the determined input,

wherein the program code for executing an action on the computing device includes altering information displayed within a graphical user interface on a display element of the computing device based at least in part on the detected change, the information displayed on the graphical user interface of the computing device excluding a representation of the user.

40. A computer program product according to claim 39, further comprising:

program code for determining an orientation of the portable computing device, the portable computing device including at least one element operable to determine the orientation; and

program code for determining a change in the orientation of the computing device,

wherein the program code for determining whether the change at least meets an input threshold corresponding to the change includes instructions for analyzing both changes in orientation and changes in relative location of at least one feature of the user.

41. A computer program product according to claim 40, wherein:

a user of the computing device is able to define an action to be executed in response to each of a plurality of detectable changes.

42. A method for altering the display of an element in a display of a computing device, comprising:

analyzing image information of a user, the image information obtained using an imaging element of the computing device, to detect a change in an orientation of the computing device with respect to the user of the computing device; and

in response to detecting the change in the orientation of the computing device, altering the display of the element.

43. A method according to claim 42, wherein:

the imagine element is a high speed video camera.