Point of view determinations for finger tracking

• US 8,947,351 B1
• Filed: 09/27/2011
• Issued: 02/03/2015
• Est. Priority Date: 09/27/2011
• Status: Active Grant

• Current Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Original Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Inventors: Noble, Isaac S.
• Primary Examiner(s): Lee, Nicholas
• Application Number:
  US13/246,561
• Time in Patent Office: 1,225 Days
• Field of Search: None
• US Class Current: 345/156
• CPC Class Codes: G06F 2203/04108 Touchless 2D- digitiser, i.e. digitiser detecting the X/Y position of the input means, finger or stylus, also when it does not touch, but is proximate to the digitiser's interaction surface without distance measurement in the Z direction
• CPC Class Codes: G06F 3/013 Eye tracking input arrangements G06F3/015 takes precedence
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/042 by opto-electronic means
• CPC Class Codes: G06F 3/0488 using a touch-screen or digitiser, e.g. input of commands through traced gestures
• CPC Class Codes: G06F 3/04883 for inputting data by handwriting, e.g. gesture or text

Abstract

A user can provide input to a computing device by moving a feature or object, such as a user'"'"'s finger, within a field of view of at least one imaging element of the computing device. In order to ensure an accuracy of the determined input, the computing device can also attempt to determine a point of view of the user, such as by determining a relative position of the user'"'"'s face or eyes. By determining a three-dimensional position of a feature and the user'"'"'s point of view, a three-dimensional vector or other directional information can be determined whereby the intersection of that vector with the computing device indicates an intended location of input corresponding to the feature from the user'"'"'s point of view.

Claims

1. A computer-implemented method of determining input for a computing device, comprising:

capturing a first image using a first camera of the computing device and a second image using a second camera of the computing device, the first and second cameras having overlapping fields of view;

analyzing, using at least one processor of the computing device, the first image to determine a first relative position of a fingertip of a user and a first relative position corresponding to eyes of the user;

analyzing, using the at least one processor of the computing device, the second image to determine a second relative position of the fingertip and a second relative position corresponding to the eyes;

based at least in part upon the first relative position of the fingertip, the second relative position of the fingertip, the first relative position corresponding to the eyes, and the second relative position corresponding to the eyes, determining a three-dimensional fingertip position and a position corresponding to the eyes in three-dimensional space;

calculating, using the at least one processor of the computing device, a vector passing through the three-dimensional fingertip position and the position corresponding to the eyes in three-dimensional space; and

determining, using the at least one processor of the computing device, an input location based at least in part upon the vector and a tilt of the computing device with respect to the user.

2. The computer-implemented method of claim 1, wherein movement of the fingertip with respect to the computing device is capable of controlling a desired input location with respect to an interface of the computing device from a perspective of the user of the computing device.

3. The computer-implemented method of claim 1, wherein determining the three-dimensional fingertip position includes:

determining a first feature vector corresponding to the first relative position of the fingertip;

determining a second feature vector corresponding to the second relative position of the fingertip; and

performing vector manipulation to determine the three-dimensional fingertip position based at least in part upon the first feature vector and the second feature vector.

4. The computer-implemented method of claim 1, wherein determining the position corresponding to the eyes in three-dimensional space includes:

determining a first viewing vector corresponding to the first relative position corresponding to the eyes;

determining a second viewing vector corresponding to the second relative position corresponding to the eyes; and

performing vector manipulation to determine the position corresponding to the eyes in three-dimensional space based at least in part upon the first viewing vector and the second viewing vector.

5. The computer-implemented method of claim 1, further comprising:

determining a respective dominance of one eye of the user; and

using the respective dominance to further determine the position corresponding to the eyes in three-dimensional space.

6. The computer-implemented method of claim 1, wherein analyzing the first image to determine the first relative position of the fingertip and the first relative position corresponding to the eyes and analyzing the second image to determine the second relative position of the fingertip and the second relative position corresponding to the eyes includes:

processing the first image and the second image using at least one image recognition or pattern recognition algorithm.

7. The computer-implemented method of claim 1, further comprising:

detecting the fingertip of the user in the first image and the second image; and

determining an intersection of a respective additional vector for each other detected fingertip for purposes of providing additional input to the computing device.

8. A computer-implemented method of determining input to an electronic device, comprising:

under control of one or more computing devices including executable instructions,

obtaining image information captured using at least two image capture elements of the electronic device;

analyzing the image information to determine a first three-dimensional position of a determined feature of a user;

analyzing the image information to determine a second three-dimensional position corresponding to at least one eye of the user; and

calculating an input location on the electronic device where the user is attempting to provide input to the electronic device, the input location corresponding to a tilt of the electronic device with respect to the user and a vector passing through the first three-dimensional position and the second three-dimensional position.

9. The computer-implemented method of claim 8, further comprising:

monitoring changes in the input location over time; and

providing a selection input to the electronic device when the input location corresponds to a selectable element of an interface for at least a minimum period of time corresponding to a selection action.

10. The computer-implemented method of claim 8, further comprising:

monitoring motions of the determined feature;

comparing the motions of the determined feature to a set of input gestures; and

upon one of the motions matching one of the set of input gestures, providing a corresponding input to the electronic device.

11. The computer-implemented method of claim 8, wherein the determined feature includes a portion of a body of the user or an object being held by the user.

12. The computer-implemented method of claim 8, wherein the image information includes at least one of ambient light information and reflected infrared radiation information.

13. The computer-implemented method of claim 8, further comprising:

activating at least one illumination element during obtaining the image information using at least one image capture element.

14. The computer-implemented method of claim 8, wherein the electronic device includes at least four imaging elements, a first pair of imaging elements capturing the image information in an ambient light detection mode and a second pair capturing the image information in an infrared detection mode.

15. The computer-implemented method of claim 8, wherein the first three-dimensional position and the second three-dimensional position are determined using at least one of stereoscopic image analysis, feature size analysis, luminance analysis, or distance information from at least one distance determination element.

16. The computer-implemented method of claim 8, further comprising:

deactivating of obtaining the image information when no determined feature of the user is detected within a specified period of time.

17. The computer-implemented method of claim 8, wherein the input location corresponds to at least one of magnifying a portion of an interface, adjusting a zoom level of content on the electronic device, selecting an interface element, moving the interface element, or navigating to different portions of the interface.

18. The computer-implemented method of claim 8, further comprising:

analyzing the image information to determine whether the user is wearing glasses; and

adjusting the second three-dimensional position to account for variations in a determined eye position of the user resulting from the glasses.

19. The computer-implemented method of claim 8, further comprising:

monitoring adjustments made by the user with respect to the input location; and

adjusting at least one calibration parameter to compensate for the adjustments.

20. The computer-implemented method of claim 8, wherein calculating the input location on the electronic device where the user is attempting to provide the input to the electronic device includes using at least one vector manipulation process to determine the vector passing through the first three-dimensional position and the second three-dimensional position.

21. The computer-implemented method of claim 8, wherein calculating the input location further includes calculating an intersection point where the vector passing through the first three-dimensional position and the second three-dimensional position intersects a plane of the electronic device, wherein the intersection point is located a distance from the electronic device.

22. A computing device, comprising:

a device processor;

at least two image capture elements; and

a memory device including instructions operable to be executed by the processor to perform a set of actions, causing the computing device to;

obtain image information captured using at least two image capture elements of the computing device;

analyze the image information to determine a first three-dimensional position of a determined feature of a user;

analyze the image information to determine a second three-dimensional position corresponding to at least one eye of the user; and

calculate an input location on the computing device where the user is attempting to provide input to the computing device, the input location corresponding to a tilt of the computing device with respect to the user and a vector passing through the first three-dimensional position and the second three-dimensional position.

23. The computing device of claim 22, wherein the instructions when executed further cause the computing device to:

monitor changes in the input location over time; and

provide a selection input to the computing device when the input location corresponds to a selectable element of an interface for at least a minimum period of time corresponding to a selection action.

24. The computing device of claim 22, further comprising:

at least one gesture sensor operable to monitor motions of the determined feature, the computing device being further caused to compare the motions of the determined feature to a set of input gestures and, upon one of the motions matching one of the set of input gestures, provide a corresponding input to the computing device.

25. The computing device of claim 22, further comprising:

at least one illumination element operable to provide illumination when the computing device is caused to obtain the image information using at least one image capture element.

26. The computing device of claim 22, wherein the computing device includes a first pair of imaging elements capturing the image information in an ambient light detection mode and a second pair capturing the image information in an infrared detection mode.

27. A non-transitory computer-readable storage medium storing instructions that, when executed by at least one processor of a computing device, cause the computing device to:

obtain image information captured using at least two image capture elements of the computing device;

analyze the image information to determine a first three-dimensional position of a determined feature of a user;

analyze the image information to determine a second three-dimensional position corresponding to at least one of the user; and

calculate an input location on the computing device where the user is attempting to provide input to the computing device, the input location corresponding to a tilt of the computing device with respect to the user and a vector passing through the first three-dimensional position and the second three-dimensional position.

28. The non-transitory computer-readable storage medium of claim 27, wherein the instructions when executed further cause the computing device to:

monitor changes in the input location over time; and

provide a selection input to the computing device when the input location corresponds to a selectable element of an interface for at least a minimum period of time corresponding to a selection action.

29. The non-transitory computer-readable storage medium of claim 27, wherein the instructions when executed further cause the computing device to:

monitor motions of the determined feature;

compare the motions of the determined feature to a set of input gestures; and

upon one of the motions matching one of the set of input gestures, provide a corresponding input to the at least one processor.

30. The non-transitory computer-readable storage medium of claim 27, wherein the instructions when executed further cause the computing device to:

capture a first portion of the image information in an ambient light detection mode using a first pair of imaging elements and a second portion of the image information in an infrared detection mode using a second pair of imaging elements.