SYSTEMS AND METHODS OF DIRECT POINTING DETECTION FOR INTERACTION WITH A DIGITAL DEVICE

• US 20160179205A1
• Filed: 06/27/2014
• Published: 06/23/2016
• Est. Priority Date: 06/27/2013
• Status: Active Grant

• Current Assignee: eyeSight Mobile Technologies Ltd.
• Original Assignee: eyeSight Mobile Technologies Ltd.
• Inventors: KATZ, Itay
• Granted Patent: US 9,846,486 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 3/013 Eye tracking input arrangements G06F3/015 takes precedence
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0304 Detection arrangements using opto-electronic means constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042
• CPC Class Codes: H04N 13/383 for tracking with gaze detection, i.e. detecting the lines of sight of the viewer's eyes

Abstract

Systems, methods, and non-transitory computer-readable media are disclosed. For example, a touch-free gesture recognition system is disclosed that includes at least one processor. The processor may be configured to enable presentation of first display information to a user to prompt a first touch-free gesture at at least a first location on a display. The processor may also be configured to receive first gesture information from at least one image sensor corresponding to a first gesturing location on the display correlated to a first touch-free gesture by the user, wherein the first gesturing location differs from a location of the first display information at least in part as a result of one eye of the user being dominant over another eye of the user. In addition, the processor may be configured to determine a first offset associated with the location of the first display information and the first gesturing location. Further, the processor may be configured to enable presentation of second information to prompt the user to make a subsequent touch-free gesture at at least a second location on the display. Additionally, the processor may be configured to receive subsequent gesture information from at least one image sensor corresponding to a subsequent touch-free gesture by the user. Also, the processor may be configured to use the first offset to determine a location on the display affected by the subsequent touch-free gesture.

Claims

1. A touch-free gesture recognition system, comprising:

at least one processor, configured to;

enable presentation of first display information to a user to prompt a first touch-free gesture at at least a first location on a display;

receive first gesture information from at least one image sensor corresponding to a first gesturing location on the display correlated to a first touch-free gesture by the user, wherein the first gesturing location differs from a location of the first display information at least in part as a result of one eye of the user being dominant over another eye of the user;

determine a first offset associated with the location of the first display information and the first gesturing location;

enable presentation of second information to prompt the user to make a subsequent touch-free gesture at at least a second location on the display;

receive subsequent gesture information from the at least one image sensor corresponding to a subsequent touch-free gesture by the user; and

use the first offset to determine a location on the display affected by the subsequent touch-free gesture.

2. The system of claim 1, wherein the first touch-free gesture and the subsequent touch-free gesture include finger pointing and wherein the at least one processor is further configured to use the first gesture information and the subsequent gesture information from the at least one image sensor to determine pointing locations on the display.

3. The system of claim 1, wherein the at least one processor is further configured to send an output to a destination reflective of a recognized gesture associated with the subsequent touch-free gesture.

4. The system of claim 1, wherein the subsequent gesturing location differs from the location of the second information on the display at least in part as the result of one eye of the user being dominant over another eye of the user.

5. The system of claim 1, wherein the at least one processor is further configured to determine the first offset by identifying at least one of a first candidate pointing area generated using at least one of a first ray associated with a first eye of the user and a second candidate pointing area generated using a second ray associated with a second eye of the user.

6. The system of claim 1, wherein the information associated with the first offset corresponds to a location on a face, other than a pupil, to serve as a starting point for a line of sight calculation, and wherein the at least one processor is further configured to use the non-pupil location on the face to determine an Intended location of the subsequent touch-free gesture.

7. The system of claim 1, wherein the at least one processor is configured to use the information associated with the first offset to determine an intended location on the display of the subsequent touch-free gesture.

8. The system of claim 1, wherein the at least one processor is further configured to determine the first gesturing location by selecting at least one of

a first point within a first region on the display associated with a first line of sight passing from a location associated with the first eye, through a fingertip, and onto the display, and

a second point within a second region on the display associated with a second line of sight passing from a second location associated with the second eye, through the fingertip, and onto the display.

9. The system of claim 8, wherein the at least one processor is further configured to determine the first gesturing location by determining at least a third point in the general vicinity of the first point and the second point.

10. The system of claim 1, wherein the at least one processor is further configured to determine differing offsets associated with the eye dominance for differing regions of the display.

11. The system of claim 1, wherein the at least one processor is further configured to determine a location of pointing by using multiple candidate pointing area regions at a plurality of differing time intervals.

12. The system of claim 1, wherein the at least one processor is further configured to store unique information associated with the first offset for each of a plurality of users.

13. The system of claim 12, wherein the at least one processor is further configured to store, as the stored unique information, information received from the at least one image sensor associated with an identity for each of the plurality of users.

14. The system of claim 12, wherein the at least one processor is further configured to store, as the stored unique information, information related to a non-pupil location on a face for each of the plurality of users.

15. The system of claim 1, wherein the information associated with the first offset includes an indication that the one eye of the user is dominant over the other eye of the user.

16. The system of claim 1, wherein the information associated with the first offset includes an indication of a degree to which the one eye of the user is dominant over the other eye of the user.

17. A non-transitory computer-readable medium comprising instructions that, when executed by at least one processor, cause the at least one processor to perform operations including:

enabling presentation of first display information to a user to prompt a first touch-free gesture at at least a first location on a display;

receiving first gesture information from at least one image sensor corresponding to a first gesturing location on the display correlated to a first touch-free gesture by the user, wherein the first gesturing location differs from a location of the first display information at least in part as a result of one eye of the user being dominant over another eye of the user;

determining a first offset associated with the location of the first display information and the first gesturing location;

enabling presentation of second information to prompt the user to make a subsequent touch-free gesture at at least a second location on the display;

receive subsequent gesture information from the at least one image sensor corresponding to a subsequent touch-free gesture by the user; and

using the first offset to determine location on the display affected by the subsequent touch-free gesture.

18. The non-transitory computer-readable medium of claim 17, wherein the subsequent gesturing location differs from the location of the second information on the display at least in part as the result of information indicative of one eye of the user being dominant over another eye of the user.

19. The non-transitory computer-readable medium of claim 17, wherein the instructions, when executed by the at least one processor, further causes the at least one processor to perform an additional operation including determining the first offset by identifying at least one of a first candidate pointing area generated using at least one of a first ray associated with a first eye of the user and a second candidate pointing area generated using a second ray associated with a second eye of the user.

20. The non-transitory computer-readable medium of claim 17, wherein the information associated with the first offset corresponds to a location on a face, other than a pupil, to serve as a starting point for a line of sight calculation, and wherein the instructions, when executed by the at least one processor, further causes the at least one processor to perform an additional operation including using the non-pupil location on the face to determine an intended location of the subsequent touch-free gesture.