Systems and methods for triggering actions based on touch-free gesture detection

• US 9,274,608 B2
• Filed: 11/13/2013
• Issued: 03/01/2016
• Est. Priority Date: 12/13/2012
• Status: Expired due to Fees

• Current Assignee: eyeSight Mobile Technologies Ltd.
• Original Assignee: eyeSight Mobile Technologies Ltd.
• Inventors: Katz, Itay, Affias, Ofer
• Primary Examiner(s): Sheng, Tom
• Application Number:
  US14/078,636
• Publication Number:
  US 20140168062A1
• Time in Patent Office: 839 Days
• Field of Search: 354/156, 354/157, 715/863
• US Class Current: 1/1
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0304 Detection arrangements using opto-electronic means constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042
• CPC Class Codes: G06F 3/0482 Interaction with lists of selectable items, e.g. menus
• CPC Class Codes: G06F 3/0484 for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range
• CPC Class Codes: G06F 3/167 Audio in a user interface, e.g. using voice commands for navigating, audio feedback

Abstract

Systems, methods and non-transitory computer-readable media for triggering actions based on touch-free gesture detection are disclosed. The disclosed systems may include at least one processor. A processor may be configured to receive image information from an image sensor, detect in the image information a gesture performed by a user, detect a location of the gesture in the image information, access information associated with at least one control boundary, the control boundary relating to a physical dimension of a device in a field of view of the user, or a physical dimension of a body of the user as perceived by the image sensor, and cause an action associated with the detected gesture, the detected gesture location, and a relationship between the detected gesture location and the control boundary.

Claims

1. A touch-free gesture recognition system, comprising:

at least one processor configured to;

receive image information from an image sensor;

determine one or more locations in the image information relating to at least one control boundary based, at least in part, on a physical dimension of a device in a field of view of a user;

detect in the image information a gesture performed by the user;

detect a gesture location in the image information, wherein at least a portion of the gesture location is detected outside the control boundary, and wherein the gesture location is associated with at least one control boundary crossed by the detected gesture; and

cause an action associated with the detected gesture, the detected gesture location, and a relationship between the detected gesture location and the control boundary.

2. The system of claim 1, wherein the processor is further configured to determine the control boundary based, at least in part, on a dimension of the device as is expected to be perceived by the user.

3. The system of claim 1, wherein the control boundary is determined based, at least in part, on at least one of an edge or corner of the device as is expected to be perceived by the user.

4. The system of claim 1, wherein the processor is further configured to distinguish between a plurality of predefined gestures to cause a plurality of actions, each associated with a differing predefined gesture.

5. The system of claim 1, wherein the processor is further configured to generate a plurality of actions, each associated with a differing relative position of the gesture location to the control boundary.

6. The system of claim 1, wherein the processor is further configured to determine the control boundary by detecting a portion of a body of the user, other than the user'"'"'s hand, and to define the control boundary based on the detected body portion, and wherein the processor is further configured to generate the action based, at least in part, on an identity of the gesture, and a relative location of the gesture to the control boundary.

7. The system of claim 1, wherein the processor is further configured to determine the control boundary based on a contour of at least a portion of a body of the user in the image information.

8. The system of claim 1, wherein the device includes a display, and wherein the processor is further configured to determine the control boundary based on dimensions of the display.

9. The system of claim 8, wherein processor is further configured to determine the control boundary based on at least one of an edge or corner of a display associated with the device.

10. The system of claim 8, wherein the processor is further configured to activate a toolbar associated with a particular edge based, at least in part, on the gesture location.

11. The system of claim 1, wherein the action is related to a number of times at least one of an edge or corner of the control boundary is crossed by a path of the gesture.

12. The system of claim 1, wherein the action is associated with a predefined motion path associated with the gesture location and the control boundary.

13. The system of claim 1, wherein the action is associated with a predefined motion path associated with particular edges or corners crossed by the gesture location.

14. The system of claim 1, wherein the processor is further configured to detect a hand in a predefined location relating to the control boundary and initiate detection of the gesture based on the detection of the hand at the predefined location.

15. The system of claim 1, wherein the processor is further configured to cause at least one of a visual or audio indication when the control boundary is crossed.

16. The system of claim 1, wherein the control boundary is determined, at least in part, based on a distance between the user and the image sensor.

17. The system of claim 1, wherein the control boundary is determined, at least in part, based on a location of the user in relation to the device.

18. The system of claim 1, wherein prior to the gesture detection, the at least one processor is configured to provide an indication that a gesture associated with the at least one control boundary is enabled.

19. The system of claim 1, wherein the processor is further configured to distinguish between a plurality of predefined motion paths associated with the same gesture to cause a plurality of actions, each action associated with a different motion path.

20. The system of claim 1, wherein the detected gesture comprises an identified predefined hand pose and an identified predefined motion path.

21. A method for a touch-free gesture recognition system, comprising:

receiving image information from an image sensor;

determining one or more locations in the image information relating to at least one control boundary based, at least in part, on a physical dimension of a device in a field of view of a user;

detecting in the image information a gesture performed by the user;

detecting a gesture location in the image information, wherein at least a portion of the gesture location is detected outside the control boundary, and wherein the gesture location is associated with at least one control boundary crossed by the detected gesture; and

causing an action associated with the detected gesture, the detected gesture location, and a relationship between the detected gesture location and the control boundary.

22. The method of claim 21, wherein the physical dimension of the device in the field of view of the user is a dimension as is expected to be perceived by the user.

23. The method of claim 21, wherein the control boundary is determined based, at least in part, on at least one of an edge or corner of the device as is expected to be perceived by the user.

24. The method of claim 21, further comprising generating a plurality of actions, each associated with a differing relative position of the gesture location to the control boundary.

25. The method of claim 21, further comprising determining the control boundary by detecting a portion of a body of the user, other than the user'"'"'s hand, and defining the control boundary based on the detected body portion, and generating the action based, at least in part, on an identity of the gesture, and a relative location of the gesture to the control boundary.

26. The method of claim 21, further comprising determining the control boundary based on dimensions of the display.

27. The method of claim 26, further comprising activating a toolbar associated with a particular edge based, at least in part, on the gesture location.

28. The method of claim 21, wherein the control boundary is determined based on at least one of an edge or a corner of the device.

29. The method of claim 21, wherein the action is associated with a predefined motion path associated with the gesture location and the control boundary.

30. The method of claim 21, wherein the action is associated with a predefined motion path associated with particular edges or corners crossed by the gesture location.

31. The method of claim 21, further comprising detecting a hand in predefined location relating to the control boundary and initiating detection of the gesture based on the detection of the hand at the predefined location.

32. The method of claim 21, wherein the control boundary is determined, at least in part, based on a distance between the user and the image sensor.

33. The method of claim 21, further comprising providing, prior to the gesture detection, an indication that a gesture associated with the at least one control boundary is enabled.

34. The method of claim 21, further comprising distinguishing between a plurality of predefined motion paths associated with the same gesture to cause a plurality of actions, each action associated with a different motion path.

35. The method of claim 21, wherein detecting the gesture comprises identifying a predefined hand pose and a predefined motion path.

36. A touch-free gesture recognition system, comprising:

at least one processor configured to;

receive image information from an image sensor;

determine at least one location in the image information associated with a control boundary based, at least in part, on a physical dimension of a device in a field of view of a user;

detect in the image information a gesture performed by the user, wherein at least a portion of the gesture is detected outside the control boundary;

determine whether the detected gesture crosses the at least one location associated with the control boundary;

cause a first action when the detected gesture crosses the at least one location associated with the control boundary; and

cause a second action when the detected gesture does not cross the at least one location associated with the control boundary.

37. The system of claim 36, wherein the detected gesture comprises an identified predefined hand pose and an identified predefined motion path.