NON-TACTILE INTERFACE SYSTEMS AND METHODS

• US 20140320408A1
• Filed: 04/25/2014
• Published: 10/30/2014
• Est. Priority Date: 04/26/2013
• Status: Active Grant

• Current Assignee: Lmi Liquidating Co., LLC
• Original Assignee: Leap Motion Incorporated (Ultraleap Limited)
• Inventors: ZAGORSEK, Michael, DABIR, Avinash, DURDIK, Paul, MERTENS, Keith
• Granted Patent: US 9,916,009 B2
• Time in Patent Office: Days
• US Class Current: 345/158
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06V 40/11 Hand-related biometrics; Hand pose recognition
• CPC Class Codes: G06V 40/28 Recognition of hand or arm movements, e.g. recognition of deaf sign language static hand signs G06V40/113

Abstract

Methods and systems for processing an input are disclosed that detect a portion of a hand and/or other detectable object in a region of space monitored by a 3D sensor. The method further includes determining a zone corresponding to the region of space in which the portion of the hand or other detectable object was detected. Also, the method can include determining from the zone a correct way to interpret inputs made by a position, shape or a motion of the portion of the hand or other detectable object.

Claims

1. A machine-implemented method for processing an input, the method including:

detecting a portion of a hand or other detectable object in a 3D sensor space;

determining a zone from among multiple zones within the of 3D sensor space in which the portion of the hand or other detectable object was detected; and

determining from the zone a correct way to interpret inputs detected by the 3D sensor as a position or a motion of the portion of the hand or other detectable object.

2. The method of claim 1, wherein detecting the portion of a hand or other detectable object in the 3D sensor space includes detecting at least one of a brush, a pen, a pencil, a stylus, a paintbrush, or an eraser.

3. The method of claim 1, wherein determining the zone in which the portion of the hand or other detectable object was detected includes:

capturing at least one image using an imaging analysis system;

analyzing the at least one captured image to detect at least one edge of the object based on changes in at least one parameter of the at least one image; and

determining a position and/or motion associated with the object based upon the at least one edge.

4. The method of claim 3, wherein the analyzing the parameter includes analyzing the at least one captured image for a change in pixel brightness.

5. The method of claim 4, wherein the analyzing step further includes:

comparing brightness of at least two pixels to a threshold; and

detecting a transition in brightness from below the threshold to above the threshold across adjacent pixels.

6. The method of claim 4, wherein the analyzing step includes:

comparing successive images captured without and with illumination by one or more light source; and

detecting a transition in brightness from a low level to a high level across successive images of individual pixels.

7. The method of claim 1, wherein determining a zone in which the portion of the hand or other detectable object was detected includes:

selecting one of multiple zones to test for presence of the object;

determining whether the object is within the selected zone; and

adding the zone to a set of zones in which the object can be found when the object is determined to be within the selected zone.

8. The method of claim 7, wherein determining whether the object is within the selected zone includes determining whether a position of the object is within boundaries of the zone.

9. The method of claim 7, wherein the object can be found in more than one zone in the set of zones, further including:

determining from the set of zones in which the object can be found, a first preferred zone;

assigning the object to the first preferred zone; and

reporting the first preferred zone for further processing.

10. The method of claim 9, wherein determining a first preferred zone includes:

applying a zone hierarchy to the set of zones to determine therefrom a zone highest on the hierarchy; and

reporting the zone highest on the hierarchy as the first preferred zone for further processing.

11. The method of claim 9, wherein determining from the set of zones in which the object can be found, a first preferred zone includes:

applying a set of one or more rules to the set of zones to determine therefrom a zone to provide as the first preferred zone according to the one or more rules; and

reporting the first preferred zone for further processing.

12. The method of claim 1, wherein determining from the zone a correct way to interpret inputs detected by the 3D sensor includes:

determining an active program and that the zone is a command input zone for the active program; and

interpreting a position or a motion of the object based taking into account the active program and the command input zone.

13. The method of claim 1, wherein determining from the zone a correct way to interpret inputs detected by the 3D sensor includes:

determining an active program and that the zone is a content input zone for the active program; and

interpreting a position or a motion of the object based taking into account the active program and the content input zone.

14. The method of claim 1, wherein determining from the zone a correct way to interpret inputs detected by the 3D sensor includes:

determining an active program and that the zone is a modifier input zone for the active program; and

interpreting a position or a motion of the object based taking into account the active program and the modifier input zone.

15. The method of claim 1, wherein determining from the zone a correct way to interpret inputs detected by the 3D sensor includes:

determining an active program and that the zone is a hover zone for the active program; and

interpreting a position or a motion of the object based taking into account the active program and the hover zone.

16. The method of claim 1, further including:

interpreting motion of the hand or other detectable object as one or more gestures; and

determining from the one or more gestures an input to at least one of (i) an application or (ii) an operating system.

17. The method of claim 1, wherein the multiple zones are at least one of contiguous and/or dis-contiguous.

18. The method of claim 1, further including:

responsive to a determining of a particular zone in which the portion of the hand or other detectable object, determining a correct way to interpret inputs detected by the 3D sensor; and

applying the correct way to interpret inputs detected by the 3D sensor to both the particular zone and other zones in the multiple zones with in the 3D sensor space.

19. A machine-implemented method to process an input, including:

detecting a portion of a hand or other detectable object in a 3D sensor space;

determining a zone corresponding to the 3D sensor space in which the portion of the hand or other detectable object was detected, by;

capturing an image using an imaging analysis system;

analyzing the captured image to detect one or more edges of the object based on changes in at least one image parameter including brightness, by;

comparing brightness of at least two pixels to a threshold; and

detecting a transition in brightness from a low level to a high level across adjacent pixels; and

determining position and/or motion of the object based upon the one or more edges, by

selecting a zone to test for presence of the object;

determining whether the object is within the selected zone; and

adding the zone to a set of zones in which the object can be found when the object is determined to be within the selected zone; and

determining from the zone a correct way to interpret inputs made by a position or a motion of the portion of the hand or other detectable object, by;

interpreting a position or a motion as a command input to an active program when determined to be corresponding to a command input zone; and

interpreting a position or a motion as a content input to an active program when determined to be corresponding to a content input zone; and

interpreting a position or a motion as a modifier input modifying a concurrent input to an active program when determined to be corresponding to a modifier input zone; and

interpreting a position or a motion as being ready to make an input to an active program when determined to be corresponding to a hover zone.

20. A system, including:

an image-capture device including at least one camera; and

an image analyzer coupled to the camera that;

detects a portion of a hand or other detectable object in a 3D sensor space;

determines a zone corresponding to the 3D sensor space in which the portion of the hand or other detectable object was detected; and

determines from the zone a correct way to interpret inputs made by a position or a motion of the portion of the hand or other detectable object.