Cursor mode switching

• US 10,254,849 B2
• Filed: 03/22/2018
• Issued: 04/09/2019
• Est. Priority Date: 05/17/2013
• Status: Active Grant

• Current Assignee: Lmi Liquidating Co., LLC
• Original Assignee: Leap Motion Incorporated (Ultraleap Limited)
• Inventors: Holz, David S.
• Primary Examiner(s): Osorio, Ricardo
• Application Number:
  US15/933,199
• Publication Number:
  US 20190018497A1
• Time in Patent Office: 383 Days
• CPC Class Codes: B65D 75/5877 connected to a planar surface of the package wall
• CPC Class Codes: G06F 3/005 Input arrangements through a video camera
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0304 Detection arrangements using opto-electronic means constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042
• CPC Class Codes: G06F 3/0346 with detection of the device orientation or free movement in a 3D space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors
• CPC Class Codes: G06V 40/20 Movements or behaviour, e.g. gesture recognition recognition of facial expressions G06V40/16

Abstract

Methods and systems for processing input from an image-capture device for gesture-recognition. The method further includes computationally interpreting user gestures in accordance with a first mode of operation; analyzing the path of movement of an object to determine an intent of a user to change modes of operation; and, upon determining an intent of the user to change modes of operation, subsequently interpreting user gestures in accordance with the second mode of operation.

Claims

1. A system for gesture-recognition, comprising:

an image-capture device for obtaining digital images of an object in 3D space;

a processor-executable analysis module for computationally determining a path of movement of the object based on captured images;

a computer memory for storing a digital representation of the path of movement; and

a processor-executable mode-control module configured for;

interpreting user gestures in accordance with a first mode of operation;

analyzing the path of movement to determine an intent of a user to change modes of operation by;

determining an amount of curvature of the path of movement; and

determining whether a velocity, an acceleration, and a range of motion of movement along the path of movement crosses a threshold to overcome a hysteresis associated with the first mode of operation;

wherein the hysteresis is determined using a physics-based model of the first mode of operation; and

upon determining the intent of the user to change modes of operation, interpreting user gestures in accordance with a second mode of operation.

2. The system of claim 1, wherein determining the amount of curvature includes determining a maximum distance the path of movement is from a straight line between a starting point and ending point of the path of movement.

3. The system of claim 1, wherein the mode-control module is configured to analyze the path of movement by determining a direction of curvature of the path of movement.

4. The system of claim 3, wherein determining the direction of curvature of the path of movement includes determining which side of a virtual reference plane a midpoint of the path of movement is on.

5. The system of claim 1, wherein the mode-control module is configured to determine whether the path of movement intersects a virtual reference plane wherein the virtual reference plane is created based on analyzing a path of movement of one or more prior gestures to determine an intent of a user in making the prior gestures.

6. The system of claim 5, wherein the virtual reference plane is created based on analyzing a path of movement of a prior gesture to determine an intent of a user to create a virtual reference plane in making the prior gesture.

7. The system of claim 1, wherein the mode-control module is configured to analyze a second path of movement to determine the intent of the user to perform a multi-object gesture and change the modes of operation, the mode-control module subsequently interpreting user gestures in accordance with the second mode of operation based on the intent.

8. The system of claim 1, wherein the mode-control module is further configured to determine the intent of the user to change modes based on prior gestures.

9. The system of claim 1, wherein the mode-control module is configured to implement a hysteresis-like effect between modes in which gestures falling below a threshold for small or slow gestures are interpreted as user intent to stay within a current mode and gestures with similarly shaped paths performed with a range of motion or velocity greater than a threshold are interpreted as user intent to switch modes.

10. A method of computationally recognizing and interpreting gestures, the method comprising:

obtaining, using an image-capture device, digital images of a real-world object in 3D space;

computationally determining a path of movement of the real-world object based on captured images;

storing, in a computer memory, a digital representation of the path of movement;

interpreting user gestures in accordance with a first mode of operation;

analyzing the path of movement to determine an intent of a user to change modes of operation by;

determining an amount of curvature of the path of movement; and

determining whether a velocity, an acceleration, and a range of motion of movement along the path of movement crosses a threshold to overcome a hysteresis associated with the first mode of operation;

wherein the hysteresis is determined using a physics-based model of the first mode of operation; and

upon determining the intent of the user to change modes of operation, interpreting user gestures in accordance with a second mode of operation.

11. The method of claim 10, wherein determining the amount of curvature includes determining a maximum distance the path of movement is from a straight line between a starting point and ending point of the path of movement.

12. The method of claim 10, wherein analyzing the path of movement comprises determining a direction of curvature of the path of movement.

13. The method of claim 12, wherein determining the direction of curvature of the path of movement includes determining which side of a virtual reference plane a midpoint of the path of movement is on.

14. The method of claim 10, wherein analyzing the path of movement comprises determining whether the path of movement intersects a virtual reference plane wherein the virtual reference plane is created based on analyzing a path of movement of one or more prior gestures to determine an intent of a user in making the prior gestures.

15. The method of claim 14, wherein the virtual reference plane is created based on analyzing a path of movement of a prior gesture to determine an intent of a user to create a virtual reference plane in making the prior gesture.

16. The method of claim 10, further comprising analyzing a second path of movement to determine the intent of the user to perform a multi-object gesture and changing from the first mode of operation to the second mode of operation based on the intent.

17. The method of claim 10, wherein the intent of the user to change modes is inferred based on prior gestures.

18. The method of claim 10, wherein analyzing the path of movement further comprises implementing a hysteresis-like effect between modes in which gestures falling below a threshold for small or slow gestures are interpreted as user intent to stay within a current mode and gestures with similarly shaped paths performed with a range of motion or velocity greater than a threshold are interpreted as user intent to switch modes.

19. A non-transitory computer readable storage medium impressed with computer program instructions to computationally interpret gestures, which instructions, when executed on a processor, implement a method comprising:

obtaining, using an image-capture device, digital images of a real-world object in 3D space;

computationally determining a path of movement of the real-world object based on captured images;

storing, in a computer memory, a digital representation of the path of movement;

interpreting user gestures in accordance with a first mode of operation;

analyzing the path of movement to determine an intent of a user to change modes of operation by;

determining an amount of curvature of the path of movement; and

determining whether a velocity, an acceleration, and a range of motion of movement along the path of movement crosses a threshold to overcome a hysteresis associated with the first mode of operation;

wherein the hysteresis is determined using a physics-based model of the first mode of operation; and

upon determining the intent of the user to change modes of operation, interpreting user gestures in accordance with a second mode of operation.

20. The non-transitory computer readable storage medium of claim 19, wherein analyzing the path of movement further comprises implement a hysteresis-like effect between modes in which gestures falling below a threshold for small or slow gestures are interpreted as user intent to stay within a current mode and gestures with similarly shaped paths performed with a range of motion or velocity greater than a threshold are interpreted as user intent to switch modes.