Dynamic user interactions for display control and identifying dominant gestures

• US 9,696,867 B2
• Filed: 01/15/2014
• Issued: 07/04/2017
• Est. Priority Date: 01/15/2013
• Status: Active Grant

• Current Assignee: Lmi Liquidating Co., LLC
• Original Assignee: Leap Motion Incorporated (Ultraleap Limited)
• Inventors: Holz, David
• Primary Examiner(s): Nguyen, Tuan S
• Application Number:
  US14/156,420
• Publication Number:
  US 20140201674A1
• Time in Patent Office: 1,266 Days
• Field of Search: None
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0346 with detection of the device orientation or free movement in a 3D space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors
• CPC Class Codes: G06F 3/04815 Interaction with a metaphor-based environment or interaction object displayed as three-dimensional, e.g. changing the user viewpoint with respect to the environment or object
• CPC Class Codes: G06F 3/04817 using icons graphical or visual programming using iconic symbols G06F8/34
• CPC Class Codes: G06F 3/04847 Interaction techniques to control parameter settings, e.g. interaction with sliders or dials
• CPC Class Codes: G06F 3/0485 Scrolling or panning
• CPC Class Codes: G06V 20/64 Three-dimensional objects
• CPC Class Codes: G06V 40/28 Recognition of hand or arm movements, e.g. recognition of deaf sign language static hand signs G06V40/113

Abstract

The technology disclosed relates to distinguishing meaningful gestures from proximate non-meaningful gestures in a three-dimensional (3D) sensory space. In particular, it relates to calculating spatial trajectories of different gestures and determining a dominant gesture based on magnitudes of the spatial trajectories. The technology disclosed also relates to uniformly responding to gestural inputs from a user irrespective of a position of the user. In particular, it relates to automatically adapting a responsiveness scale between gestures in a physical space and resulting responses in a gestural interface by automatically proportioning on-screen responsiveness to scaled movement distances of gestures in the physical space, user spacing with the 3D sensory space, or virtual object density in the gestural interface. The technology disclosed further relates to detecting if a user has intended to interact with a virtual object based on measuring a degree of completion of gestures and creating interface elements in the 3D space.

Claims

1. A method of distinguishing meaningful gestures from proximate non-meaningful gestures in a three-dimensional (3D) sensory space, the method including:

distinguishing at least one selected from wrist flexing and finger flexing from an overall trajectory of an arm making a gesture;

wherein at least some of the wrist flexing and finger flexing occurs while the arm is in motion, including;

detecting positions of the arm and attached wrist and fingers in a three dimensional (3D) sensory space using an electronic sensor;

calculating from the detected positions a spatial trajectory of a waving gesture executed by the arm;

calculating from the detected positions a spatial trajectory of a flexing gesture made by the wrist and/or fingers; and

determining whether the waving gesture or the flexing gesture is a dominant gesture based on magnitudes of the respective spatial trajectories; and

triggering a response to the dominant gesture without triggering a response to a non-dominant gesture.

2. The method of claim 1, wherein the magnitude of the spatial trajectory of the waving gesture is determined at least in part by a distance traversed in performance of the waving gesture.

3. The method of claim 1, wherein the magnitude of the spatial trajectory of the flexing gesture is determined at least in part by a scale of curling of the fingers.

4. The method of claim 1, further including:

recognizing a first valid gesture corresponding to the dominant gesture and associated with a first response; and

recognizing a second valid gesture corresponding to the non-dominant gesture and associated with a second response.

5. The method of claim 4, wherein the triggering includes:

based upon the determining, triggering the first response associated with the dominant gesture while excluding triggering the second response associated with the non-dominant gesture.

6. A method of distinguishing between two simultaneous gestures originating from a single object in a three-dimensional (3D) sensory space, the method including:

distinguishing finger and wrist flexing from an overall trajectory of an arm that is performing a gesture, the finger and wrist flexing occurring while the arm is in motion, including;

calculating from a series of detected positions of the arm a spatial trajectory of a waving gesture executed by the arm, wherein a magnitude of the spatial trajectory is determined at least in part by a distance traversed by the arm in performance of the waving gesture;

calculating spatial trajectories of flexing gestures made by the wrist and/or fingers, wherein magnitudes of the spatial trajectories are determined at least in part by a scale of curling of the fingers and degrees of freedom between the fingers;

evaluating a magnitude of the spatial trajectory of the waving gesture and the magnitudes of the spatial trajectories of the flexing gestures and determining a dominant gesture based on the magnitudes; and

triggering a response to the overall trajectory based on the dominant gesture.

7. A non-transitory computer readable storage medium storing instructions to distinguish meaningful gestures from proximate non-meaningful gestures in a three-dimensional (3D) sensory space, which instructions are executable by a processor to perform:

distinguishing at least one selected from wrist flexing and finger flexing from an overall trajectory of an arm making a gesture;

wherein at least some of the wrist flexing and finger flexing occurs while the arm is in motion, including;

detecting positions of the arm and attached wrist and fingers in a three dimensional (3D) sensory space using an electronic sensor;

calculating from the detected positions a spatial trajectory of a waving gesture executed by the arm;

calculating from the detected positions a spatial trajectory of a flexing gesture made by the wrist and/or fingers; and

determining whether the waving gesture or the flexing gesture is a dominant gesture based on magnitudes of the respective spatial trajectories; and

triggering a response to the overall trajectory of the arm based on the dominant gesture without triggering a response to a non-dominant gesture.

8. The non-transitory computer readable storage medium of claim 7, wherein the magnitude of the spatial trajectory of the waving gesture is determined at least in part by a distance traversed in performance of the waving gesture.

9. The non-transitory computer readable storage medium of claim 7, wherein the magnitude of the spatial trajectory of the flexing gesture is determined at least in part by a scale of curling of the fingers.

10. The non-transitory computer readable storage medium of claim 7, further including instructions executable by a processor to perform:

recognizing a first valid gesture corresponding to the dominant gesture and associated with a first response; and

recognizing a second valid gesture corresponding to the non-dominant gesture and associated with a second response.

11. The non-transitory computer readable storage medium of claim 10, wherein the triggering includes:

based upon the determining, triggering the first response associated with the dominant gesture while excluding triggering the second response associated with the non-dominant gesture.

12. A system including:

an electronic sensor to capture motion information;

a memory; and

one or more processors operable to execute instructions, stored in the memory, to perform;

distinguishing at least one selected from wrist flexing and finger flexing from an overall trajectory of an arm making a gesture;

wherein at least some of the wrist flexing and finger flexing occurs while the arm is in motion, including;

detecting positions of the arm and attached wrist and fingers in a three dimensional (3D) sensory space using the electronic sensor;

calculating from the detected positions a spatial trajectory of a waving gesture executed by the arm;

calculating from the detected positions a spatial trajectory of a flexing gesture made by the wrist and/or finders; and

determining whether the waving gesture or the flexing gesture is a dominant gesture based on magnitudes of the respective spatial trajectories; and

triggering a response to the overall trajectory of the arm based on the dominant gesture without triggering a response to a non-dominant gesture.

13. The system of claim 12, further including instructions, stored in the memory, to perform:

recognizing a first valid gesture corresponding to the dominant gesture and associated with a first response; and

recognizing a second valid gesture corresponding to the non-dominant gesture and associated with a second response.

14. The system of claim 13, wherein the triggering includes:

based upon the determining, triggering the first response associated with the dominant gesture while excluding triggering the second response associated with the non-dominant gesture.

15. The system of claim 12, wherein the non-dominant gesture is ignored.