SYSTEMS AND METHODS OF INTERPRETING COMPLEX GESTURES

• US 20150029092A1
• Filed: 07/23/2014
• Published: 01/29/2015
• Est. Priority Date: 07/23/2013
• Status: Abandoned Application

• Current Assignee: Lmi Liquidating Co., LLC
• Original Assignee: Leap Motion Incorporated (Ultraleap Limited)
• Inventors: HOLZ, David, PEREZ, Matias, ROY, Neeloy, Sills, Maxwell
• Application Number:
  US14/339,367
• Publication Number:
  US 20150029092A1
• Time in Patent Office: Days
• US Class Current: 345/156
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883

Abstract

The technology disclosed relates to using a curvilinear gestural path of a control object as a gesture-based input command for a motion-sensing system. In particular, the curvilinear gestural path can be broken down into curve segments, and each curve segment can be mapped to a recorded gesture primitive. Further, certain sequences of gesture primitives can be used to identify the original curvilinear gesture.

Claims

1. A method of interpreting complex gestures, the method including:

capturing a plurality of digital images of a non-linear free-form gesture in a three-dimensional (3D) sensory space performed by a control object;

determining a path of movement of the control object during the non-linear free-form gesture;

segmenting the path into multiple curve segments at least one of vertices, mid-points, and inflection points;

piecewise fitting at least some of the curve segments to second or third order curves;

identifying curve primitives in a library that match the piecewise fitted curve segments;

mapping one or more geometric attributes of the piecewise fitted curve segments to parameters of the curve primitives; and

forwarding the mapped parameters and curve primitives to a further process for interpretation as commands.

2. The method of claim 1, wherein the geometric attributes of the curve segments include at least starting and ending points of the curve segments.

3. The method of claim 1, wherein the geometric attributes of the curve segments include at least degrees of curvature of the curve segments.

4. The method of claim 1, wherein the geometric attributes of the curve segments include at least torsion of the curve segments.

5. The method of claim 1, wherein the geometric attributes of the curve segments include at least gradients of the curve segments.

6. The method of claim 1, wherein the geometric attributes of the curve segments include at least orientation of the curve segments.

7. The method of claim 1, wherein the geometric attributes of the curve segments include at least radius of the curve segments.

8. The method of claim 1, wherein mapping geometric attributes of the curve segments to parameters of the curve segments further includes approximating a best-fit curve for the curve segments.

9. The method of claim 1, further including mapping one or more kinematic attributes of the curve segments to parameters of the curve primitives.

10. The method of claim 9, wherein the kinematic attributes of the curve segments include at least one of speed, velocity, and acceleration of the control object during respective curve segments of the free-form gesture.

11. The method of claim 1, further including anticipating a future motion of the control object based on comparing a sequence of curve primitives mapped to the curve segments to a pre-defined ordering of curve primitives that includes the mapped curve primitives.

12. The method of claim 11, further including determining control manipulations responsive to the free-form gesture by:

representing the control manipulations as unique gesture-tag sequences; and

responsive to identifying a subset of the unique gesture-tag sequences in the sequence of curve primitives mapped to the curve segments, performing the control manipulations represented by the subset.

13. The method of claim 1, further including detecting erroneous interpretation of the free-form gesture by:

representing a first sequence of curve primitives mapped to a first set of curve segments as a first gesture-tag sequence;

based on a gesture template that specifies at least one of temporal sequence and combination of gestural-tags representing occurrences of curve segments in a gestural path, identifying a potential gestural-tag sequence that represents a subsequent sequence of curve primitives to be mapped to a future set of curve segments that most likely follow the first set of curve segments; and

detecting an erroneous fitting of the curve segments when a second gesture-tag sequence representing a second set of curve segments following the first set of curve segments differs from the potential gestural-tag sequence above a maximum threshold.

14. A method of detecting erroneous interpretation of a gesture, further including:

representing a sequence of gesture primitives mapped to gesture segments of a gesture as a gesture-tag sequence, wherein the gesture-tag sequence includes one or more characters;

anticipating a next component of gesture-tags in the gesture-tag sequence based on a model sequence of gesture primitives that identifies future occurrences of one or more subsequent gesture-tags given prior occurrences of one or more previous gesture-tags;

comparing the anticipated component with an actual component of gesture-tags representing next gesture primitives; and

determining an erroneous fitting of the gesture segments responsive to detecting a mismatch between the next component and actual component.

15. The method of claim 14, further including preventing erroneous interpretation of the gesture by not forwarding the mismatched actual component of gesture-tags for interpretation as commands.

16. The method of claim 14, further including preventing erroneous interpretation of the gesture by automatically forwarding the anticipated component of gesture-tags for interpretation as commands instead of the mismatched actual component of gesture-tags.

17. The method of claim 14, further including preventing erroneous interpretation of the gesture by presenting the mismatched actual component of gesture-tags for human rejection or ratification.

18. A system of interpreting complex gestures, the system including:

a processor coupled to memory, the memory including computer instructions that, when executed, cause the processor to;

capture a plurality of digital images of a non-linear free-form gesture in a three-dimensional (3D) sensory space performed by a control object;

determine a path of movement of the control object during the non-linear free-form gesture;

segment the path into multiple curve segments at vertices and inflection points;

piecewise fit at least some of the curve segments to second or third order curves;

identify curve primitives in a library that match the piecewise fitted curve segments;

map one or more geometric attributes of the piecewise fitted curve segments to parameters of the curve primitives; and

forward the mapped parameters and curve primitives to a further process for interpretation as commands.

19. The system of claim 18, further configured to determine control manipulations responsive to the free-form gesture by:

representing the control manipulations as unique gesture-tag sequences; and

responsive to identifying a subset of the unique gesture-tag sequences in a sequence of curve primitives mapped to the curve segments, performing the control manipulations represented by the subset.

20. The system of claim 18, further configured to detect erroneous interpretation of the free-form gesture by:

representing a first sequence of curve primitives mapped to a first set of curve segments as a first gesture-tag sequence;

based on a gesture template that specifies at least one of temporal sequence and combination of gestural-tags representing occurrences of curve segments in a gestural path, identifying a potential gestural-tag sequence that represents a subsequent sequence of curve primitives to be mapped to a future set of curve segments that most likely follow the first set of curve segments; and

detecting an erroneous fitting of the curve segments when a second gesture-tag sequence representing a second set of curve segments following the first set of curve segments differs from the potential gestural-tag sequence above a maximum threshold.