NON-LINEAR MOTION CAPTURE USING FRENET-SERRET FRAMES

• US 20150022447A1
• Filed: 07/22/2014
• Published: 01/22/2015
• Est. Priority Date: 07/22/2013
• Status: Active Grant

• Current Assignee: Lmi Liquidating Co., LLC
• Original Assignee: Leap Motion Incorporated (Ultraleap Limited)
• Inventors: MERTENS, Keith, HOLZ, David, HARE, Gabriel A., PEREZ, Matias, ROY, Neeloy
• Granted Patent: US 9,857,876 B2
• Time in Patent Office: Days
• US Class Current: 345/158
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0304 Detection arrangements using opto-electronic means constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042

Abstract

Implementations of the technology disclosed convert captured motion from Cartesian/(x,y,z) space to Frenet-Serret frame space, apply one or more filters to the motion in Frenet-Serret space, and output data (for display or control) in a desired coordinate space—e.g., in a Cartesian/(x,y,z) reference frame. The output data can better represent a user'"'"'s actual motion or intended motion.

Claims

1. A method of accurately capturing non-linear gestural path of a control object in three-dimensional (3D) sensory space, the method including:

capturing a non-linear free-form gesture in a three-dimensional (3D) sensory space performed by a control object;

determining a set of rectangular coordinates of a plurality of 3D positions of control object'"'"'s movement path during the non-linear free-form gesture;

transforming the set of rectangular coordinates to a set of curvilinear coordinates;

applying at least one filter to the set of curvilinear coordinates to obtain filtered curvilinear motion data for the control object'"'"'s movement path; and

generating data for a smoothened representation of the control object'"'"'s movement path based on the filtered curvilinear motion data.

2. The method of claim 1, wherein the curvilinear coordinates are Frenet-Serret coordinates including orthonormal vectors.

3. The method of claim 2, further including using the Frenet-Serret coordinates to determine orientation invariant trajectory of the control object along the control object'"'"'s movement path during the non-linear free-form gesture.

4. The method of claim 1, wherein transforming the set of rectangular coordinates to the set of curvilinear coordinates further includes generating a Jacobian of a transformation of the rectangular coordinates.

5. The method of claim 1, wherein transforming the set of rectangular coordinates to the set of curvilinear coordinates includes generating an inverse Jacobian of a Transformation of the rectangular coordinates.

6. The method of claim 1, wherein transforming the set of rectangular coordinates to the set of curvilinear coordinates further includes generating non-curvilinear orthonormal vectors from a helix defined in the rectangular coordinates.

7. The method of claim 1, further including applying at least a moving average filter to the set of curvilinear coordinates to obtain filtered curvilinear motion data for the control object'"'"'s movement path.

8. The method of claim 1, further including applying at least a weight average filter to the set of curvilinear coordinates to obtain filtered curvilinear motion data for the control object'"'"'s movement path.

9. The method of claim 1, further including transforming the filtered curvilinear motion data into a set of Cartesian coordinates and generating data for a smoothened representation of the control object'"'"'s movement path based on the transformed set of Cartesian coordinates.

10. A method of representing an object'"'"'s non-linear path of movement in a monitored space, the method including:

receiving a sequence of images of an object'"'"'s non-linear motion path through a monitored space;

using the sequence of images to determine a set of spatial coordinates corresponding to a three-dimensional (3D) location of the object;

automatically converting the set of coordinates to a set of values corresponding to a Frenet-Serret frame including a tangent unit vector, a normal unit vector, and a binormal unit vector;

automatically applying a filter to the set of values to obtain a set of filtered Frenet-Serret values; and

generating data for a smoothened representation of the object'"'"'s non-linear motion path based on the filtered Frenet-Serret values.

11. The method of claim 10, further including using the Frenet-Serret values to determine orientation invariant trajectory of the object'"'"'s non-linear motion path through a monitored space.

12. The method of claim 10, wherein the set of filtered values further includes a curvature parameter.

13. The method of claim 10, wherein the set of filtered values further includes a torsion parameter.

14. The method of claim 10, wherein the filter is a moving-average filter.

15. The method of claim 10, wherein the filter is a weight-average filter.

16. The method of claim 10, wherein generating data for a smoothened representation of the object'"'"'s motion path further includes approximating a best-fit curve that predicts a trajectory of the object and connects points along the object'"'"'s non-linear motion path.

17. The method of claim 10, wherein generating data for a smoothened representation of the object'"'"'s motion path further includes constructing an Euler spiral to model boundary segments between extrema of the object'"'"'s non-linear motion path.

18. The method of claim 10, further including converting the set of filtered Frenet-Serret values into a set of Cartesian coordinates and generating data for a smoothened representation of the object'"'"'s non-linear motion path based on the converted set of Cartesian coordinates.

19. A system of accurately capturing non-linear gestural path of a control object in three-dimensional (3D) sensory space, the system including:

at least one camera to capture a non-linear free-form gesture in a three-dimensional (3D) sensory space performed by a control object;

a processor;

a processor-executable object-detection model to determine a set of rectangular coordinates of a plurality of 3D positions of control object'"'"'s movement path during the non-linear free-form gesture;

a processor-executable filtering module to;

transform the set of rectangular coordinates to a set of curvilinear coordinates; and

apply at least one filter to the set of curvilinear coordinates to obtain filtered curvilinear motion data for the control object'"'"'s movement path; and

generate for display a smoothened representation of the control object'"'"'s movement path based on the filtered curvilinear motion data.

20. The system of claim 19, wherein the curvilinear coordinates are Frenet-Serret coordinates including orthonormal vectors.

21. The system of claim 19, further configured to use the Frenet-Serret coordinates to determine orientation invariant trajectory of the control object along the control object'"'"'s movement path during the non-linear free-form gesture.

22. The system of claim 19, wherein transforming the set of rectangular coordinates to the set of curvilinear coordinates further includes generating a Jacobian of a Transformation of the rectangular coordinates.

23. The system of claim 19, wherein transforming the set of rectangular coordinates to the set of curvilinear coordinates includes generating an inverse Jacobian of a Transformation of the rectangular coordinates.

24. The system of claim 19, wherein transforming the set of rectangular coordinates to the set of curvilinear coordinates further includes generating non-curvilinear orthonormal vectors from a helix defined in the rectangular coordinates.

25. The system of claim 19, further configured to apply at least a moving average filter to the set of curvilinear coordinates to obtain filtered curvilinear motion data for the control object'"'"'s movement path.

26. The system of claim 19, further configured to apply at least a moving average filter to the set of curvilinear coordinates to obtain filtered curvilinear motion data for the control object'"'"'s movement path.

27. The system of claim 19, further configured to transform the filtered curvilinear motion data into a set of Cartesian coordinates and generating data for a smoothened representation of the control object'"'"'s movement path based on the transformed set of Cartesian coordinates.