INITIALIZING PREDICTIVE INFORMATION FOR FREE SPACE GESTURE CONTROL AND COMMUNICATION

• US 20150153835A1
• Filed: 12/04/2014
• Published: 06/04/2015
• Est. Priority Date: 12/04/2013
• Status: Active Grant

• Current Assignee: Lmi Liquidating Co., LLC
• Original Assignee: Leap Motion Incorporated (Ultraleap Limited)
• Inventors: Perez, Matias, Horowitz, Kevin
• Granted Patent: US 9,645,654 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06V 20/647 by matching two-dimensional images to three-dimensional objects
• CPC Class Codes: G06V 40/28 Recognition of hand or arm movements, e.g. recognition of deaf sign language static hand signs G06V40/113

Abstract

The technology disclosed relates generally to image analysis and, in particular embodiments, to identifying shapes and capturing motions of objects in three-dimensional space. This is accomplished by calculation of numerous span lengths between opposing sides of a control object wherein each control object can consist of a plurality of span modes, each span mode identified by a frequency distribution of a plurality of sample points. The relevant sample points are derived from the pairing of boundary points on the opposing sides of the control object. For each span mode, span width parameters are calculated from at least part of the distribution of the span lengths, using the span width parameters to initialize at least a portion of a model of the control object, and generating predictive information from the initialized model.

Claims

1. A computer implemented method for initializing a model, comprising:

receiving at least one image of a control object observed in a 3D imaging space;

calculating span lengths between opposing sides of the control object;

determining a number of span modes of the control object applicable to a frequency distribution of the span lengths;

for each span mode, calculating one or more span width parameters from at least part of the distribution of the span lengths;

using the span width parameters to initialize at least a portion of a model of the control object; and

generating predictive information from an initialized model.

2. The method of claim 1, further including:

receiving an image of a hand as the control object;

determining that the span modes include at least a finger width span mode and a palm width span mode; and

using the span width parameters for the finger width and palm width span modes to initialize a model of the hand.

3. The method of claim 1, further including:

receiving an image of a hand as the control object;

determining that the span modes include at least a finger width span mode, a palm width span mode, and a wrist width span mode; and

using the span width parameters for the finger width, palm width, and wrist width span modes to initialize a model of the hand and arm.

4. The method of claim 1, further including calculating at least ten span lengths.

5. The method of claim 1, further including calculating between ten and 1000 span lengths.

6. The method of claim 1, further including calculating between 100 and 10,000 span lengths.

7. The method of claim 1, further including calculating between 1,000 and 1,000,000 span lengths.

8. The method of claim 1, further including:

pairing boundary points on the opposing sides of the control object by selecting pairs of points on opposing boundaries that have opposing boundary normals and that can be connected by convex curve, which includes a straight line, without the convex curve crossing any other boundary points of the control object;

wherein each boundary normal is perpendicular to a boundary and pointed away from a center of the control object; and

calculating the span length between paired boundary points as a shortest convex curve between the paired boundary points.

9. The method of claim 1, further including:

pairing boundary points on the opposing sides of the control object by selecting points on opposing boundaries that have opposing boundary normals and that can be connected by geodesic segment without the geodesic segment crossing any other boundary points of the control object;

wherein each boundary normal is perpendicular to a boundary and pointed away from a center of the control object; and

calculating the span length between paired boundary points as a shortest geodesic segment between the paired boundary points.

10. The method of claim 1, further including:

determining from the image of the control object a number of modes of span width distributions present in the image; and

using at least the number of modes present to select the model of the control object used to generate the predictive information.

11. The method of claim 10, further including:

bucketing the span lengths to form a histogram distribution; and

analyzing at least a frequency of span lengths in buckets to determine the number of modes present.

12. The method of claim 10, further including:

selecting a plurality of radial basis functions to approximate distribution of span lengths;

fitting a curve to the radial basis functions; and

analyzing a fitted curve to determine the number of modes present.

13. The method of claim 10, further including:

testing a distribution of the span lengths against alternative expected values of span lengths for a plurality of control object models; and

determining the number of modes present from the testing of the distribution against the control object models.

14. The method of claim 1, further including:

scaling the model of the control object based at least in part on the span width parameters.

15. The method of claim 1, further including:

determining from the image of the control object a number of modes of span width distributions present in the image; and

using the number of modes present to select a software mode of operation, wherein the software mode of operation is selected from a group that includes at least tool movement interpretation, hand gesture interpretation, and facial expression interpretation.

16. The method of claim 1, further including:

using the span width parameters to select a software mode of operation, wherein the software mode of operation is selected from a group that includes at least tool movement interpretation, hand gesture interpretation, and facial expression interpretation.

17. The method of claim 1, further including:

representing portions of the model as capsoloids; and

generating predictive information including detection of collisions between two or more of the capsoloids.

18. The method of claim 1, further including:

representing portions of the model as capsoloids; and

generating predictive information including detection of non-rigid translations of portions of the model.

19. The method of claim 1, wherein determining the frequency distribution of span lengths comprises:

fitting one or more radial basis functions to observed span lengths;

applying a function to the one or more radial basis functions to form a result; and

providing the result comprising frequency of occurrence of span lengths.

20. The method of claim 19, wherein fitting one or more radial basis functions to the observed span length comprises:

selecting for one or more observed span length values a radial basis function; and

determining at least one property of the radial basis function selected.

21. The method of claim 20, wherein determining at least one property of the radial basis function selected comprises:

determining a variance of a dataset including the span lengths;

dividing the variance by an expectation value determined from an expected number of modes in the dataset determined from observed information; and

providing the variance divided by the expectation value as a width of the radial basis function.

22. The method of claim 19, wherein applying a function to the one or more radial basis functions comprises:

applying a summation function to sum the radial basis functions thereby producing a frequency of occurrence for the span lengths.

23. The method of claim 1, wherein selecting span lengths as initialization parameters based at least in part upon a frequency of occurrence comprises:

determining one or more modes from the frequency of occurrence of observed span lengths;

selecting a mode of interest; and

determining the initialization parameter from the mode of interest.

24. The method of claim 1, wherein applying the initialization parameter to at least a portion of a model within the predictive information;

thereby initializing the model portion comprises;

scaling span lengths computed for the model portion by the initialization parameter.

25. A computer implemented system for initializing a model, comprising:

at least one camera that captures at least one image of a control object observed in a 3D imaging space;

a processor coupled to the camera, memory coupled to the processor and computer instructions in the memory that implement a method including;

calculating numerous span lengths between opposing sides of the control object;

determining a number of span modes of the control object applicable to a frequency distribution of the span lengths;

for each span mode, calculating one or more span width parameters from at least part of the distribution of the span lengths;

using the span width parameters to initialize at least a portion of a model of the control object; and

generating predictive information from the initialized model.

26. The system of claim 25, wherein an image analysis module pairs boundary points on the opposing sides of the control object by selecting pairs of points on opposing boundaries that have opposing boundary normals and that can be connected by convex curve, which includes a straight line, without the convex curve crossing any other boundary points of the control object;

wherein each boundary normal is perpendicular to a boundary and pointed away from a center of the control object; and

calculating the span length between paired boundary points as a shortest convex curve between the paired boundary points.

27. The system of claim 25, wherein the image analysis module determines from the image of the control object a number of modes of span width distributions present in the image;

and

using at least the number of modes present to select the model of the control object used to generate the predictive information.

28. The system of claim 25, wherein the image analysis module determines the frequency distribution of span lengths comprising:

fitting one or more radial basis functions to observed span lengths;

applying a function to the one or more radial basis functions to form a result; and

providing the result comprising the frequency of occurrence of span lengths.