Initializing orientation in space for predictive information for free space gesture control and communication

• US 9,659,403 B1
• Filed: 01/06/2015
• Issued: 05/23/2017
• Est. Priority Date: 01/06/2014
• Status: Active Grant

• Current Assignee: Lmi Liquidating Co., LLC
• Original Assignee: Leap Motion Incorporated (Ultraleap Limited)
• Inventors: Horowitz, Kevin
• Primary Examiner(s): Werner, Brian P
• Application Number:
  US14/590,983
• Time in Patent Office: 868 Days
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0304 Detection arrangements using opto-electronic means constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042
• CPC Class Codes: G06T 2207/30196 Human being; Person
• CPC Class Codes: G06T 7/12 Edge-based segmentation
• CPC Class Codes: G06T 7/70 Determining position or orientation of objects or cameras camera calibration G06T7/80
• CPC Class Codes: G06V 40/28 Recognition of hand or arm movements, e.g. recognition of deaf sign language static hand signs G06V40/113

Abstract

The technology disclosed relates to initializing orientation of a three-dimensional (3D) model of an object. In particular, it relates to accessing at least one three-dimensional (3D) model of an object and observed information of the object movable in space and determining a primary orientation parameter of the model from the observed information. The method further includes detecting contours of the object in the observed information and calculating a representative normal to the detected contours, accessing a vector representing a 3D angle from the object to a point of observation, calculating a primary orientation of the object as a cross-product of the representative normal and the vector, and using the calculated primary orientation parameter to initialize the model.

Claims

1. A computer implemented method of initializing an orientation of a three-dimensional (3D) model of an object, comprising:

accessing at least one three-dimensional (3D) model of the object and observed information of the object moving in space; and

determining a primary orientation parameter for the 3D model from the observed information, including;

detecting a plurality of contours for the object in the observed information and calculating a representative normal vector for at least some of the detected contours;

accessing a tangent vector representing a 3D angle tangent to a point on a surface of the object and drawn to a point of observation;

calculating a primary orientation parameter for the object using a cross-product of the representative normal vector and the tangent vector; and

using the calculated primary orientation parameter to initialize the orientation of the 3D model of the object.

2. The method of claim 1, further including filtering the plurality of contours to detect edges and calculating the representative normal vector from the filtered contours.

3. The method of claim 1, further including using brightness gradients to calculate the representative normal vector.

4. The method of claim 1, further including using brightness contours to calculate the representative normal vector.

5. The method of claim 1, wherein the calculating the representative normal vector includes averaging a plurality of normal vectors.

6. The method of claim 5, further including preparing a histogram of the plurality of normal vectors;

calculating an argument value for the histogram; and

using the argument value calculating the representative normal vector.

7. The method of claim 1, further including:

identifying a plurality of normal vectors normal to at least some of the contours of the object and determining for a selected contour, a representative normal vector from the plurality of normal vectors associated with the selected contour based upon a magnitude computed for the plurality of normal vectors; and

selecting from among the plurality of normal vectors for the selected contour a normal vector having a maximum magnitude as the representative normal vector for the selected contour.

8. The method of claim 1, further including:

identifying a plurality of normal vectors normal to at least some of the contours of the object; and

selecting, as the representative normal vector, a normal vector selected from the plurality of normal vectors, the selected representative normal vector having a magnitude determined from magnitudes of the plurality of normal vectors and a direction determined from the plurality of normal vectors.

9. The method of claim 8, the determining further includes determining an average of the magnitudes and the directions of the plurality of normal vectors.

10. The method of claim 8, the determining further includes determining a mean of the magnitudes and the directions of the plurality of normal vectors.

11. The method of claim 1, further including initializing a model portion by applying the primary orientation parameter to at least a portion of the model and aligning the portion to an observed orientation of the object.

12. The method of claim 1, wherein the object is a human body.

13. The method of claim 1, wherein the object is a hand.

14. The method of claim 1, wherein the object is a tool.

15. The method of claim 1, further including modifying power consumption profiles of a machine sensory control system responsive to initialization of the model.

16. A system of initializing an orientation of a three-dimensional (3D) model of an object, comprising:

a processor and a computer readable storage medium storing computer instructions configured to cause the processor to;

access at least one three-dimensional (3D) model of the object and observed information of the object moving in space; and

determine a primary orientation parameter for the 3D model from the observed information, including;

detecting a plurality of contours for the object in the observed information and calculating a representative normal vector for at least some of the detected contours;

accessing a tangent vector representing a 3D angle tangent to a point on a surface of the object and drawn to a point of observation;

calculating a primary orientation parameter for the object using a cross-product of the representative normal vector and the tangent vector; and

using the calculated primary orientation parameter to initialize the orientation of the 3D model of the object.

17. The system of claim 16, further configured to filter the plurality of contours to detect edges and calculating the representative normal vector from the filtered contours.

18. The system of claim 16, further configured to use brightness gradients to calculate the representative normal vector.

19. The system of claim 16, further configured to use brightness contours to calculate the representative normal vector.

20. The system of claim 16, further configured to average a plurality of normal vectors.

21. The system of claim 20, further configured to prepare a histogram of the plurality of normal vectors;

calculate an argument value for the histogram; and

using the argument value as the representative normal vector.

22. A non-transitory computer readable storage medium impressed with computer program instructions to initialize an orientation of a three-dimensional (3D) model of an object, the instructions, when executed on a processor, implement a method comprising:

accessing at least one three-dimensional (3D) model of the object and observed information of the object moving in space; and

determining a primary orientation parameter for the 3D model from the observed information, including;

detecting a plurality of contours for the object in the observed information and calculating a representative normal vector for at least some of the detected contours;

accessing a tangent vector representing a 3D angle tangent to a point on a surface of the object and drawn to a point of observation;

calculating a primary orientation parameter for the object using a cross-product of the representative normal vector and the tangent vector; and

using the calculated primary orientation parameter to initialize the orientation of the 3D model of the object.