Systems and methods for capturing motion in three-dimensional space

• US 9,070,019 B2
• Filed: 12/21/2012
• Issued: 06/30/2015
• Est. Priority Date: 01/17/2012
• Status: Active Grant

• Current Assignee: Lmi Liquidating Co., LLC
• Original Assignee: Leap Motion Incorporated (Ultraleap Limited)
• Inventors: Holz, David
• Primary Examiner(s): Bella, Matthew
• Assistant Examiner(s): Rosario, Dennis
• Application Number:
  US13/724,357
• Publication Number:
  US 20130182897A1
• Time in Patent Office: 921 Days
• Field of Search: 382/131, 382/196, 382/154, 703/2
• US Class Current: 1/1
• CPC Class Codes: A61B 2017/00207 with hand gesture control or hand gesture recognition
• CPC Class Codes: G01B 11/2433 for measuring outlines by shadow casting
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06T 17/00 Three dimensional [3D] modelling, e.g. data description of 3D objects
• CPC Class Codes: G06T 17/10 Constructive solid geometry [CSG] using solid primitives, e.g. cylinders, cubes
• CPC Class Codes: G06T 2207/10152 Varying illumination
• CPC Class Codes: G06T 2207/30196 Human being; Person
• CPC Class Codes: G06T 7/292 Multi-camera tracking
• CPC Class Codes: G06T 7/507 from shading G06T7/586 takes precedence
• CPC Class Codes: G06T 7/586 from multiple light sources, e.g. photometric stereo
• CPC Class Codes: G06T 7/593 from stereo images
• CPC Class Codes: G06V 10/145 Illumination specially adapted for pattern recognition, e.g. using gratings
• CPC Class Codes: G06V 20/40 in video content extracting overlay text G06V20/62; video retrieval G06F16/70; processing of video elementary streams in video servers H04N21/234; processing of video elementary streams in video clients H04N21/44
• CPC Class Codes: G06V 20/64 Three-dimensional objects
• CPC Class Codes: G06V 40/10 Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
• CPC Class Codes: G06V 40/107 Static hand or arm
• CPC Class Codes: G06V 40/11 Hand-related biometrics; Hand pose recognition
• CPC Class Codes: G06V 40/113 Recognition of static hand signs
• CPC Class Codes: G06V 40/20 Movements or behaviour, e.g. gesture recognition recognition of facial expressions G06V40/16
• CPC Class Codes: G06V 40/28 Recognition of hand or arm movements, e.g. recognition of deaf sign language static hand signs G06V40/113

Abstract

Methods and systems for capturing motion and/or determining the shapes and positions of one or more objects in 3D space utilize cross-sections thereof. In various embodiments, images of the cross-sections are captured using a camera based on reflections therefrom or shadows cast thereby.

Claims

1. A method of locating a control object appendage in three dimensional (3D) space, including:

recording images of a control object appendage in 3D space using at least two geometrically distinct predetermined vantages;

calculating by a processor four co-planar tangents to observed edges of the control object appendage from the recorded images;

fitting an ellipse by a processor to the cross-section of the control object appendage by selecting the ellipse from a family of ellipses that fit the four co-planar tangents using an assumed parameter;

repeatedly fitting contiguous cross-sections to the control object appendage; and

constructing the control object appendage in 3D space from the contiguous cross-sections.

2. The method of claim 1, further including substituting one or more fitted parameters of a first fitted ellipse for one or more of the four co-planar tangents and the assumed parameter when fitting an adjacent second ellipse.

3. A method of locating a complex control object in 3D space, including:

for a complex control object model that includes a palm and multiple fingers, applying the method of claim 2 to construct the multiple fingers of control object appendages; and

fitting cross sections of a palm to observed edges from the images in positions correlated with the multiple fingers.

4. The method of claim 1, further including using a first fitted ellipse to filter fits of additional ellipses to the contiguous cross-sections.

5. A method of tracking motion of a complex control object in 3D space, including:

repeatedly applying the method of claim 4 over time; and

calculating motion of the complex control object over time based on differences between modeled locations of the complex control object over time.

6. A method of tracking motion of a control object appendage in 3D space, including:

repeatedly applying the method of claim 1 over time; and

calculating motion of the control object appendage over time based on differences between modeled locations of the control object appendage over time.

7. The method of claim 1, further including:

determining that only three co-planar tangents are available in portions of the recorded images and fitting a circle instead of an ellipse for the portions of the recorded images, including;

calculating by a processor the three co-planar tangents to observed edges of the control object appendage from the recorded images; and

fitting a circle by a processor to the cross-section of the control object appendage using at least the three co planar tangents.

8. A method of locating a complex control object in 3D space, including:

for a complex control object model that includes a palm and multiple fingers, applying the method of claim 7 to construct the multiple fingers of control object appendages; and

fitting cross sections of a palm to observed edges from the images in positions correlated with the multiple fingers.

9. A method of tracking motion of a complex control object in 3D space, including:

repeatedly applying the method of claim 8 over time; and

calculating motion of the complex control object over time based on differences between modeled locations of the complex control object over time.

10. A method of tracking motion of a control object appendage in 3D space, including:

repeatedly applying the method of claim 7 over time; and

calculating motion of the control object appendage over time based on differences between modeled locations of the control object appendage over time.