SYSTEMS AND METHODS FOR MACHINE CONTROL

• US 20140369558A1
• Filed: 05/16/2014
• Published: 12/18/2014
• Est. Priority Date: 01/17/2012
• Status: Active Grant

• Current Assignee: Leap Motion Incorporated (Ultraleap Limited)
• Original Assignee: David Holz
• Inventors: HOLZ, DAVID
• Granted Patent: US 9,679,215 B2
• Time in Patent Office: Days
• US Class Current: 382/103
• CPC Class Codes: G06F 2218/08 Feature extraction
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/30196 Human being; Person
• CPC Class Codes: G06T 7/586 from multiple light sources, e.g. photometric stereo
• CPC Class Codes: G06V 10/145 Illumination specially adapted for pattern recognition, e.g. using gratings
• CPC Class Codes: G06V 10/25 Determination of region of interest [ROI] or a volume of interest [VOI]
• CPC Class Codes: G06V 20/64 Three-dimensional objects
• CPC Class Codes: G06V 40/107 Static hand or arm

Abstract

A region of space may be monitored for the presence or absence of one or more control objects, and object attributes and changes thereto may be interpreted as control information provided as input to a machine or application. In some embodiments, the region is monitored using a combination of scanning and image-based sensing.

Claims

1. A computer implemented method for conducting machine control, comprising:

conducting scanning of a region of space including;

directing at least two emission cycles to form at least two scan patterns from an emission region to a region of space that might include a control object;

detecting a reflectance of the at least two scan patterns; and

determining that the detected reflectance indicates a presence of an object in the region of space;

determining a first object attribute set of one or more object attributes of the object for a first one of the at least two scan patterns and a second object attribute set of one or more object attributes of the object for a second one of the at least two scan patterns;

analyzing the first object attribute set and the second object attribute set to determine a potential control portion of the object;

comparing the first object attribute set and the second object attribute set to determine control information; and

responding to the indicated control information according to response criteria.

2. A computer implemented method according to claim 1, wherein one or more of the object attributes comprise attributes of the control portion of the object;

and wherein comparing the first object attribute set and the second object attribute set includes comparing control portion attributes selected from the first set with control portion attributes selected from the second set.

3. A computer implemented method according to claim 2, wherein directing comprises directing an emission cycle in a first scan pattern;

and wherein determining first and second sets of one or more object attributes comprises;

conducting a second scanning in a second pattern.

4. A computer implemented method according to claim 3, wherein the second scan pattern is more refined than the first scan pattern.

5. A computer implemented method according to claim 4, wherein the refined scan pattern captures surface detail about the control object.

6. A computer implemented method according to claim 5, wherein determining first and second sets of one or more object attributes includes determining control-portion attributes based at least in part on captured surface detail about the object.

7. A computer implemented method according to claim 6, wherein comparing the first object attribute set with the second object attribute set to determine control information includes comparing differences in captured surface detail of control portion(s) of the object from control portion attributes of the first scan and the second scan to determine a change indicating control information.

8. A computer implemented method according to claim 7, wherein determining a change indicating control information includes determining whether an engagement gesture has occurred.

9. A computer implemented method according to claim 7, wherein determining a change indicating control information includes determining whether a prospective user has come into vicinity of a machine under control.

10. A computer implemented method according to claim 7, wherein determining a change indicating control information includes determining whether an object is there or not present.

11. A computer implemented method according to claim 7, wherein determining a change indicating control information includes determining a correction to control information by comparing the control information to a prior determined control information.

12. A computer implemented method according to claim 3, wherein the detecting includes detecting reflectance of the first pattern using at least one suitable photo-sensitive element.

13. A computer implemented method according to claim 3, wherein the detecting includes capturing an image of the reflectance of the first pattern using at least one suitable photo-sensitive array element.

14. A computer implemented method of claim 1, wherein the directing is configured to cause a light emission pattern of a first light emitter to interfere or to create an interference pattern with a light emission pattern of a second light emitter.

15. A computer implemented method of claim 1, wherein the directing includes a first power state variance at a first light emitter at least partially concurrent with second power state variance at a second light emitter.

16. A computer implemented method of claim 1, wherein the directing includes extinguishing a first light emitter completely before activating a second light emitter.

17. A computer implemented method of claim 1, wherein the emission region includes a first emission point and a second emission point rotationally pivot-able about one or more axes having an origin point within the emission region.

18. A computer implemented method of claim 17, wherein the first emission point is same as or different from the second emission point.

19. A computer implemented method according to claim 7, wherein analyzing first and second object attribute sets to determine a potential control portion of the object includes finding a known feature.

20. A computer implemented method according to claim 19, finding a known feature includes determining a reflector affixed to the object.

21. A computer implemented method according to claim 1, wherein response criteria includes determining whether to respond to the control information.

22. A computer implemented method according to claim 2, wherein determining object attribute set comprises:

in addition to detecting the at least two scan patterns, capturing at least one image with an imaging system including at least one camera.

23. A computer implemented method for conducting machine control, comprising:

monitoring a region of space for a presence of an object using a first sensing modality;

upon detection of the presence of an object, using a second sensing modality to determine first and second sets of object attributes of the object;

comparing the first object attribute set and the second object attribute set to determine control information; and

responding to the indicated control information according to response criteria.

24. A computer implemented method according to claim 23, wherein monitoring the region using the first sensing modality comprises scanning the region with an emission and detecting a reflectance.

25. A computer implemented method according to claim 24, wherein using the second sensing modality comprises acquiring images of the region.

26. A computer implemented method according to claim 23, wherein monitoring the region using the first sensing modality comprises acquiring images of the region.

27. A computer implemented method according to claim 26, wherein using the second sensing modality comprises scanning the region with an emission and detecting a reflectance.

28. A system for conducting machine control, comprising:

an emission module for directing an emission cycle to a region of interest in accordance with a scan pattern;

a detection module for (i) detecting a time-variable signal indicative of reflection of the emission by an object in the region of interest, and (ii) acquiring images of the region of interest;

a controller for coordinating operation of the emission and detection modules;

a computational facility for processing the reflection signal and the images to determine therefrom a presence of the object and object attributes associated therewith, and for inferring machine-control information from the object attributes.

29. A system according to claim 28, wherein the detection module comprises one or more cameras for acquiring the images of the region of interest.

30. A system according to claim 28, wherein the detection module comprises a photosensitive element for detecting the reflection.

31. A system according to claim 28, wherein the detection module further comprises circuitry for extracting the time-variable signal based at least in part upon the reflection.