DYNAMIC GESTURE BASED SHORT-RANGE HUMAN-MACHINE INTERACTION

• US 20130278504A1
• Filed: 11/01/2011
• Published: 10/24/2013
• Est. Priority Date: 11/01/2011
• Status: Active Grant

• Current Assignee: Intel Corporation
• Original Assignee: Intel Corporation
• Inventors: Tong, Xiaofeng, Ding, Dayong, Li, Wenlong
• Granted Patent: US 9,164,589 B2
• Time in Patent Office: Days
• US Class Current: 345/158
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06V 40/28 Recognition of hand or arm movements, e.g. recognition of deaf sign language static hand signs G06V40/113

Abstract

Systems, devices and methods are described including starting a gesture recognition engine in response to detecting an initiation gesture and using the gesture recognition engine to determine a hand posture and a hand trajectory in various depth images. The gesture recognition engine may then use the hand posture and the hand trajectory to recognize a dynamic hand gesture and provide corresponding user interface command.

Claims

1-30. -30. (canceled)

31. A computer-implemented method for recognizing a dynamic hand gesture, comprising:

detecting an initiation gesture;

starting a gesture recognition engine in response to detecting the initiation gesture;

determining, using the gesture recognition engine, a hand posture in at least one image of a plurality of images;

determining, using the gesture recognition engine, a hand trajectory in the plurality of images;

determining, using the gesture recognition engine, a dynamic hand gesture in response to the hand posture and the hand trajectory; and

providing, using the gesture recognition engine, a user interface command in response to determining the dynamic hand gesture.

32. The method of claim 31, wherein determining the hand posture comprises:

detecting a hand in the at least one image;

segmenting, in response to detecting the hand, the at least one image into a binary image including a hand region; and

determining at least one shape feature in the hand region.

33. The method of claim 32, further comprising:

determining, using artificial neural multi-layer perceptron (MLP) analysis, a class corresponding to the at least one shape feature.

34. The method of claim 32, wherein the a least one shape feature comprises at least one of eccentricity, compactness, orientation, rectangularity, width center, height center, minimum box angle, minimum box width, number of defects, difference between left and right portions, or difference between top and bottom portions shape features.

35. The method of claim 32, wherein detecting the hand comprises using a cascade speeded-up robust feature (SURF) detector to detect the hand.

36. The method of claim 31, wherein determining the hand trajectory comprises:

determining a plurality of hue-saturation-depth (HSD) histograms; and

tracking, using mean-shift analysis, a moving hand in response to the plurality of HSD histograms.

37. The method of claim 31, wherein determining the dynamic hand gesture comprises applying a hidden-markov model (HMM) to identify the dynamic hand gesture.

38. An article comprising a computer program product having stored therein instructions that, if executed, result in:

detecting an initiation gesture;

starting a gesture recognition engine in response to detecting the initiation gesture;

determining, using the gesture recognition engine, a hand posture in at least one image of a plurality of images;

determining, using the gesture recognition engine, a hand trajectory in the plurality of images;

determining, using the gesture recognition engine, a dynamic hand gesture in response to the hand posture and the hand trajectory; and

providing, using the gesture recognition engine, a user interface command in response to determining the dynamic hand gesture.

39. The article of claim 38, wherein determining the hand posture comprises:

detecting a hand in the at least one image;

segmenting, in response to detecting the hand, the at least one image into a binary image including a hand region; and

determining at least one shape feature in the hand region.

40. The article of claim 39, wherein the a least one shape feature comprises at least one of eccentricity, compactness, orientation, rectangularity, width center, height center, minimum box angle, minimum box width, number of defects, difference between left and right portions, or difference between top and bottom portions shape features.

41. The article of claim 39, wherein detecting the hand comprises using a cascade speeded-up robust feature (SURF) detector to detect the hand.

42. The article of claim 38, wherein determining the hand trajectory comprises:

determining a plurality of hue-saturation-depth (HSD) histograms; and

tracking, using mean-shift analysis, a moving hand in response to the plurality of HSD histograms.

43. The article of claim 38, wherein determining the dynamic hand gesture comprises applying a hidden-markov model (HMM) to identify the dynamic hand gesture.

44. An apparatus, comprising:

a processor configured to;

detect an initiation gesture;

start a gesture recognition engine in response to detecting the initiation gesture;

determine, using the gesture recognition engine, a hand posture in at least one image of a plurality of images;

determine, using the gesture recognition engine, a hand trajectory in the plurality of images;

determine, using the gesture recognition engine, a dynamic hand gesture in response to the hand posture and the hand trajectory; and

provide, using the gesture recognition engine, a user interface command in response to determining the dynamic hand gesture.

45. The apparatus of claim 44, wherein to determine the hand posture the processor is configured to:

detect a hand in the at least one image;

segment, in response to detecting the hand, the at least one image into a binary image including a hand region; and

determine at least one shape feature in the hand region.

46. The apparatus of claim 45, wherein the a least one shape feature comprises at least one of eccentricity, compactness, orientation, rectangularity, width center, height center, minimum box angle, minimum box width, number of defects, difference between left and right portions, or difference between top and bottom portions shape features.

47. The apparatus of claim 45, wherein to detect the hand the processor is configured to use a cascade speeded-up robust feature (SURF) detector.

48. The apparatus of claim 44, wherein to determine the hand trajectory the processor is configured to:

determine a plurality of hue-saturation-depth (HSD) histograms; and

track, using mean-shift analysis, a moving hand in response to the plurality of HSD histograms.

49. The apparatus of claim 44, wherein to determine the dynamic hand gesture the processor is configured to apply a hidden-markov model (HMM).

50. A system comprising:

an imaging device; and

a computing system, wherein the computing system is communicatively coupled to the imaging device and wherein, in response to depth images received from the imaging device, the computing system is to;

detect an initiation gesture;

start a gesture recognition engine in response to detecting the initiation gesture;

determine, using the gesture recognition engine, a hand posture in at least one image of a plurality of images;

determine, using the gesture recognition engine, a hand trajectory in the plurality of images;

determine, using the gesture recognition engine, a dynamic hand gesture in response to the hand posture and the hand trajectory; and

provide, using the gesture recognition engine, a user interface command in response to determining the dynamic hand gesture.

51. The system of claim 50, wherein to determine the hand posture the computing system is to:

detect a hand in the at least one image;

segment, in response to detecting the hand, the at least one image into a binary image including a hand region; and

determine at least one shape feature in the hand region.

52. The system of claim 51, wherein the a least one shape feature comprises at least one of eccentricity, compactness, orientation, rectangularity, width center, height center, minimum box angle, minimum box width, number of defects, difference between left and right portions, or difference between top and bottom portions shape features.

53. The system of claim 51, wherein to detect the hand the computing system is to use a cascade speeded-up robust feature (SURF) detector.

54. The system of claim 50, wherein to determine the hand trajectory the computing system is to:

determine a plurality of hue-saturation-depth (HSD) histograms; and

track, using mean-shift analysis, a moving hand in response to the plurality of HSD histograms.

55. The system of claim 50, wherein to determine the dynamic hand gesture the computing system is to apply a hidden-markov model (HMM).