Gesture recognition method and gesture recognition apparatus using the same

• US 9,576,193 B2
• Filed: 02/25/2015
• Issued: 02/21/2017
• Est. Priority Date: 11/17/2014
• Status: Active Grant

• Current Assignee: Wistron Corporation
• Original Assignee: Wistron Corporation
• Inventors: Yang, Jie-Ci, Lin, Hui-Chen, Shen, Tzu-Chiang, Kao, Meng-Chao
• Primary Examiner(s): OSORIO, RICARDO
• Application Number:
  US14/631,822
• Publication Number:
  US 20160140384A1
• Time in Patent Office: 727 Days
• US Class Current: 1/1
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0304 Detection arrangements using opto-electronic means constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042
• CPC Class Codes: G06T 2207/10048 Infrared image
• CPC Class Codes: G06T 7/194 involving foreground-background segmentation
• CPC Class Codes: G06T 7/254 involving subtraction of images
• CPC Class Codes: G06V 10/143 Sensing or illuminating at different wavelengths
• CPC Class Codes: G06V 40/28 Recognition of hand or arm movements, e.g. recognition of deaf sign language static hand signs G06V40/113
• CPC Class Codes: H04N 5/33 Transforming infrared radiation cameras or camera modules for generating image signals from infrared radiation H04N23/20; circuitry of SSIS for transforming infrared radiation into image signals H04N25/20

Abstract

A gesture recognition method and an gesture recognition apparatus using the same method are proposed. The method includes the following steps: transmitting an infrared signal, and detecting whether an object appears in a predetermined range based on a first infrared signal retrieved from the infrared signal being reflected; when the object appears in the predetermined range, capturing a plurality of images including the object and a plurality of second infrared signals retrieved from the infrared signal being reflected; and recognizing the images to retrieve a horizontal movement direction of a gesture of the object within the predetermined range, and recognizing the second infrared signals to retrieve a depth movement direction of the gesture within the predetermined range.

Claims

1. A gesture recognition method, suitable for a gesture recognition apparatus having an image capturing unit, and comprising:

transmitting an infrared signal, and detecting whether an object appears in a predetermined range based on a first infrared signal retrieved from the infrared signal being reflected;

when the object appears in the predetermined range, identifying a current position of the object according to a signal strength of the first infrared signal;

adjusting an image capturing range to face the object based on the current position to capture a plurality of images including the object and a plurality of second infrared signals retrieved from the infrared signal being reflected; and

recognizing the images to retrieve a horizontal movement direction of a gesture of the object within the predetermined range, and recognizing the second infrared signals to retrieve a depth movement direction of the gesture within the predetermined range.

2. The gesture recognition method of claim 1, wherein the object is a human body, and the step of adjusting the image capturing range to face the object based on the current position to capture the plurality of images including the object and the plurality of second infrared signals retrieved from the infrared signal being reflected comprises:

executing a face recognition mechanism for the human body to recognize a face position; and

using a reference point in the face position to serve as a data center point of the image capturing range.

3. The gesture recognition method of claim 1, wherein the step of recognizing the second infrared signals to retrieve a depth movement direction of the gesture within the predetermined range comprises:

determining whether an absolute value of a signal strength difference between a t^th second infrared signal and a (t+1)^th second infrared signal among the second infrared signals is greater than a predetermined threshold;

when the absolute value is greater than the predetermined threshold, determining whether a signal strength of the (t+1)^th second infrared signal lasts for a predetermined time;

when the signal strength of the (t+1)^th second infrared signal lasts for the predetermined time, determining whether the signal strength of the (t+1)^th second infrared signal is greater than a signal strength of the t^th second infrared signal;

if yes, recognizing that the gesture is a pushing gesture; and

if no, recognizing that the gesture is a pulling gesture.

4. The gesture recognition method of claim 1, wherein the step of recognizing the images to retrieve the horizontal movement direction of the gesture of the object within the predetermined range comprises:

recognizing a moving palm appearing in the images, and defining a plurality of feature points on the moving palm;

retrieving a plurality of estimated moving distances of the feature points in the images;

finding a plurality of robust feature points from the feature points based on the estimated moving distances;

retrieving a plurality of movement parameters of each of the robust feature points in the images;

defining a plurality of moving directions of the robust feature points according to the movement parameters of each of the robust feature points; and

deciding a gesture moving direction of the gesture according to the moving directions of the robust feature points.

5. The gesture recognition method of claim 4, wherein and the step of recognizing the moving palm appearing in the images comprises:

building a background model based on the images;

generating a plurality of gray scale images of the images, wherein a k^th gray scale image among the gray scale images is corresponding to a k^th image among the images;

for a (k+1)^th gray scale image and the k^th gray scale image, subtracting the k^th gray scale image from the (k+1)^th gray scale image to generate a gray scale difference image; and

finding the moving palm from the gray scale difference image.

6. The gesture recognition method of claim 5, further comprising:

when a proportion of the background model occupied by a foreground in the images is greater than a predetermined threshold, stopping recognizing the gesture.

7. The gesture recognition method of claim 4, wherein an i^th feature point among the feature points is corresponding to an i^th estimated moving distance among the estimated moving distances, and the step of finding the robust feature points from the feature points based on the estimated moving distances comprises:

when the i^th estimated moving distance is greater than a predetermined distance threshold, defining that the i^th feature point is one of the robust feature points.

8. The gesture recognition method of claim 4, wherein the movement parameters of a j^th robust feature point among the robust feature points comprises a total horizontal movement distance, a total vertical movement distance, a horizontal moving start point, a horizontal moving destination, a vertical moving start point and a vertical moving destination in the images, and the j^th robust feature point is corresponding to a j^th moving direction among the moving directions,

wherein the step of defining the moving directions of the robust feature points according to the movement parameters of each of the robust feature points comprises:

for the j^th robust feature point, determining whether the total horizontal movement distance is greater than the total vertical movement distance;

when the total horizontal movement distance is greater than the total vertical movement distance, determining whether the horizontal moving start point is greater than the horizontal moving destination;

if yes, defining that the j^th moving direction is a leftward direction;

if no, defining that the j ^th moving direction is a rightward direction;

when the total horizontal movement distance is not greater than the total vertical movement distance, determining whether the vertical moving start point is greater than the vertical moving destination;

if yes, defining that the j^th moving direction is an upward direction; and

if no, defining that the j^th moving direction is a downward direction.

9. The gesture recognition method of claim 8, wherein the step of deciding the gesture moving direction of the gesture according to the moving directions of the robust feature points comprises:

counting an amount of each of the upward direction, the downward direction, the leftward direction and the rightward direction among the moving directions; and

defining the moving direction with a highest amount among the moving directions is the gesture moving direction of the gesture.

10. The gesture recognition method of claim 1, wherein the gesture recognition apparatus comprises an infrared transmitter for transmitting the infrared signal and the image capturing unit for continuously capturing the images and the second infrared signals, and the step of recognizing the images and the second infrared signals is performed only when detecting that the object appears in the predetermined range.

11. The gesture recognition method of claim 1, wherein the gesture recognition apparatus comprises an infrared transmitter for transmitting the infrared signal, an infrared receiver for receiving the first infrared signal and the second infrared signals, and the image capturing unit for capturing the images and the second infrared signals, and the image capturing unit is activated to capture the images and the second infrared signals only when detecting that the object appears in the predetermined range.

12. A gesture recognition apparatus, comprising:

an infrared transmitter, transmitting a plurality of infrared signals;

an image capturing unit, configured to receive a first infrared signal, a plurality of second infrared signals and capture a plurality of images, wherein each of the first infrared signal and the second infrared signals is one of the infrared signals being reflected;

a storage unit, storing a plurality of modules; and

a processing unit, connected to the image capturing unit, the infrared transmitter and the storage unit, accessing and executing the modules, and the modules comprising;

a detection module, detecting whether an object appears in a predetermined range based on the first infrared signal;

an identification module, identifying a current position of the object according to a signal strength of the first infrared signal when the object appears in the predetermined range;

an adjusting module, adjusting an image capturing range of the image capturing unit to face the object for the image capturing unit to capture the images and the second infrared signals including the object; and

a recognition module, configured to recognize the images to retrieve a horizontal movement direction of a gesture of the object within the predetermined range, and recognize the second infrared signals to retrieve a depth movement direction of the gesture within the predetermined range.

13. The gesture recognition apparatus of claim 12, wherein the modules further comprising:

an activation module, configured to activate the identification module to recognize the images and the second infrared signals only when detecting that object exists in the predetermined range.

14. A gesture recognition apparatus, comprising:

an infrared transmitter, transmitting a plurality of infrared signals;

an image capturing unit, configured to capture a plurality of images;

an infrared receiver, receiving a first infrared signal and a plurality of second infrared signals, wherein each of the first infrared signal and the second infrared signals is one of the infrared signals being reflected;

a storage unit, storing a plurality of modules; and

a processing unit, connected to the image capturing unit, the infrared transmitter, the infrared receiver and the storage unit, accessing and executing the modules, and the modules comprising;

a detection module, detecting whether an object appears in a predetermined range based on the first infrared signal;

an identification module, identifying a current position of the object according to a signal strength of the first infrared signal when the object appears in the predetermined range;

an adjusting module, adjusting an image capturing range of the image capturing unit to face the object for the image capturing unit to capture the images and the second infrared signals including the object; and

a recognition module, configured to recognize the images to retrieve a horizontal movement direction of a gesture of the object within the predetermined range, and recognize the second infrared signals to retrieve a depth movement direction of the gesture within the predetermined range.

15. The gesture recognition apparatus of claim 14, wherein the modules further comprising:

an activation module, configured to activate the image capturing module to capture the images only when detecting that the object exists in the predetermined range.