PALM GESTURE RECOGNITION METHOD AND DEVICE AS WELL AS HUMAN-MACHINE INTERACTION METHOD AND APPARATUS

• US 20140198031A1
• Filed: 12/30/2013
• Published: 07/17/2014
• Est. Priority Date: 01/16/2013
• Status: Active Grant

• Current Assignee: Ricoh Company Limited
• Original Assignee: Ricoh Company Limited
• Inventors: XIONG, Huaixin
• Granted Patent: US 9,104,242 B2
• Time in Patent Office: Days
• US Class Current: 345/156
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06V 40/113 Recognition of static hand signs

Abstract

Disclosed is a palm gesture recognition method comprising a step of obtaining plural images according to an order of time; a step of acquiring plural palm shaped images from the plural images; a step of extracting plural features describing an open or closed palm gesture from each of the plural palm shaped images; a step of calculating a maximum feature difference vector formed by a maximum difference of each of the plural features; and a step of determining, on the basis of the maximum feature difference vector, that there is the open or closed palm gesture or there isn'"'"'t the open or closed palm gesture in the plural images.

Claims

1. An open-or-closed palm gesture recognition method used to recognize an open or closed palm gesture by utilizing image processing, the method comprising:

an image obtainment step of obtaining plural images according to an order of time;

a palm shape image acquirement step of acquiring plural palm shaped images from the plural images;

a feature extraction step of extracting plural features describing the open or closed palm gesture from each of the plural palm shaped images, wherein, each of the plural features has a monotonic characteristic that values corresponding to this feature obtained during a process between the open palm gesture and the closed palm gesture gradually decrease or increase;

a maximum feature difference vector calculation step of calculating a maximum feature difference vector formed by a maximum difference of each of the plural features, wherein, the maximum difference of the corresponding feature, which has a sign attribute and an width attribute, is calculated by obtaining a difference related to the corresponding features between two of the plural palm shaped images according to a predetermined order, and an absolute value of the maximum difference is a maximum absolute value of a difference related to the corresponding features between any two of the plural palm shaped images; and

an open-or-closed palm gesture determination step of determining, based on the maximum feature difference vector, that there is the open or closed palm gesture or there isn'"'"'t the open or closed palm gesture in the plural images.

2. The method according to claim 1, wherein:

in a case where there is the open or closed palm gesture, a start frame and an end frame of a hand action related to the open or closed palm gesture are determined,

wherein,

two images related to a maximum difference of each of the plural features are determined, and the two images are regarded as a candidate start frame and a candidate end frame of the hand action related to the open or closed palm gesture in a view of the corresponding features; and

the start frame and the end frame of the hand action related to the open or closed palm gesture are determined based on a weight of each of the plural features as well as the candidate start frame and the candidate end frame of the hand action related to the open or closed palm gesture.

3. The method according to claim 1, wherein:

a number of the plural images is determined based on a frequency histogram of a number of hand action frames and a predetermined level of confidence.

4. The method according to claim 1, wherein, the plural image are selected in sequence from an input image sequence by using a sliding window, and

the method further comprises:

a step of moving a position of the sliding window in sequence along the image sequence;

a step of carrying out the palm shape image acquirement step, the feature extraction step, the maximum feature difference vector calculation step, and the open-or-closed palm gesture recognition step with respect to plural images in the sliding window after movement; and

a step of combining, if an open or closed palm gesture obtained from a result of open-or-closed palm gesture recognition carried out with regard to plural images in the sliding window before movement is the same with that obtained from a result of open-or-closed palm gesture recognition carried out with respect to plural images in the sliding window after movement, the two results of open-or-closed palm gesture recognition as one open or closed palm gesture, and correcting a start frame and an end frame of a hand action related to the open or closed palm gesture accordingly.

5. The method according to claim 1, wherein:

The plural features extracted in the feature extraction step are selected from a length, a width, and an area of a minimum rectangle covering a palm along a main direction of the palm;

a length, an area, and a defective value of a convex polygon based on an outline of the palm; and

a length of the outline of the palm and an area of the palm surrounded by the outline.

10. A human-machine interaction method comprising:

a step of controlling, based on a motion of a hand and a previous gesture of the hand, when an open-or-closed palm gesture recognition process starts;

a step of carrying out, when the open-or-closed palm gesture recognition process is controlled to start, the open-or-closed palm gesture recognition process by utilizing the open-or-closed palm gesture recognition method according to claim 1; and

a step of letting, when an open or closed palm gesture is recognized, the open or closed palm gesture corresponding to a control command, and then carrying out the corresponding control command so as to control an operation and display of a target on a display device.

6. An open-or-closed palm gesture recognition device configured to recognize an open or closed palm gesture by utilizing image processing, the device comprising:

an image obtainment part configured to obtain plural images according to an order of time;

a palm shape image acquirement part configured to acquire plural palm shaped images from the plural images;

a feature extraction part configured to extract plural features describing the open or closed palm gesture from each of the plural palm shaped images, wherein, each of the plural features has a monotonic characteristic that values corresponding to this feature obtained during a process between the open palm gesture and the closed palm gesture gradually decrease or increase;

a maximum feature difference vector calculation part configured to calculate a maximum feature difference vector formed by a maximum difference of each of the plural features, wherein, the maximum difference of the corresponding feature, which has a sign attribute and a width attribute, is calculated by obtaining a difference related to the corresponding features between two of the plural palm shaped images according to a predetermined order, and an absolute value of the maximum difference is a maximum absolute value of a difference related to the corresponding features between any two of the plural palm shaped images; and

an open-or-closed palm gesture determination part configured to determine, based on the maximum feature difference vector, that there is the open or closed palm gesture or there isn'"'"'t the open or closed palm gesture in the plural images.

7. A human-machine interaction apparatus comprising:

the open-or-closed palm gesture recognition device according to claim 6;

an open-or-closed palm gesture recognition control device configured to control, based on a motion of a hand and a previous gesture of the hand, when the open-or-closed palm gesture recognition device starts to carry out an open-or-closed palm gesture recognition process; and

a gesture-command correspondence device configured to let, when an open or closed palm gesture is recognized, the open or closed palm gesture correspond to a control command, and to carry out the corresponding control command.

8. The apparatus according to claim 7, further comprising:

a palm position detector configured to detect whether or not a position of a palm is unchanged, wherein,

the open-or-closed palm gesture recognition control device controls, based on a result detected by the palm position detector, when the open-or-closed palm gesture recognition device starts to carry out the open-or-closed palm gesture recognition process.

9. The apparatus according to claim 8, wherein, the open-or-closed palm gesture recognition device controls, based on the result detected by the palm position detector, when the open-or-closed palm gesture recognition device stars to carry out the open-or-closed palm gesture recognition process, by carrying out steps as follows:

(a) initializing a flag bFlag to FALSE;

(b) determining that the flag bFlag is TRUE or FALSE, and then, detecting by the palm position detector, if the flag bFlag is FALSE, whether or not the position of the palm is unchanged;

(c) carrying out an open-or-closed palm gesture recognition process by utilizing the open-or-closed palm gesture recognition device if the flag fFlag is TRUE or it is detected that the position of the palm is unchanged;

(d) setting the flag bFlag to TRUE if a closed palm gesture is recognized, or setting the flag bFlag to FALSE if an open palm gesture is recognized; and

(e) carrying out STEPS (b) to (d) in sequence with respect to a next input image sequence.