Electronic device and gesture recognition method applied therein

• US 10,282,601 B2
• Filed: 07/18/2017
• Issued: 05/07/2019
• Est. Priority Date: 06/21/2017
• Status: Active Grant

• Current Assignee: Hon Hai Precision Industry Co., Ltd.
• Original Assignee: Hon Hai Precision Industry Co., Ltd.
• Inventors: Yang, Jung-Hao, Tsai, Tung-Tso, Chuang, Chih-Yuan, Kuo, Chin-Pin
• Primary Examiner(s): Chang, Jon
• Application Number:
  US15/652,698
• Publication Number:
  US 20180373927A1
• Time in Patent Office: 658 Days
• CPC Class Codes: G06F 18/2415 based on parametric or probabilistic models, e.g. based on likelihood ratio or false acceptance rate versus a false rejection rate
• CPC Class Codes: G06F 18/28 Determining representative reference patterns, e.g. by averaging or distorting; Generating dictionaries
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0304 Detection arrangements using opto-electronic means constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042
• CPC Class Codes: G06T 2207/10024 Color image
• CPC Class Codes: G06T 2207/10028 Range image; Depth image; 3D point clouds
• CPC Class Codes: G06T 2207/20072 Graph-based image processing
• CPC Class Codes: G06T 2207/20081 Training; Learning
• CPC Class Codes: G06T 2207/30196 Human being; Person
• CPC Class Codes: G06T 2207/30241 Trajectory
• CPC Class Codes: G06T 7/223 using block-matching
• CPC Class Codes: G06T 7/248 involving reference images or patches
• CPC Class Codes: G06T 7/55 from multiple images
• CPC Class Codes: G06V 10/25 Determination of region of interest [ROI] or a volume of interest [VOI]
• CPC Class Codes: G06V 10/764 using classification, e.g. of video objects
• CPC Class Codes: G06V 40/113 Recognition of static hand signs
• CPC Class Codes: G06V 40/28 Recognition of hand or arm movements, e.g. recognition of deaf sign language static hand signs G06V40/113

Abstract

An electronic device, which can recognize and be controlled by a gesture of a hand, obtains an image that comprises the hand and image depth levels of objects within the image. Static objects are filtered from the image. A first establishing module obtains hand coordinate information and establishes a first block comprising the hand. Modules further obtain a depth level of each pixel of the first block and counts the number of pixels of each depth level. A second establishing module obtains hand depth information according to the counting result and establishes a second block including the hand. A recognizing module detects a moving track of the hand in the second block and recognizes a gesture of the hand according to a library of gestures. A gesture recognition method is also provided.

Claims

1. A gesture recognition method comprising:

obtaining an image that comprises a hand and image depth information;

filtering static objects which are comprised in the image;

obtaining hand coordinate information and establishing a first block comprising the hand based on the hand coordinate information;

obtaining depth information of each pixel of the first block and counting a number of pixels of each depth level;

obtaining hand depth information according to a counting result and establishing a second block according to the hand depth information; and

detecting a moving track of the hand in the second block and recognizing a gesture of the hand according to the moving track.

2. The gesture recognition method of claim 1, wherein the image is a red, green, and blue (RGB) colors image and the image comprises at least one object.

3. The gesture recognition method of claim 1, wherein the step of filtering the static objects comprised in the image comprises:

filtering the static objects comprised in the image through a Gaussian mixture model (GMM).

4. The gesture recognition method of claim 1, wherein the step of obtaining the hand coordinate information and establishing the first block comprising the hand according to the hand coordinate information comprises:

establishing characteristic values of the hand through a deep learning algorithm;

obtaining the hand coordinate information according to the characteristic values; and

establishing the first block comprising the hand according to the hand coordinate information.

5. The gesture recognition method of claim 4, wherein a percentage of an area of the hand in a total area of the first block is greater than a predetermined percentage value.

6. The gesture recognition method of claim 4, wherein the step of obtaining the depth information of each pixel of the first block and counting the number of pixels of each depth level comprises:

extracting the depth information of each pixel of the first block according to the coordinate information of each pixel; and

counting the number of pixels of each depth level through a histogram.

7. The gesture recognition method of claim 6, wherein the step of obtaining the hand depth information according to the counting result and establishing the second block according to the hand depth information comprises:

filtering first pixels from the histogram that depth level of each first pixel is less than a predetermined depth value;

extracting two depth levels from the histogram, wherein the two depth levels comprise top two most numbers of pixels; and

selecting a smaller value depth level from the two depth levels as the hand depth information to establish the second block.

8. The gesture recognition method of claim 7, wherein the step of establishing the second block according to the hand depth information comprises:

establishing a depth level range according to the hand depth information;

filter second pixels from the first block which are not within the depth level range;

generating a planar block comprising the hand according a filtered result; and

establishing the second block according to the planar block and the hand depth information.

9. The gesture recognition method of claim 1, wherein the step of recognizing the gesture of the hand according to the moving track comprises:

establishing a gesture library through a deep learning algorithm; and

recognizing the gesture of the hand according to the moving track and the gesture library.

10. An electronic device comprising:

at least one storage configured to store a plurality of modules, the plurality of modules being a collection of instructions of an application operable in the device;

at least one processor configured to execute the plurality of modules, the modules comprising;

an image obtaining module obtaining an image that comprises a hand and image depth information;

a first filtering module filtering static objects which are comprised in the image;

a first establishing module obtaining hand coordinate information and establishing a first block comprising the hand according to the hand coordinate information;

a counting module obtaining depth information of each pixel of the first block and counting a number of pixels of each depth level;

a second establishing module obtaining hand depth information according to a counting result and establishing a second block according to the hand depth information; and

a recognizing module detecting a moving track of the hand in the second block and recognizing a gesture of the hand according to the moving track.

11. The electronic device of claim 10, wherein the image is a RGB colors image and the image comprises at least one object.

12. The electronic device of claim 10, wherein the first filtering module filters the static objects comprised in the image through a GMM.

13. The electronic device of claim 10, wherein the first establishing module establishes characteristic values of the hand through a deep learning algorithm and obtains the hand coordinate information according to the characteristic values, the first establishing module further establishes the first block comprising the hand according to the hand coordinate information.

14. The electronic device of claim 13, wherein a percentage of an area of the hand in a total area of the first block is greater than a predetermined percentage value.

15. The electronic device of claim 13, wherein the counting module extracts the depth information of each pixel of the first block according to the coordinate information of each pixel and counts the number of pixels of each depth level through a histogram.

16. The electronic device of claim 15, wherein the second establishing module filters first pixels from the histogram that depth information of each first pixel is less than a predetermined depth value and extracts two depth levels from the histogram, wherein the two depth information comprise top two most numbers of pixels, the second establishing module further selects a smaller value depth level from the two depth levels as the hand depth information to establish the second block.

17. The electronic device of claim 16, wherein the second establishing module establishes a depth level range according to the hand depth information and filters second pixels from the first block which are not within the depth information range, the second establishing module further generates a planar block comprising the hand according a filtered result and establishes the second block according to the planar block and the hand depth information.

18. The electronic device of claim 10, wherein the recognizing module establishes a gesture library through a deep learning algorithm and recognizes the gesture of the hand according to the moving track and the gesture library.