GESTURE RECOGNITION SYSTEM

• US 20160085312A1
• Filed: 09/24/2014
• Published: 03/24/2016
• Est. Priority Date: 09/24/2014
• Status: Abandoned Application

• Current Assignee: Himax Technologies Ltd. (Himax Technologies Incorporated), NCKU Research & Development Foundation
• Original Assignee: Himax Technologies Ltd. (Himax Technologies Incorporated), NCKU Research & Development Foundation
• Inventors: SHIEH, MING-DER, YANG, DER-WEI, WANG, TZUNG-REN, Gan, Jia-Ming
• Application Number:
  US14/495,808
• Publication Number:
  US 20160085312A1
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06V 40/107 Static hand or arm
• CPC Class Codes: G06V 40/28 Recognition of hand or arm movements, e.g. recognition of deaf sign language static hand signs G06V40/113
• CPC Class Codes: H04N 13/271 wherein the generated image signals comprise depth maps or disparity maps
• CPC Class Codes: H04N 2213/003 Aspects relating to the "2D+depth" image format

Abstract

A gesture recognition system includes a candidate node detection unit coupled to receive an input image in order to generate a candidate node; a posture recognition unit configured to recognize a posture according to the candidate node; a multiple hands tracking unit configured to track multiple hands by pairing between successive input images; and a gesture recognition unit configured to obtain motion accumulation amount according to tracking paths from the multiple hands tracking unit, thereby recognizing a gesture.

Claims

1. A gesture recognition system, comprising:

a candidate node detection unit coupled to receive an input image in order to generate a candidate node;

a posture recognition unit configured to recognize a posture according to the candidate node;

a multiple hands tracking unit configured to track multiple hands by pairing between successive input images; and

a gesture recognition unit configured to obtain motion accumulation amount according to tracking paths from the multiple hands tracking unit, thereby recognizing a gesture.

2. The system of claim 1, wherein the candidate node detection unit performs the following steps:

extracting features according to color, depth and motion, thereby generating a color reliability map, a depth reliability map and a motion reliability map, respectively;

determining weightings of the color, depth and motion with respect to operation status; and

combining the color reliability map, the depth reliability map and the motion reliability map with the respective weightings, thereby generating a hybrid reliability map, which provides the candidate node.

3. The system of claim 2, wherein the color reliability map is generated according to skin color of the input image.

4. The system of claim 2, wherein the depth reliability map is generated according to hand depth of the input image.

5. The system of claim 4, wherein a higher value is assigned to a pixel that is within a hand depth range in the depth reliability map.

6. The system of claim 2, wherein the motion reliability map is generated according to motion of a sequence of input images.

7. The system of claim 6, wherein the motion in the motion reliability map is measured by sum of absolute differences (SAD) between two input images.

8. The system of claim 2, wherein the operation status comprises initial statement, motion, whether hand is close to face or combination thereof.

9. The system of claim 1, wherein the posture recognition unit performs the following steps:

segmenting a palm from a hand associated with the candidate node;

generating a distance curve by recording relative distances between a center of the segmented palm and perimeter of the segmented palm, thereby recognizing a posture; and

classifying a plurality of the recognized postures.

10. The system of claim 9, wherein the plurality of the recognized postures are classified according to an amount of recognized unfolding fingers.

11. The system of claim 1, in the multiple hands tracking unit, a tracking path is deleted in case of unmatched track hand due to object leave.

12. The system of claim 1, in the multiple hands tracking unit, an expected track hand is generated by extrapolation technique in case of unmatched track hand due to occlusion.

13. The system of claim 1, in the multiple hands tracking unit, a new tracking path is generated in case of unmatched track hand due to object arrival.

14. The system of claim 1, wherein feedback is fed from the multiple hands tracking unit to the candidate node detection unit in case of unmatched track hands.

15. The system of claim 1, wherein the recognized gesture is fed to a natural user interface for performing a pre-defined task.

16. The system of claim 15, wherein a user draw a line using the recognized gesture according to the natural user interface.