GESTURE RECOGNITION SYSTEM AND GESTURE RECOGNITION METHOD THEREOF

• US 20150177844A1
• Filed: 10/15/2014
• Published: 06/25/2015
• Est. Priority Date: 12/19/2013
• Status: Active Grant

• Current Assignee: Lite-On Technology Corporation
• Original Assignee: Lite-On Technology Corporation
• Inventors: CHEN, JAU-YU
• Granted Patent: US 9,285,887 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0304 Detection arrangements using opto-electronic means constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042
• CPC Class Codes: G06F 3/0325 using a plurality of light emitters or reflectors or a plurality of detectors forming a reference frame from which to derive the orientation of the object, e.g. by triangulation or on the basis of reference deformation in the picked up image
• CPC Class Codes: G06T 2200/04 involving 3D image data
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/30196 Human being; Person
• CPC Class Codes: G06T 7/20 Analysis of motion motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51
• CPC Class Codes: G06T 7/246 using feature-based methods, e.g. the tracking of corners or segments
• CPC Class Codes: G06T 7/521 from laser ranging, e.g. using interferometry; from the projection of structured light
• CPC Class Codes: G06V 10/141 Control of illumination
• CPC Class Codes: G06V 40/28 Recognition of hand or arm movements, e.g. recognition of deaf sign language static hand signs G06V40/113

Abstract

A gesture recognition system is disclosed. The gesture recognition system comprises at least one lighting module, at least one angle detecting module, an image capturing module and a gesture information control module. The lighting module is used for emitting a light. The angle detecting module receives a reflected light of the light and accordingly outputs a light sensing signal. The image capturing module is used for capturing at least one 2-D reference image for at least one hand of the user. The distance between the angle detecting module and the image capturing module is a fixed length. The gesture information control module acquires a plurality of angle data according to the light sensing signal and the 2-D reference image to calculate at least one depth data for at least one hand according to the plurality of angle data and the fixed length, so as to recognize the gestures.

Claims

1. A gesture recognition system, recognizing single-hand gestures, comprising:

a first lighting module for emitting a first light according to a first lighting control signal wherein the first light has a first light illumination range;

an angle detecting module receiving a first reflection light of the first light according to an angle detecting control signal, and outputting a light sensing signal according to the first reflection light, wherein the first reflection light is generated from the reflection resulted when the first light illuminates the single hand;

an image capturing module capturing at least one two-dimensional reference image of the single hand, and outputting the two-dimensional reference image, wherein the distance between the angle detecting module and the image capturing module is a fixed length; and

a gesture information control module electrically connected to the first lighting module, the image capturing module and the angle detecting module, and the gesture information control module acquiring a first angle reference data and a second angle reference data respectively according to the light sensing signal and the two dimension reference image;

wherein the gesture information control module calculates a depth data and a horizontal displacement data of the single hand according to the first angle reference data, the second angle reference data and the fixed length, so as to recognize gestures of the single hand;

wherein the gesture information control module respectively transmits the first lighting control signal and the angle detecting control signal to the first lighting module and the angle detecting module.

2. The gesture recognition system according to claim 1, wherein the angle detecting module has an optical wavelength detecting range that is matching optical wavelength of the first light or a light detecting period that is matching lighting pulse period of the first light.

3. The gesture recognition system according to claim 1, wherein the gesture information control module comprises:

an image control unit electrically connected to the image capturing module, and the image control unit receiving the two dimension reference image and accordingly calculating the second angle reference data;

a first lighting control unit electrically connected to the first lighting module, the first lighting control unit outputting the first lighting control signal according to a first lighting instruction;

an angle control unit electrically connected to the angle detecting module, the angle control unit receiving the light sensing signal and accordingly calculating the first angle reference data, wherein the angle control unit transmits the angle detecting control signal to the angle detecting module according to an angle detecting instruction; and

a gesture recognition processing unit electrically connected to the image control unit, the first lighting control unit and the angle control unit, and the gesture recognition processing unit receiving the first angle reference data and the second angle reference data and calculating a depth data and a horizontal displacement data of the single hand according to the first angle reference data, the second angle reference data and the fixed length, so as to recognize single-hand gestures;

wherein the gesture recognition processing unit respectively transmits the first lighting instruction and the angle detecting instruction to the first lighting control unit and the angle control unit.

4. The gesture recognition system according to claim 1, further comprising:

at least one second lighting module for emitting a second light according to a second lighting control signal so as to expand an overall light illumination range of the gesture recognition system, wherein the second light has a second light illumination range;

wherein the angle detecting module further receives a second reflection light of the second light and outputs the light sensing signal according to the second reflection light, wherein the second reflection light is generated from the reflection resulted when the second light illuminates the single hand;

wherein the gesture information control module transmits the second lighting control signal to the second lighting module.

5. The gesture recognition system according to claim 4, wherein the gesture information control module further comprises:

at least one second lighting control unit electrically connected to the second lighting module and the gesture recognition processing unit, and the second lighting control unit outputting the second lighting control signal according to a second lighting instruction transmitted by the gesture recognition processing unit.

6. The gesture recognition system according to claim 4, wherein the angle detecting module has an optical wavelength detecting range that is matching optical wavelength of the first light and optical wavelength of the second light, or a light detecting period that is matching lighting pulse period of the first light and lighting pulse period of the second light.

7. The gesture recognition system according to claim 4, wherein the first light and the second light have the same optical wavelength, or the first light and the second light have the same lighting pulse period.

8. A gesture recognition system, recognizing two-hands gestures, comprising:

a first lighting module for emitting a first light according to a first lighting control signal, wherein the first light has a first light illumination range;

a third lighting module for emitting a third light according to a third lighting control signal, wherein the third light has a third light illumination range;

a first angle detecting module for receiving a first reflection light of the first light according to an first angle detecting control signal, and outputting a first light sensing signal according to the first reflection light, wherein the first reflection light is generated from the reflection resulted when the first light illuminates a first hand;

a second angle detecting module for receiving a third reflection light of the third light according to an second angle detecting control signal, and outputting a second light sensing signal according to the third reflection light, wherein the third reflection light is generated from the reflection resulted when the third light illuminates a second hand;

an image capturing module capturing at least one two-dimensional reference image of the first hand and the second hand, and outputting the two-dimensional reference image, wherein the distance between the first angle detecting module and the image capturing module is a first fixed length and the distance between the second angle detecting module and the image capturing module is a second fixed length; and

a gesture information control module electrically connected to the first lighting module, the third lighting module, the first angle detecting module, the second angle detecting module and the image capturing module, and the gesture information control module acquiring a first angle reference data and a second angle reference data respectively according to the first light sensing signal and the two dimension reference image and acquiring a third angle reference data and a fourth angle reference data respectively according to the second light sensing signal and the two-dimensional reference image;

wherein the gesture information control module calculates a first depth data and a first horizontal displacement data of the first hand according to the first angle reference data, the second angle reference data and the first fixed length and calculates a second depth data and a second horizontal displacement data of the second hand according to the third angle reference data, the fourth angle reference data and the second fixed length, so as to recognize gestures of the first hand and the second hand;

wherein the gesture information control module respectively transmits the first lighting control signal, the third lighting control signal, the first angle detecting control signal and the second angle detecting control signal to the first lighting module, the third lighting module, the first angle detecting module and the second angle detecting module.

9. The gesture recognition system according to claim 8, wherein the first angle detecting module has an optical wavelength detecting range that is matching optical wavelength of the first light or a light detecting period that is matching lighting pulse period of the first light, and the second angle detecting module has an optical wavelength detecting range that is matching optical wavelength of the third light or a light detecting period that is matching lighting pulse period of the third light.

10. The gesture recognition system according to claim 8, wherein the first light and the third light have different optical wavelengths, or the first light and the third light have different lighting pulse periods.

11. The gesture recognition system according to claim 8, wherein the gesture information control module comprises:

an image control unit electrically connected to the image capturing module, and the image control unit receiving the two-dimensional reference image and accordingly calculating the second angle reference data and the fourth angle reference data;

a first lighting control unit electrically connected to the first lighting module, and the first lighting control unit outputting the first lighting control signal according to a first lighting instruction;

a first angle control unit electrically connected to the first angle detecting module, and the first angle control unit receiving the first light sensing signal and accordingly calculating the first angle reference data, wherein the first angle control unit transmits the first angle detecting control signal to the first angle detecting module according to a first angle detecting instruction;

a third lighting control unit electrically connected to the third lighting module, and the third lighting control unit outputting the third lighting control signal according to a third lighting instruction;

a second angle control unit electrically connected to the second angle detecting module, and the second angle control unit receiving the second light sensing signal and accordingly calculating the third angle reference data, wherein the second angle control unit transmits the second angle detecting control signal to the second angle detecting module according to a second angle detecting instruction; and

a gesture recognition processing unit electrically connected to the image control unit, the first lighting control unit, the third lighting control unit, the first angle control unit and the second angle control unit, and the gesture recognition processing unit receiving the first angle reference data, the second angle reference data, the third angle reference data and the fourth angle reference data;

wherein the gesture recognition processing unit calculates the first depth data and the first horizontal displacement data of the first hand according to the first angle reference data, the second angle reference data and the first fixed length, and calculates the second depth data and the second horizontal displacement data of the second hand according to the third angle reference data, the fourth angle reference data and the second fixed length, so as to recognize gestures of the first hand and the second hand;

wherein the gesture recognition processing unit respectively transmits the first lighting instruction, the third lighting instruction, the first angle detecting instruction and the second angle detecting instruction to the first lighting control unit, the third lighting control unit, the first angle control unit and the second angle control unit.

12. The gesture recognition system according to claim 8, further comprising:

at least one second lighting module for emitting a second light according to a second lighting control signal so as to expand an overall light illumination range of the gesture recognition system, wherein the second light has a second light illumination range; and

at least one fourth lighting module for emitting a fourth light according to a fourth lighting control signal so as to expand an overall light illumination range of the gesture recognition system, wherein the fourth light has a fourth light illumination range;

wherein the first angle detecting module further receives a second reflection light of the second light and outputs the first light sensing signal according to the second reflection light, wherein the second reflection light is generated from the reflection resulted when the second light illuminates the first hand;

wherein the second angle detecting module further receives a fourth reflection light of the fourth light and outputs the second light sensing signal according to the fourth reflection light, wherein the fourth reflection light is generated from the reflection resulted when the fourth light illuminates the second hand;

wherein the gesture information control module transmits the second lighting control signal and the fourth lighting control signal to the second lighting module and the fourth lighting module.

13. The gesture recognition system according to claim 12, wherein the gesture information control module further comprises:

at least one second lighting control unit electrically connected to the second lighting module and the gesture recognition processing unit, and the second lighting control unit outputting the second lighting control signal according to a second lighting instruction transmitted by the gesture recognition processing unit; and

at least one fourth lighting control unit electrically connected to the fourth lighting module and the gesture recognition processing unit, and the fourth lighting control unit outputting the fourth lighting control signal according to a fourth lighting instruction transmitted by the gesture recognition processing unit.

14. The gesture recognition system according to claim 12, wherein the first light and the second light have the same optical wavelength, or the first light and the second light have the same lighting pulse period.

15. The gesture recognition system according to claim 12, wherein the third light and the fourth light have the same optical wavelength, or the third light and the fourth light have the same lighting pulse period.

16. The gesture recognition system according to claim 12, wherein the first angle detecting module has an optical wavelength detecting range that is matching optical wavelength of the first light and optical wavelength of the second light or a light detecting period that is matching lighting pulse period of the first light and lighting pulse period of the second light, and wherein the second angle detecting module has an optical wavelength detecting range that is matching optical wavelength of the third light and optical wavelength of the fourth light or a light detecting period that is matching lighting pulse period of the third light and lighting pulse period of the fourth light.

17. A gesture recognition method, used in a gesture recognition system recognizing single-hand gestures, the gesture recognition system comprising a first lighting module, an angle detecting module, an image capturing module and a gesture information control module, the gesture information control module electrically connected to the first lighting module, the image capturing module and the angle detecting module, and the gesture recognition method comprising:

emitting a first light by the first lighting module according to a first lighting control signal, wherein the first light has a first light illumination range;

receiving a first reflection light of the first light by the angle detecting module according to an angle detecting control signal and outputting a light sensing signal according to the first reflection light, wherein the first reflection light is generated from the reflection resulted when the first light illuminates the single hand;

capturing at least one two-dimensional reference image of the single hand from the image capturing module, wherein the distance between the angle detecting module and the image capturing module is a fixed length;

acquiring a first angle reference data and a second angle reference data by the gesture information control module respectively according to the light sensing signal and the two-dimensional reference image; and

calculating a depth data and a horizontal displacement data of the single hand by the gesture information control module according to the first angle reference data, the second angle reference data and the fixed length, so as to recognize single-hand gestures;

wherein the gesture information control module respectively transmits the first lighting control signal and the angle detecting control signal to the first lighting module and the angle detecting module.

18. The gesture recognition method according to claim 17, wherein the angle detecting module has an optical wavelength detecting range that is matching optical wavelength of the first light, or a light detecting period that is matching lighting pulse period of the first light.

19. The gesture recognition method according to claim 17, wherein the gesture information control module comprises:

an image control unit electrically connected to the image capturing module, and the image control unit receiving the two dimension reference image and accordingly calculating the second angle reference data;

a first lighting control unit electrically connected to the first lighting module, and the first lighting control unit outputting the first lighting control signal according to a first lighting instruction;

an angle control unit electrically connected to the angle detecting module, and the angle control unit receiving the light sensing signal and accordingly calculating the first angle reference data, wherein the angle control unit transmits the angle detecting control signal to the angle detecting module according to an angle detecting instruction; and

a gesture recognition processing unit electrically connected to the image control unit, the first lighting control unit and the angle control unit, and the gesture recognition processing unit receiving the first angle reference data and the second angle reference data and calculating a depth data and a horizontal displacement data of the single hand according to the first angle reference data, the second angle reference data and the fixed length, so as to recognize single-hand gestures;

wherein the gesture recognition processing unit respectively transmits the first lighting instruction and the angle detecting instruction to the first lighting control unit and the angle control unit.

20. The gesture recognition method according to claim 17, wherein the gesture recognition system further comprises:

at least one second lighting module for emitting a second light according to a second lighting control signal so as to expand an overall light illumination range of the gesture recognition system, wherein the second light has a second light illumination range;

wherein the angle detecting module further receives a second reflection light of the second light and outputs the light sensing signal according to the second reflection light, wherein the second reflection light is generated from the reflection resulted when the second light illuminates the single hand;

wherein the gesture information control module transmits the second lighting control signal to the second lighting module.

21. The gesture recognition method according to claim 20, wherein the gesture information control module further comprises:

at least one second lighting control unit electrically connected to the second lighting module and the gesture recognition processing unit, and the second lighting control unit outputting the second lighting control signal according to a second lighting instruction transmitted by the gesture recognition processing unit.

22. The gesture recognition method according to claim 20, wherein the angle detecting module has an optical wavelength detecting range that is matching optical wavelength of the first light and optical wavelength of the second light, or a light detecting period that is matching lighting pulse period of the first light and lighting pulse period of the second light.

23. The gesture recognition method according to claim 20, wherein the first light and the second light have the same optical wavelength, or the first light and the second light have the same lighting pulse period.

24. A gesture recognition method, used in a gesture recognition system for recognizing two-hands gestures, the gesture recognition system comprises a first lighting module, a third lighting module, a first angle detecting module, a second angle detecting module, an image capturing module and a gesture information control module, the gesture information control module electrically connected to the first lighting module, the third lighting module, the first angle detecting module, the second angle detecting module and the image capturing module, and the gesture recognition method comprising:

emitting a first light by the first lighting module according to a first lighting control signal wherein the first light has a first light illumination range;

emitting a third light by third lighting module the according to a third lighting control signal wherein the third light has a third light illumination range;

receiving a first reflection light of the first light by the first angle detecting module according to a first angle detecting control signal, and outputting a first light sensing signal according to the first reflection light, wherein the first reflection light is generated from the reflection resulted when the first light illuminates a first hand;

receiving a third reflection light of the third light by the second angle detecting module according to a second angle detecting control signal, and outputting a second light sensing signal according to the third reflection light, wherein the third reflection light is generated from the reflection resulted when the third light illuminates a second hand;

capturing at least one two-dimensional reference image of the first hand and the second hand by the image capturing module, and outputting the two-dimensional reference image, wherein the distance between the first angle detecting module and the image capturing module is a first fixed length and the distance between the second angle detecting module and the image capturing module is a second fixed length;

acquiring a first angle reference data and a second angle reference data respectively according to the first light sensing signal and the two-dimensional reference image by the gesture information control module, and acquiring a third angle reference data and a fourth angle reference data respectively according to the second light sensing signal and the two-dimensional reference image; and

calculating a first depth data and a first horizontal displacement data of the first hand according to the first angle reference data, the second angle reference data and the first fixed length by the gesture information control module, and calculating a second depth data and a second horizontal displacement data of the second hand according to the third angle reference data, the fourth angle reference data and the second fixed length by the gesture information control module, so as to recognize gestures of the second hand;

wherein the gesture information control module respectively transmits the first lighting control signal, the third lighting control signal, the first angle detecting control signal and the second angle detecting control signal to the first lighting module, the third lighting module, the first angle detecting module and the second angle detecting module.

25. The gesture recognition method according to claim 24, wherein the first angle detecting module has an optical wavelength detecting range that is matching optical wavelength of the first light or a light detecting period that is matching lighting pulse period of the first light, and the second angle detecting module has an optical wavelength detecting range that is matching optical wavelength of the third light or a light detecting period that is matching lighting pulse period of the third light.

26. The gesture recognition method according to claim 24, wherein the first light and the third have different optical wavelengths, or the first light and the third have different lighting pulse periods.

27. The gesture recognition method according to claim 24, wherein the gesture information control module comprises:

an image control unit electrically connected to the image capturing module, and the image control unit receiving the two dimension reference image and accordingly calculating the second angle reference data and the fourth angle reference data;

a first lighting control unit electrically connected to the first angle detecting module, the first lighting control unit outputting the first lighting control signal according to a first lighting instruction;

a first angle control unit electrically connected to the first angle detecting module, and the first angle control unit receiving the first light sensing signal and accordingly calculating the first angle reference data, wherein the first angle control unit transmits the first angle detecting control signal to the first angle detecting module according to a first angle detecting instruction;

a third lighting control unit electrically connected to the third lighting module, and the third lighting control unit outputting the third lighting control signal according to a third lighting instruction;

a second angle control unit electrically connected to the second angle detecting module, and the second angle control unit receiving the second light sensing signal and accordingly calculating the third angle reference data, wherein the second angle control unit transmits the second angle detecting control signal to the second angle detecting module according to a second angle detecting instruction; and

a gesture recognition processing unit electrically connected to the image control unit, the first lighting control unit, the third lighting control unit, the first angle control unit, the second angle control unit and the image control unit, and the gesture recognition processing unit receiving the first angle reference data, the second angle reference data, the third angle reference data and the fourth angle reference data;

wherein the gesture recognition processing unit calculates the first depth data and the first horizontal displacement data of the first hand according to the first angle reference data, the second angle reference data and the first fixed length, and calculates the second depth data and the second horizontal displacement data of the second hand according to the third angle reference data, the fourth angle reference data and the second fixed length, so as to recognize gestures of the first and the second hand;

wherein the gesture recognition processing unit respectively transmits the first lighting instruction, the third lighting instruction, the first angle detecting instruction and the second angle detecting instruction to the first lighting control unit, the third lighting control unit, the first angle control unit and the second control unit.

28. The gesture recognition method according to claim 24, further comprising:

at least one second lighting module for emitting a second light according to a second lighting control signal so as to expand an overall light illumination range of the gesture recognition system, wherein the second light has a second light illumination range; and

at least one fourth lighting module for emitting a fourth light according to a fourth lighting control signal so as to expand an overall light illumination range of the gesture recognition system, wherein the fourth light has a fourth light illumination range;

wherein the first angle detecting module further receives a second reflection light of the second light, and outputs the first light sensing signal according to the second reflection light, where the second reflection light is generated from the reflection resulted when the first light illuminates the first hand;

wherein the second angle detecting module further receives a fourth reflection light of the fourth light, and outputs the second light sensing signal according to the fourth reflection light, wherein the fourth reflection light is generated from the reflection resulted when the first light illuminates the second hand;

wherein the gesture information control module transmits the second lighting control signal and the fourth lighting control signal to the second lighting module and the fourth lighting module.

29. The gesture recognition method according to claim 28, wherein the gesture information control module further comprises:

at least one second lighting control unit electrically connected to the second lighting module and the gesture recognition processing unit, the second lighting control unit outputting the second lighting control signal according to a second lighting instruction transmitted by the gesture recognition processing unit; and

at least one fourth lighting control unit electrically connected to the fourth lighting module and the gesture recognition processing unit, the fourth lighting control unit outputting the fourth lighting control signal according to a fourth lighting instruction transmitted by the gesture recognition processing unit.

30. The gesture recognition method according to claim 28, wherein the first light and the second light have the same optical wavelength, or the first light and the second light have the same lighting pulse period.

31. The gesture recognition method according to claim 28, wherein the third light and the fourth light have the same optical wavelength, or the third light and the fourth light have the same lighting pulse period.

32. The gesture recognition method according to claim 28, wherein the first angle detecting module has an optical wavelength detecting range that is matching optical wavelength of the first light and optical wavelength of the second light or the first angle detecting module has a light detecting period that is matching lighting pulse period of the first light and lighting pulse period of the second light, and wherein the second angle detecting module has an optical wavelength detecting range that is matching optical wavelength of the third light and optical wavelength of the fourth light or the second angle detecting module has a light detecting period that is matching lighting pulse period of the third light and lighting pulse period of the fourth light.