Gesture recognition system and gesture recognition method

• US 10,061,389 B2
• Filed: 06/02/2015
• Issued: 08/28/2018
• Est. Priority Date: 06/03/2014
• Status: Active Grant

• Current Assignee: Beijing Transborder Information Technologies Co., Ltd.
• Original Assignee: Beijing Transborder Information Technologies Co., Ltd.
• Inventors: Jiang, Xiaodan, Liu, Qin, He, Yongzhen
• Primary Examiner(s): Chowdhury, Afroza
• Application Number:
  US14/728,113
• Publication Number:
  US 20150346833A1
• Time in Patent Office: 1,183 Days
• CPC Class Codes: G06F 3/014 Hand-worn input/output arrangements, e.g. data gloves
• CPC Class Codes: G06F 3/015 Input arrangements based on nervous system activity detection, e.g. brain waves [EEG] detection, electromyograms [EMG] detection, electrodermal response detection
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883

Abstract

A gesture recognition system comprises: a signal collection terminal, configured to collect and preprocess gesture data of a gesture object; a local recognition device, configured to extract features from the gesture data received from the signal collection terminal, form a multi-dimensional feature vector based on extracted features, establish local gesture models based on the multi-dimensional feature vector and perform local gesture recognition according to the local gesture models; and a cloud server, configured to receive the multi-dimensional feature vector from the local recognition device when there is a network connection between the local recognition device and the cloud server, establish cloud gesture models based on the received multi-dimensional feature vector and perform cloud gesture recognition according to the cloud gesture models. A gesture recognition method is also disclosed.

Claims

1. A gesture recognition system, comprising:

a signal collection terminal, configured to collect and preprocess gesture data of a gesture object;

a local recognition device, configured to extract features from the gesture data received from the signal collection terminal, form a multi-dimensional feature vector based on extracted features, establish local gesture models based on the multi-dimensional feature vector and perform local gesture recognition according to the local gesture models; and

a cloud server, configured to receive the multi-dimensional feature vector from the local recognition device when there is a network connection between the local recognition device and the cloud server, establish cloud gesture models based on the received multi-dimensional feature vector, and perform cloud gesture recognition according to the cloud gesture models;

wherein the cloud server is further configured to transmit the cloud gesture models to the local recognition device, and the local recognition device is further configured to update the local gesture models with the cloud gesture models transmitted by the cloud server.

2. The gesture recognition system according to claim 1, wherein the signal collection terminal comprises:

a collection module, configured to collect the gesture data of the gesture object;

a preprocessing module, configured to perform noise deduction preprocessing for the gesture data; and

a communication module, configured to transmit the preprocessed gesture data to the local recognition device.

3. The gesture recognition system according to claim 2, wherein the collection module comprises a plurality of bioelectrical electrodes and a plurality of acceleration sensors, wherein the gesture data includes bioelectrical signals sensed from the plurality of bioelectrical electrodes and space motion signals sensed from the plurality of acceleration sensors.

4. The gesture recognition system according to claim 3, wherein the bioelectrical signals include myoelectricity signals and skin impedance signals, the space motion signals include motion acceleration signals and space rotation signals.

5. The gesture recognition system according to claim 2, wherein the communication module transmits the preprocessed gesture data through a wireless protocol.

6. The gesture recognition system according to claim 2, wherein the preprocessing module comprises a band-pass filter unit and/or a band-stop filter unit.

7. The gesture recognition system according to claim 1, wherein the local recognition device comprises:

a segmentation module, configured to receive the gesture data from the signal collection terminal, identify a beginning point and an ending point of each gesture so as to obtain independent gesture signal segments;

a feature extraction module, configured to perform feature extraction for each gesture signal segment, and combine the feature data extracted from different gesture signal segments so as to obtain a multi-dimensional feature vector;

a local data set module, configured to store the multi-dimensional feature vector and upload the multi-dimensional feature vector to the cloud server when there is a network connection between the local recognition module and the cloud server;

a local classification model module, configured to establish local gesture models based on the multi-dimensional feature vector and update the local gesture models with the transmitted cloud gesture models; and

a local gesture recognition module, configured to perform real-time recognition for the multi-dimensional feature vector according to the local gesture models, so as to obtain a local gesture recognition result.

8. The gesture recognition system according to claim 7, wherein the local data set module is further configured to delete the multi-dimensional feature vector after uploading the multi-dimensional feature vector to the cloud server.

9. The gesture recognition system according to claim 7, wherein the segmentation module is configured to obtain gesture original signals by baseline detection, smooth filtering, zero crossing detection and calculation of area under a waveform curve for the preprocessed data, and normalize the obtained gesture original signals, convert absolute magnitudes of the gesture original signals to relative magnitudes between 0 and 1, compare the relative magnitudes with predetermined energy thresholds so as to identify the beginning point and the ending point of each gesture.

10. The gesture recognition system according to claim 9, wherein the segmentation module is further configured to update the energy thresholds with energy threshold sent from the cloud server.

11. The gesture recognition system according to claim 7, wherein the cloud server comprises:

a cloud data set module, configured to receive the multi-dimensional feature vectors uploaded from one or more local data set modules of one or more local recognition devices, and form a massive data set;

a cloud classification model module, configured to establish cloud gesture models using the massive data set and transmit the cloud gesture models to the local classification model module; and

a cloud gesture recognition module, configured to perform recognition for the multi-dimensional feature vector received from the local recognition device according to the cloud gesture models, so as to obtain a cloud gesture recognition result.

12. The gesture recognition system according to claim 11, wherein the cloud classification model module is further configured to cut the cloud gesture models before transmitting the cloud gesture models to the local classification model module.

13. The gesture recognition system according to claim 11, wherein the local gesture recognition module is further configured to receive the cloud gesture recognition result from the cloud gesture recognition module, and obtain a final gesture recognition result based on combination of the local gesture recognition result and the cloud gesture recognition result.

14. The gesture recognition system according to claim 1, wherein the signal collection terminal is a wristband, wherein the wristband comprises two parallel annular members, the first annular member is continuous and is used as a reference electrode, the second annular member includes a plurality of sensing units and a power supply unit.

15. The gesture recognition system according to claim 14, wherein the plurality of sensing units comprise a plurality of bioelectrical electrodes configured to sense bioelectrical signals at different positions on skin of a forearm and a plurality of acceleration sensor configured to sense space motion signals of a hand.

16. A gesture recognition method, comprising:

collecting and preprocessing gesture data of a gesture object;

extracting features from the gesture data and forming a multi-dimensional feature vector based on extracted features;

establishing local gesture models based on the multi-dimensional feature vector and performing local gesture recognition according to the local gesture models;

establishing a cloud gesture models based on the multi-dimensional feature vector and performing cloud gesture recognition according to the cloud gesture models when there is a network connection; and

updating the local gesture models with the cloud gesture models.

17. The gesture recognition method according to claim 16, further comprising obtaining a final gesture recognition result based on combination of the result of the step of performing local gesture recognition and result of the step of performing cloud gesture recognition when there is the network connection.

18. The gesture recognition method according to claim 16, wherein the gesture data includes bioelectrical signals and space motion signals.