Gesture recognition method and apparatus

• US 10,732,724 B2
• Filed: 04/20/2018
• Issued: 08/04/2020
• Est. Priority Date: 10/21/2015
• Status: Active Grant

• Current Assignee: Huawei Technologies Co., Ltd. (Huawei Investment & Holding Co., Ltd.)
• Original Assignee: Huawei Technologies Co., Ltd. (Huawei Investment & Holding Co., Ltd.)
• Inventors: Xue, Jiantao, Li, Anjian, Su, Bin
• Primary Examiner(s): McLoone, Peter D
• Application Number:
  US15/958,380
• Publication Number:
  US 20180239438A1
• Time in Patent Office: 837 Days
• CPC Class Codes: G06F 18/214 Generating training patterns; Bootstrap methods, e.g. bagging or boosting
• CPC Class Codes: G06F 2218/00 Aspects of pattern recognition specially adapted for signal processing
• CPC Class Codes: G06F 3/01 Input arrangements or combined input and output arrangements for interaction between user and computer G06F3/16 takes precedence
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/04883 for inputting data by handwriting, e.g. gesture or text
• CPC Class Codes: G06V 40/20 Movements or behaviour, e.g. gesture recognition recognition of facial expressions G06V40/16
• CPC Class Codes: H04L 5/0048 Allocation of pilot signals, i.e. of signals known to the receiver
• CPC Class Codes: H04W 72/0453 Resources in frequency domain, e.g. a carrier in FDMA

Abstract

The method is applied to a terminal device, the terminal device stores training sample data, and the training sample data includes a correlation spectrum characteristic vector of a reflected signal of a sounding reference signal (SRS) and a corresponding user gesture identifier. The method includes: transmitting, by the terminal device, a first sounding reference signal (SRS); receiving, by the terminal device, a multipath reflected signal that is of the first sounding reference signal (SRS) and that is reflected from a gesture-making object, and performing a related operation on the reflected signal and a signal template corresponding to the first sounding reference signal (SRS), to obtain a correlation spectrum characteristic vector of the reflected signal; and matching, by the terminal device, the correlation spectrum characteristic vector of the reflected signal against the training sample data, to recognize an input gesture.

Claims

1. A gesture recognition method, applied to a terminal device for transmitting a mobile communications network signal, wherein the method comprises:

transmitting, by the terminal device, a first sounding reference signal (SRS) and a second SRS to a same gesture-making object, wherein the terminal device stores training sample data that comprises correlation spectrum characteristic vectors of reflected signals of SRSs and, gesture identifiers corresponding to the correlation spectrum characteristic vectors;

receiving, by the terminal device, a first multipath reflected signal that is of the first SRS and that is reflected from the gesture-making object, and performing a first related operation on the first multipath reflected signal and a signal template corresponding to the first SRS, to obtain a first correlation spectrum characteristic vector of the first multipath reflected signal;

matching, by the terminal device, the first correlation spectrum characteristic vector against the training sample data, to recognize a gesture input by the gesture-making object;

generating a second correlation spectrum characteristics vector of the second SRS by a second related operation, wherein the second related operations is performed on a second multipath reflected signal from the gesture-making object and a signal template corresponding to the second SRS;

adding the second correlation spectrum characteristics vector and a corresponding gesture identifier to the training sample data stored in the terminal device.

2. The method according to claim 1, wherein

the training sample data is pre-stored in the terminal device;

or

the training sample data is obtained from a network device by the terminal device.

3. The method according to claim 2, further comprising:

sending the second correlation spectrum characteristics vector and the corresponding gesture identifier to a network device.

4. The method according to claim 1, wherein

the first related operation comprises:

filtering out a reflected signal, wherein a distance between the reflected signal and the terminal device exceeds a specified distance when reflection occurs.

5. The method according to claim 1, wherein the method further comprises:

determining, by the terminal device according to an SRS set pre-stored in the terminal device, an SRS used for gesture recognition;

or

receiving, by the terminal device, information that is about an SRS set and that is sent by the network device; and

determining, by the terminal device according to the received information about the SRS set, an SRS used for gesture recognition.

6. The method according to claim 5, wherein

the SRS set comprises a parameter value section corresponding to at least one SRS, and the parameter value section comprises any one or more of the following:

a transmit interval, a transmit bandwidth, a transmit carrier frequency, or a transmit power of the SRS;

or

the SRS set comprises a parameter value corresponding to at least one SRS, and the parameter value belongs to a parameter value section.

7. The method according to claim 6, wherein

the parameter value section corresponding to the SRS comprises any one or more of the following:

the transmit interval is less than or equal to 10 milliseconds;

the transmit bandwidth is greater than or equal to 100 MHz;

the transmit carrier frequency is greater than or equal to 10 GHz;

or the transmit power is less than or equal to 0 dBm.

8. A terminal device, applied to a mobile communications system, wherein the terminal device comprises

a storage;

a transmitter;

a receiver; and

a processor;

wherein the storage is configured to store a database, wherein the database stores training sample data that comprises correlation spectrum characteristic vectors of reflected signal of sounding reference signals (SRSs) and gesture identifiers corresponding to the correlation spectrum characteristic vectors;

wherein the transmitter is configured to transmit a first SRS and a second SRS to same gesture-making object, wherein the first SRS is determined by the processor and that is used for gesture recognition, and wherein the second SRS is determined by the processor and is used for training data collection;

wherein the receiver is configured to receive a first multipath reflected signal that is of the first SRS and that is reflected from the gesture-making object;

wherein the processor is configured to;

determine the first SRS used for gesture recognition, and perform a first related operation on the first multipath reflected signal received by the receiver and a signal template corresponding to the first SRS, to obtain a first correlation spectrum characteristic vector of the first multipath reflected signal;

match the first correlation spectrum characteristic vector against the training sample data in the database, to recognize a gesture input by the gesture-making object;

generate a second correlation spectrum characteristics vector of the second SRS by a second related operation, wherein the second related operations is performed on a second multipath reflected signal from the gesture-making object and a signal template corresponding to the second SRS; and

adding the second correlation spectrum characteristics vector and a corresponding gesture identifier to the training sample data stored in the terminal device.

9. The terminal device according to claim 8, wherein

the training sample data is pre-stored in the storage;

or

the training sample data is obtained from a network device by the terminal device by using the receiver, and is stored in the database of the storage.

10. The terminal device according to claim 9, wherein

the transmitter further sends the training sample data to the network device.

11. The terminal device according to claim 8, wherein the first related operation comprises:

filtering out a reflected signal, wherein a distance between the reflected signal and the terminal device exceeds a specified distance when reflection occurs.

12. The terminal device according to claim 8, wherein

the storage pre-stores an SRS set;

and

the processor determines, according to the SRS set, an SRS used for gesture recognition;

or

the receiver receives information that is about an SRS set and that is sent by the network device; and

the processor determines, according to the information that is about the SRS set and that is received by the receiver, an SRS used for gesture recognition.

13. The terminal device according to claim 12, wherein

the SRS set comprises a parameter value section corresponding to at least one SRS, and the parameter value section comprises any one or more of the following:

a transmit interval, a transmit bandwidth, a transmit carrier frequency, or a transmit power of the SRS;

or

the SRS set comprises a parameter value corresponding to at least one SRS, and the parameter value belongs to a parameter value section.

14. The terminal device according to claim 13, wherein

the parameter value section corresponding to the SRS comprises any one or more of the following:

the transmit interval is less than or equal to 10 milliseconds;

the transmit bandwidth is greater than or equal to 100 MHz;

the transmit carrier frequency is greater than or equal to 10 GHz;

or the transmit power is less than or equal to 0 dBm.