Gesture recognition method and gesture recognition system

• US 10,817,712 B2
• Filed: 08/30/2018
• Issued: 10/27/2020
• Est. Priority Date: 02/04/2018
• Status: Active Grant

• Current Assignee: KaiKuTeK Inc.
• Original Assignee: KaiKuTeK Inc.
• Inventors: Tai, Tsung-Ming, Jhang, Yun-Jie, Hwang, Wen-Jyi, Kuo, Chun-Hsuan
• Primary Examiner(s): Pham, Timothy X
• Application Number:
  US16/118,398
• Publication Number:
  US 20190242974A1
• Time in Patent Office: 789 Days
• CPC Class Codes: G01S 13/50 Systems of measurement based on relative movement of target
• CPC Class Codes: G01S 13/584 adapted for simultaneous range and velocity measurements
• CPC Class Codes: G01S 13/89 for mapping or imaging
• CPC Class Codes: G01S 7/356 involving particularities of FFT processing
• CPC Class Codes: G01S 7/414 Discriminating targets with respect to background clutter
• CPC Class Codes: G01S 7/415 Identification of targets based on measurements of movement associated with the target
• CPC Class Codes: G01S 7/417 involving the use of neural networks
• CPC Class Codes: G06F 17/18 for evaluating statistical data , e.g. average values, frequency distributions, probability functions, regression analysis forecasting specially adapted for a specific administrative, business or logistic context G06Q10/04
• CPC Class Codes: G06F 18/214 Generating training patterns; Bootstrap methods, e.g. bagging or boosting
• CPC Class Codes: G06F 18/2155 characterised by the incorporation of unlabelled data, e.g. multiple instance learning [MIL], semi-supervised techniques using expectation-maximisation [EM] or naïve labelling
• CPC Class Codes: G06F 18/217 Validation; Performance evaluation; Active pattern learning techniques
• CPC Class Codes: G06F 18/22 Matching criteria, e.g. proximity measures
• CPC Class Codes: G06F 18/24 Classification techniques
• CPC Class Codes: G06F 18/24133 Distances to prototypes
• CPC Class Codes: G06F 2218/12 Classification; Matching
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 9/5027 the resource being a machine, e.g. CPUs, Servers, Terminals
• CPC Class Codes: G06N 20/00 Machine learning
• CPC Class Codes: G06N 3/044 Recurrent networks, e.g. Hopfield networks
• CPC Class Codes: G06N 3/045 Combinations of networks
• CPC Class Codes: G06N 3/08 : Learning methods
• CPC Class Codes: G06T 2207/10028 : Range image; Depth image; 3D point clouds
• CPC Class Codes: G06T 2207/20056 : Discrete and fast Fourier transform, [DFT, FFT]
• CPC Class Codes: G06T 2207/20081 : Training; Learning
• CPC Class Codes: G06T 2207/20084 : Artificial neural networks [ANN]
• CPC Class Codes: G06T 7/20 : Analysis of motion motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51
• CPC Class Codes: G06V 40/20 : Movements or behaviour, e.g. gesture recognition recognition of facial expressions G06V40/16
• CPC Class Codes: H03B 21/02 : by plural beating, i.e. for frequency synthesis ; Beating in combination with multiplication or division of frequency digital frequency synthesis using a ROM G06F1/02; digital frequency synthesis in general H03K; indirect frequency synthesis using a PLL H03L7/16

Abstract

A gesture recognition system executes a gesture recognition method. The gesture recognition method includes steps of: receiving a training signal; selecting one of the sensing frames of the sensing signal; generating a sensing map; selecting a cell having the max-amplitude; determining a frame amplitude, a frame phase, and a frame range of the selected one of the sensing frames; setting the frame amplitudes, the frame phases, and the frame ranges of all of the sensing frames to input data of a neural network to classify a gesture event. The present invention just uses a few data to be the input data of the neural network. Therefore, the neural network may not require high computational complexity, the gesture recognition system may decrease the calculation load of the processing unit, and the gesture recognition function may not influence a normal operation of a smart device.

Claims

1. A gesture recognition method, comprising a performing procedure;

wherein the performing procedure is executed by a central processing unit of a smart device, and comprises;

receiving a sensing signal;

wherein the sensing signal comprises a plurality of sensing frames;

wherein the sensing signal is a Range Doppler Image (RDI) signal generated by a Doppler radar;

selecting one of the sensing frames of the sensing signal;

generating a sensing map according to the selected one of the sensing frames;

wherein the sensing map comprises a plurality of chirps, each of the chirps comprises a plurality of cells, and each of the cells has an amplitude and a phase;

selecting the cell having the max-amplitude in each of the chirps as an interested cell;

determining a frame amplitude, a frame phase, and a frame range of the selected one of the sensing frames according to the amplitudes and the phases of the interested cells of the chirps;

determining whether the frame amplitudes, the frame phases, and the frame ranges of all of the sensing frames of the sensing signal are determined;

when the frame amplitudes, the frame phases, and the frame ranges of all of the sensing frames of the sensing signal are determined, setting the frame amplitudes, the frame phases, and the frame ranges of all of the sensing frames of the sensing signal to input data of a neural network to classify a gesture event.

2. The gesture recognition method as claimed in claim 1, wherein the frame amplitude, the frame phase, and the frame range the selected one of the sensing frames are determined by:

calculating an average amplitude by averaging the amplitudes of the interested cells of the chirps;

setting the average amplitude to the frame amplitude of the selected one of the sensing frames;

calculating an average phase by averaging the phases of the interested cells of the chirps;

setting the average phase to the frame phase of the selected one of the sensing frames;

calculating ranges of the interested cells according to the amplitudes of the interested cells;

calculating an average range by averaging ranges of the interested cells of the chirps; and

setting the average range to the frame range of the selected one of the sensing frames.

3. The gesture recognition method as claimed in claim 1, wherein when the amplitude, the phase, and the range of all of the sensing frames of the sensing signal are not determined, selecting another one of the sensing frames of the sensing signal, and generating the sensing map according to the selected one of the sensing frames again.

4. The gesture recognition method as claimed in claim 1, wherein the sensing map is generated by transforming the selected one of the sensing frames using a Fast Fourier Transform.

5. The gesture recognition method as claimed in claim 1, wherein the neural network is a recurrent neural network (RNN).

6. The gesture recognition method as claimed in claim 5, wherein the recurrent neural network is a Long Short Memory (LSTM) network, a Gated Recurrent Unit (GRU), a Simple Gated Unit (SGU), or a Minimal Gated Unit (MGU).

7. A gesture recognition system, comprising a performing module;

wherein the performing module comprises;

a sensing unit, receiving a sensing signal;

wherein the sensing signal comprises a plurality of sensing frames;

wherein the sensing unit is a Doppler radar, and the sensing signal is a Range Doppler Image (RDI) signal generated by the Doppler radar;

a memory unit, storing a neural network;

a processing unit, electrically connected to the sensing unit and the memory unit, and receiving the sensing signal from the sensing unit;

wherein the processing unit selects one of the sensing frames of the sensing signal, and generates a sensing map according to the selected one of the sensing frames;

wherein the sensing map comprises a plurality of chirps, each of the chirps comprises a plurality of cells, and each of the cells has an amplitude and a phase;

wherein the processing unit is a central processing unit of a smart device;

wherein the processing unit selects the cell having the max-amplitude in each of the chirps as an interested cell, and determines a frame amplitude, a frame phase, and a frame range of the selected one of the sensing frames according to the amplitudes and the phases of the interested cells of the chirps;

wherein the processing unit further determines whether the frame amplitudes, the frame phases, and the frame ranges of all of the sensing frames of the sensing signal are determined;

wherein when the frame amplitudes, the frame phases, and the frame ranges of all of the sensing frames of the sensing signal are determined, the processing unit loads the neural network from the memory, and sets the frame amplitudes, the frame phases, and the frame ranges of all of the sensing frames of the sensing signal to input data of the neural network to classify a gesture event.

8. The gesture recognition system as claimed in claim 7, wherein the processing unit determines the frame amplitude, the frame phase, and the frame range of the selected one of the sensing frames by steps of:

calculating an average amplitude by averaging the amplitudes of the interested cells of the chirps;

setting the average amplitude to the frame amplitude of the selected one of the sensing frames;

calculating an average phase by averaging the phases of the interested cells of the chirps;

setting the average phase to the frame phase of the selected one of the sensing frames;

calculating ranges of the interested cells according to the amplitudes of the interested cells;

calculating an average range by averaging ranges of the interested cells of the chirps;

setting the average range to the frame range of the selected one of the sensing frames.

9. The gesture recognition system as claimed in claim 7, wherein when the amplitude, the phase, and the range of all of the sensing frames of the sensing signal are not determined, the processing unit selects another one of the sensing frames of the sensing signal, and generates the sensing map according to the selected one of the sensing frames again.

10. The gesture recognition system as claimed in claim 7, wherein the processing unit transforms the selected one of the sensing frames by a Fast Fourier Transform to generate the sensing map.

11. The gesture recognition system as claimed in claim 7, wherein the neural network is a recurrent neural network (RNN).

12. The gesture recognition system as claimed in claim 11, wherein the recurrent neural network is a Long Short Memory (LSTM) network, a Gated Recurrent Unit (GRU), a Simple Gated Unit (SGU), or a Minimal Gated Unit (MGU).