Wide-field radar-based gesture recognition

• US 10,139,916 B2
• Filed: 04/29/2016
• Issued: 11/27/2018
• Est. Priority Date: 04/30/2015
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google LLC (Alphabet Inc.)
• Inventors: Poupyrev, Ivan
• Primary Examiner(s): Lee, Nicholas
• Assistant Examiner(s): Pham Lu, Ngan T
• Application Number:
  US15/142,619
• Publication Number:
  US 20160320852A1
• Time in Patent Office: 942 Days
• Field of Search: 345156
• CPC Class Codes: G01S 13/88 Radar or analogous systems specially adapted for specific applications electromagnetic prospecting or detecting of objects, e.g. near-field detection, G01V3/00
• CPC Class Codes: G01S 7/415 Identification of targets based on measurements of movement associated with the target
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0325 using a plurality of light emitters or reflectors or a plurality of detectors forming a reference frame from which to derive the orientation of the object, e.g. by triangulation or on the basis of reference deformation in the picked up image
• CPC Class Codes: G06V 40/28 Recognition of hand or arm movements, e.g. recognition of deaf sign language static hand signs G06V40/113

Abstract

This document describes techniques using, and devices embodying, wide-field radar-based gesture recognition. These techniques and devices can enable a great breadth of gestures and uses for those gestures, such as gestures to use, control, and interact with computing and non-computing devices, from software applications to refrigerators.

Claims

1. A computer-implemented method comprising:

providing, by an emitter of a radar system, a radar field, the radar field comprising a contiguous radar field;

receiving, at a receiver of the radar system, a first set of reflection signals caused by a first interaction of a first human action performing a gesture within the contiguous radar field, the first set of reflection signals representing a first time period during which the first human action interacts with the contiguous radar field;

receiving, at the receiver of the radar system, a second set of reflection signals caused by a second interaction of a second human action performing the gesture within the contiguous radar field, the second set of reflection signals representing a second time period during which the second human action interacts with the contiguous radar field;

analyzing the first and second sets of reflection signals to determine a signal element common to the first and second sets of reflection signals, the analyzing comprising breaking the first and second sets of reflection signals into many signal elements and determining which of the many signal elements corresponds to both the first interaction of the first human action performing the gesture and the second interaction of the second human action performing the gesture, the determined signal element corresponding to the gesture; and

associating the determined signal element with the gesture effective to enable a later received reflection signal including the determined signal element caused by an interaction by a different human action to be associated with the gesture.

2. The computer-implemented method as described in claim 1, wherein analyzing the first and second sets of reflection signals is not based on tracking points or elements of the first or second human action.

3. The computer-implemented method as described in claim 1, wherein the radar system is not a phased antenna array.

4. The computer-implemented method as described in claim 1, further comprising receiving multiple other sets of reflection signals caused by multiple other interactions of the first, second, or other human actions performing the gesture within the contiguous radar field and wherein analyzing the first and second sets of reflection signals analyzes the first set of reflection signals, the second set of reflection signals, and the multiple other sets of reflection signals.

5. The computer-implemented method as described in claim 4, wherein the contiguous radar field in which the first, second, and other sets of reflection signals is received is a same contiguous radar field provided by a same emitter of the same radar system.

6. The computer-implemented method as described in claim 1, further comprising refining the signal element associated with the gesture based on other signal elements associated with other gestures.

7. The computer-implemented method as described in claim 1, wherein the contiguous radar field comprises continuous wave radar or pulsed radar.

8. The computer-implemented method as described in claim 1 wherein breaking the first and second sets of reflection signals into many signal elements comprises digitally breaking up the reflection signal and analyzing, one at a time, each of the broken-up signals.

9. The computer-implemented method as described in claim 1 wherein the first time period occurs non-simultaneously with or prior to the second time period.

10. A computer-implemented method comprising:

providing a wide contiguous radar field;

receiving a reflection signal for a gesture made within the wide contiguous radar field;

determining signal elements of the reflection signal, the determining comprising breaking the reflection signal into the signal elements, the determined signal elements corresponding to the gesture;

comparing the determined signal elements of the reflection signal to known signal elements associated with known gestures;

determining, based on the determined signal elements of the reflection signal corresponding to the known signal elements, that the gesture made in the wide contiguous radar field is one of the known gestures; and

passing the one of the known gestures to an application or operating system.

11. The method of claim 10, further comprising:

receiving a second reflection signal for a second gesture;

determining second signal elements of the second reflection signal;

comparing the second signal elements of the second reflection signal to the known signal elements associated with the known gestures;

determining, based on the second signal elements not corresponding to the known signal elements, that the second gesture was not properly recognized; and

altering the known signal elements for the gesture based on the determined second signal elements.

12. The method of claim 10, further comprising:

receiving, from a user, multiple reflection signals from multiple iterations of a unique gesture made within the wide contiguous radar field, the unique gesture not being one of the known gestures;

determining signal elements of the multiple reflection signals, the signal elements sufficient to determine that a later-received reflection signal for a later-received gesture of the user matches the unique gesture; and

associating the determined signal elements to the unique gesture.

13. The method of claim 12, further comprising associating the unique gesture to a control of an application, the associating responsive to an assignment of the control and the application selected by the user.

14. The method of claim 10, wherein breaking the reflection signal into the signal elements comprises digitally breaking up the reflection signal and analyzing, one at a time, each of the broken-up signals.

15. The method of claim 10, wherein the wide contiguous radar field includes multiple contiguous radar fields from different directions or multiple contiguous radar fields having different frequencies or phases.

16. The method of claim 10, wherein the wide contiguous radar field is a continuous-wave radar field.

17. The method of claim 16, wherein the continuous-wave radar field is a single tone, linear frequency modulated (FM), or stepped FM field.

18. The method of claim 10, wherein the wide contiguous radar field is a pulsed-wave radar field, an impulse radar field, or a chirped radar field.

19. An apparatus comprising:

a wide-field radar-based gesture-recognition system comprising;

a radar-emitting element configured to provide a wide contiguous radar field; and

an antenna element configured to;

receive reflection signals from human tissue that is within the wide contiguous radar field; and

pass the received reflection signals; and

one or more computer-readable storage media having instructions stored thereon that, responsive to execution by the one or more computer processors, perform operations comprising;

causing the wide-field radar-based gesture-recognition system to provide the wide contiguous radar field with the radar-emitting element;

causing the wide-field radar-based gesture-recognition system to receive reflection signals for an interaction in the wide contiguous radar field with the antenna element;

breaking the received reflection signals into signal elements;

determining the signal elements corresponding to the interaction;

comparing the determined signal elements of the received reflection signals to known signal elements associated with known gestures;

determining, based on the determined signal elements corresponding to the known signal elements, that the interaction made in the wide contiguous radar field corresponds to one of the known gestures; and

passing the one of the known gestures to an application or operating system.

20. The apparatus of claim 19, wherein the one or more computer-readable storage media is further configured to perform operations comprising:

receiving multiple reflection signals from multiple iterations of a complex gesture made within the wide contiguous radar field, the complex gesture not being one of the known gestures;

determining signal elements of the multiple reflection signals, the signal elements sufficient to determine that a later-received reflection signal for a later-received gesture matches the complex gesture; and

associating the determined signal elements to the complex gesture.