Type-agnostic RF signal representations

• US 10,310,620 B2
• Filed: 04/29/2016
• Issued: 06/04/2019
• Est. Priority Date: 04/30/2015
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.), Board of Trustees of the Leland Stanford Junior University (Stanford Management Co.)
• Original Assignee: Google LLC (Alphabet Inc.), Board of Trustees of the Leland Stanford Junior University (Stanford Management Co.)
• Inventors: Lien, Jaime, Amihood, Patrick M., Poupyrev, Ivan
• Primary Examiner(s): Bythrow, Peter M
• Application Number:
  US15/142,829
• Publication Number:
  US 20160320854A1
• Time in Patent Office: 1,131 Days
• CPC Class Codes: G01S 13/08 Systems for measuring distance only indirect measurement G01S13/46
• CPC Class Codes: G01S 13/58 Velocity or trajectory determination systems; Sense-of-movement determination systems
• CPC Class Codes: G01S 13/88 Radar or analogous systems specially adapted for specific applications electromagnetic prospecting or detecting of objects, e.g. near-field detection, G01V3/00
• CPC Class Codes: G01S 7/292 Extracting wanted echo-signals
• CPC Class Codes: G01S 7/354 Extracting wanted echo-signals Doppler systems G01S13/50
• CPC Class Codes: G01S 7/415 Identification of targets based on measurements of movement associated with the target
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883

Abstract

This document describes techniques and devices for type-agnostic radio frequency (RF) signal representations. These techniques and devices enable use of multiple different types of radar systems and fields through type-agnostic RF signal representations. By so doing, recognition and application-layer analysis can be independent of various radar parameters that differ between different radar systems and fields.

Claims

1. One or more non-transitory computer-readable storage media having instructions stored thereon that, responsive to execution by one or more computer processors, implement an abstraction module and a recognition module:

the abstraction module configured to;

receive different types of type-specific raw data, each of the different types of type-specific raw data representing a reflection signal made by an object moving within a different type of type-specific radar field;

model the object as a set of scattering centers, based on the different types of type-specific raw data, the model comprising a superposition of each of the set of the scattering centers at a location of an element of the object; and

transform each of the different types of type-specific raw data of the model of the object into a type-agnostic signal representation; and

the recognition module configured to;

receive the type-agnostic signal representations; and

determine, for each of the type-agnostic signal representations and based on the type-agnostic signal representations, a gesture or action performed by the object within the respective different type of type-specific radar field.

2. The non-transitory computer-readable storage media of claim 1, wherein the abstraction module is further configured to determine a transient or late-time electromagnetic (EM) response based on the set of scattering centers.

3. The non-transitory computer-readable storage media of claim 1, wherein the abstraction module includes a raw signal processor, the raw signal processor configured to provide a complex signal based on the type-specific raw data, the complex signal having amplitude and phase information from which a phase of the type-specific raw data can be extracted and unwrapped.

4. The non-transitory computer-readable storage media of claim 1, wherein the abstraction module includes a signal transformer, the signal transformer configured to determine a range-Doppler-time profile, range-time profile, micro-Doppler profile, or a fast-time spectrogram for the type-specific raw data or a complex signal determined from the type-specific raw data.

5. The non-transitory computer-readable storage media of claim 1, wherein the recognition module includes a feature extractor, the feature extractor configured to extract type-agnostic features.

6. The non-transitory computer-readable storage media of claim 5, wherein the extracted type-agnostic features of which the feature extractor is configured to determine include signal transformations, engineered features, computer-vision features, machine-learned features, or inferred target features.

7. The non-transitory computer-readable storage media of claim 1, wherein the recognition module includes a gesture recognizer, the gesture recognizer configured to determine the gesture or action through determination of a gesture classification, motion parameter tracking, regression estimate, or gesture probability.

8. The non-transitory computer-readable storage media of claim 1, wherein the object is a human hand and the reflection signal is caused by an interaction of the human hand moving within the type-specific radar field.

19. The non-transitory computer-readable storage media of claim 1, wherein the element of the object comprises a fingertip, a knuckle, a joint, or a contact location of a finger or a thumb of a hand.

20. The non-transitory computer-readable storage media of claim 1, wherein each of the set of scattering centers has a reflectivity that is dependent on a shape, size, aspect, or material of the element of the object.

21. The non-transitory computer-readable storage media of claim 1, wherein the model comprising the superposition of each of the set of the scattering centers comprises one scattering center.

22. The non-transitory computer-readable storage media of claim 1, wherein the model comprising the superposition of each of the set of the scattering centers comprises two or more scattering centers.

9. A computer-implemented method comprising:

receiving different types of type-specific raw data representing two or more different reflection signals, the two or more different reflection signals each reflected from an object moving in a different radar field, the different radar fields provided through different modulation schemes or different types of hardware radar-emitting elements;

modeling the object as a set of scattering centers, based on the different types of type-specific raw data, the model comprising a superposition of each of the set of the scattering centers at a location of an element of the object;

transforming the different types of type-specific raw data of the model of the object into two or more type-agnostic signal representations;

determining, for each of the two or more type-agnostic signal representations, a gesture or action performed by the object within the respective different radar fields; and

passing each the determined gestures or actions to an application effective to control or alter a display, function, or capability associated with the application.

10. The method of claim 9, wherein the different radar fields are provided through different modulation schemes using a same type of hardware radar-emitting element.

11. The method of claim 9, wherein the object moving in each of the different radar fields is a same object making a same movement and wherein determining the gesture or action of the movement of the object is based on both of the two or more type-agnostic signal representations.

12. The method of claim 9, wherein each of the two or more type-agnostic signal representations is independent of the different radar field in which the movement is made by the object, respectively.

13. The method of claim 9, wherein the different radar fields have different modulation, frequency, amplitude, or phase, and the two or more type-agnostic signal representations are independent of modulation, frequency, amplitude, or phase.

14. The method of claim 9, wherein the different types of type-specific raw data are specific to a first type of radar system or a second type of radar system, the first type and the second type each being different ones of the following types of radar systems:

single tone, stepped frequency modulated, linear frequency modulated, impulse, or chirped.

23. The method of claim 9, wherein the model comprising the superposition of each of the set of the scattering centers comprises one scattering center.

24. The method of claim 9, wherein the model comprising the superposition of each of the set of the scattering centers comprises two or more scattering centers.

15. An apparatus comprising:

one or more computer processors;

one or more type-specific radar systems configured to provide two or more different radar fields, the two or more different radar fields provided though two or more modulation schemes or two or more different types of hardware radar-emitting elements, the one or more type-specific radar systems comprising;

one or more radar-emitting elements configured to provide the two or more different radar fields; and

one or more antenna elements configured to receive two or more different reflection signals, each of the reflection signals reflected from an object moving in one of the two or more different radar fields; and

one or more computer-readable storage media having instructions stored thereon that, responsive to execution by one or more computer processors, implement an abstraction module and a recognition module;

the abstraction module configured to;

receive different types of type-specific raw data representing the two or more different reflection signals;

model the object as a set of scattering centers, based on the different types of type-specific raw data, the model comprising a superposition of each of the set of the scattering centers at a location of an element of the object; and

transform the different types of type-specific raw data of the model of the object into two or more type-agnostic signal representations; and

the recognition module configured to;

receive the two or more of the type-agnostic signal representations; and

determine, for each of the two or more type-agnostic signal representations, a gesture or action of the object within the respective one of the two or more different radar fields; and

pass each the determined gestures or actions to an application executing on the apparatus effective to control or alter a display, function, or capability of the apparatus or associated with the apparatus.

16. The apparatus of claim 15, wherein the one or more type-specific radar systems is two or more type-specific radar systems, each of the two or more type-specific radar systems having a respective one of the two or more different types of hardware radar-emitting elements.

17. The apparatus of claim 15, wherein the apparatus is a mobile computing device having an integral display and one of the determined gestures or actions is a gesture controlling a user interface associated with the application and presented on the display.

18. The apparatus of claim 15, wherein the abstraction module includes a signal transformer, the signal transformer configured to determine, for each of the two or more different reflection signals, a range-Doppler profile, a range profile, a micro-Doppler profile, or a fast-time spectrogram, the determination enabling recognition of the gesture or action independent of parameters of the two or more different radar fields.

25. The apparatus of claim 15, wherein the model comprising the superposition of each of the set of the scattering centers comprises one scattering center.

26. The apparatus of claim 15, wherein the model comprising the superposition of each of the set of the scattering centers comprises two or more scattering centers.