RF-Based Micro-Motion Tracking for Gesture Tracking and Recognition

• US 20160320853A1
• Filed: 04/29/2016
• Published: 11/03/2016
• Est. Priority Date: 04/30/2015
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.), Board of Trustees of the Leland Stanford Junior University (Stanford Management Co.)
• Original Assignee: Google Inc. (Alphabet Inc.), Board of Trustees of the Leland Stanford Junior University (Stanford Management Co.)
• Inventors: Olson, Erik M., Lien, Jaime, Amihood, Patrick M., Poupyrev, Ivan
• Granted Patent: US 10,241,581 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G01S 13/58 Velocity or trajectory determination systems; Sense-of-movement determination systems
• CPC Class Codes: G01S 13/66 Radar-tracking systems; Analogous systems
• CPC Class Codes: G01S 13/88 Radar or analogous systems specially adapted for specific applications electromagnetic prospecting or detecting of objects, e.g. near-field detection, G01V3/00
• CPC Class Codes: G01S 13/89 for mapping or imaging
• CPC Class Codes: G01S 7/415 Identification of targets based on measurements of movement associated with the target
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883

Abstract

This document describes techniques for radio frequency (RF) based micro-motion tracking. These techniques enable even millimeter-scale hand motions to be tracked. To do so, radar signals are used from radar systems that, with conventional techniques, would only permit resolutions of a centimeter or more.

Claims

1. A computer-implemented method comprising:

receiving a radar signal representing a superposition of reflections of two or more points of a hand within a radar field provided by a radar system, a first of the two or more points having a first movement and a second of the two or more points having a second movement;

determining a velocity profile of the radar signal, the velocity profile indicating energies at multiple velocities, a first of the energies representing a reflection of the first point having the first movement and a second of the energies representing a reflection of the second point having the second movement;

extracting, from the velocity profile and one or more prior-determined or later-determined velocity profiles over time, relative velocities between the first and second points of the hand; and

determining a displacement trajectory by integrating the relative velocities, the displacement trajectory representing a relative displacement between the first and second points of the hand.

2. The method of claim 1, wherein determining the velocity profile of the radar signal representing the superposition determines a first and a second micro-Doppler centroid for the first and the second points, respectively.

3. The method of claim 2, further comprising determining the first and second points from the one or more points based on the energies associated with the first and second micro-Doppler centroids being higher than other energies associated with other points of the one or more points.

4. The method of claim 1, further comprising increasing a resolution of the displacement trajectory by extracting millimeter or sub-millimeter displacements using a phase change of the radar signal.

5. The method of claim 1, further comprising filtering the radar signal prior to determining the velocity profile.

6. The method of claim 5, wherein filtering the radar signal includes isolating the first point from the second point using a moving target indicator (MTI) filter to determine points of the two or more points having greater movement.

7. The method of claim 1, wherein the first point is a first finger and the second point is a second finger, and the relative displacement between the first and second points during the first movement and the second movement indicate a millimeter-scale movement of the first finger relative to the second finger.

8. The method of claim 7, wherein the relative displacement for the millimeter-scale movement is at a resolution of one millimeter or finer.

9. The method of claim 1, further comprising determining, based on the relative displacement between the first and second points during the first movement and the second movement, a gesture.

10. The method of claim 9, further comprising passing the gesture to an application or device effective to control or alter a display, function, or capability associated with the application or the device.

11. One or more computer-readable storage media having instructions stored thereon that, responsive to execution by one or more computer processors, implement a micro-motion tracking module and a recognition module:

the micro-motion tracking module configured to;

receive a radar signal representing a superposition of reflections of two or more points of a hand within a radar field provided by a radar system, a first of the two or more points having a first movement and a second of the two or more points having a second movement;

determine a velocity profile of the radar signal, the velocity profile indicating energies at multiple velocities, a first of the energies representing a reflection of the first point having the first movement and a second of the energies representing a reflection of the second point having the second movement;

extract, from the velocity profile and one or more prior-determined or later-determined velocity profiles over time, relative velocities between the first and second points of the hand; and

determine a displacement trajectory by integrating the relative velocities, the displacement trajectory representing a relative displacement between the first and second points of the hand, and

the recognition module configured to;

determine, based on the displacement trajectory, a gesture of the first and second points of the hand; and

pass the gesture effective to control or alter a display, function, or capability of a device.

12. The computer-readable storage media of claim 11, wherein the micro-motion tracking module is further configured to determine a weighted average of the relative velocities and integrating the relative velocities integrates the weighted average of the relative velocities.

13. The computer-readable storage media of claim 12, wherein the determination of the weighted average is based on a higher probability of an accurate reading or lower noise.

14. The computer-readable storage media of claim 11, wherein determining the velocity profile of the radar signal uses a range-Doppler-time data cube for the radar signal.

15. The computer-readable storage media of claim 14, wherein integrating the relative velocities quantitatively combines the relative velocities from the range-Doppler-time data cube for the radar signal and an unwrapped signal phase of the radar signal.

16. The computer-readable storage media of claim 11, wherein the displacement trajectory represents a sub-gesture step of a complete gesture and wherein the micro-motion tracking module is further configured to determine multiple subsequent displacement trajectories and the recognition module is further configured to determine sub-gestures of the complete gesture and to pass the sub-gestures effective to control or alter the display, function, or capability of the device in real time.

17. An apparatus comprising:

one or more computer processors;

a radar system comprising;

one or more radar-emitting elements configured to provide a radar field; and

one or more antenna elements configured to receive a radar signal representing a superposition of reflections of two or more points of a hand within the radar field; and

one or more computer-readable storage media having instructions stored thereon that, responsive to execution by the one or more computer processors, implement a micro-motion tracking module and a recognition module;

the micro-motion tracking module configured to;

receive the radar signal representing the superposition of reflections of the two or more points of the hand within the radar field provided by the radar system, a first of the two or more points having a first movement and a second of the two or more points having a second movement;

determine a velocity profile of the radar signal, the velocity profile indicating energies at multiple velocities, a first of the energies representing a reflection of the first point having the first movement and a second of the energies representing a reflection of the second point having the second movement;

extract, from the velocity profile and one or more prior-determined or later-determined velocity profiles over time, relative velocities between the first and second points of the hand; and

determine a displacement trajectory by integrating the relative velocities, the displacement trajectory representing a relative displacement between the first and second points of the hand, and

the recognition module configured to;

determine, based on the displacement trajectory, a gesture of the first and second points of the hand; and

pass the gesture to an application of the apparatus or a remote or peripheral device effective to control or alter a display, function, or capability of the apparatus or the remote or peripheral device.

18. The apparatus of claim 17, wherein the radar system has a hardware-parameter-based displacement-sensing resolution limit and a resolution of the relative displacement is finer than the hardware-parameter-based displacement-sensing resolution limit.

19. The apparatus of claim 18, wherein the hardware-parameter-based displacement-sensing resolution limit is based on a wavelength of the radar field provided by the radar system, and wherein the resolution of the relative displacement is finer than the wavelength.

20. The apparatus of claim 17, wherein the one or more radar-emitting elements is one radar-emitting element and the one or more antenna elements is one radar antenna element, and the radar field provided by the radar system is a broad beam, full contiguous radar field.