Non-Line-of-Sight Radar-Based Gesture Recognition

• US 20160098089A1
• Filed: 12/24/2014
• Published: 04/07/2016
• Est. Priority Date: 10/02/2014
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Poupyrev, Ivan
• Granted Patent: US 9,600,080 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G01S 13/02 Systems using reflection of radio waves, e.g. primary radar systems; Analogous systems
• CPC Class Codes: G01S 13/0209 Systems with very large relative bandwidth, i.e. larger than 10 %, e.g. baseband, pulse, carrier-free, ultrawideband
• CPC Class Codes: G01S 13/04 Systems determining presence of a target based on relative movement of target G01S13/56
• CPC Class Codes: G01S 13/72 for two-dimensional tracking, e.g. combination of angle and range tracking, track-while-scan radar
• CPC Class Codes: G01S 13/723 by using numerical data
• CPC Class Codes: G01S 13/867 Combination of radar systems with cameras
• CPC Class Codes: G01S 2013/462 using multipath signals
• CPC Class Codes: G01S 2013/464 using only the non-line-of-sight signal(s), e.g. to enable survey of scene 'behind' the target only the indirect signal is evaluated
• CPC Class Codes: G01S 7/40 Means for monitoring or calibrating
• CPC Class Codes: G01S 7/41 using analysis of echo signal for target characterisation; Target signature; Target cross-section
• CPC Class Codes: G01S 7/415 Identification of targets based on measurements of movement associated with the target
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0304 Detection arrangements using opto-electronic means constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042
• CPC Class Codes: G06V 40/20 Movements or behaviour, e.g. gesture recognition recognition of facial expressions G06V40/16
• CPC Class Codes: G06V 40/28 Recognition of hand or arm movements, e.g. recognition of deaf sign language static hand signs G06V40/113
• CPC Class Codes: H04N 23/80 Camera processing pipelines; Components thereof

Abstract

This document describes techniques and devices for non-line-of-sight radar-based gesture recognition. Through use of the techniques and devices described herein, users may control their devices through in-the-air gestures, even when those gestures are not within line-of-sight of their device'"'"'s sensors. Thus, the techniques enable users to control their devices in many situations in which control is desired but conventional techniques do permit effective control, such as to turn the temperature down in a room when the user is obscured from a thermostat'"'"'s gesture sensor, turn up the volume on a media player when the user is in a different room than the media player, or pause a television program when the user'"'"'s gesture is obscured by a chair, couch, or other obstruction.

Claims

1. A computer-implemented method comprising:

determining a volume of a room in which there is no line-of-sight to a non-line-of-sight gesture-recognition system; and

determining, by providing radar transmissions from the non-line-of-sight gesture-recognition system and into the room, a set of transmission paths sufficient to cause a reflected radar field to be provided within at least a portion of the volume, the reflected radar field enabling capture of gesture interactions with the reflected radar field.

2. The computer-implemented method of claim 1, wherein determining the volume of the room comprises:

capturing an image using a non-radar camera to provide an image of the room from a perspective of the non-line-of-sight gesture-recognition system;

receiving or determining dimensions of the room;

determining, based on the dimensions of the room and the image of the room, non-imaged but existing floor, wall, or ceiling areas; and

determining, based on the non-imaged but existing floor, wall, or ceiling areas, the volume of the room in which there is no line-of-sight to the non-line-of-sight gesture-recognition system.

3. The computer-implemented method of claim 1, wherein determining the volume of the room is based on tracking, with a radar transmission, a person or animal'"'"'s movement in the room and, during a portion of the movement, not receiving a reflection from the person or animal or a part of the person or the animal.

4. The computer-implemented method of claim 1, wherein determining the volume of the room comprises:

providing a radar field effective to receive reflections from objects in the room; and

determining the volume of the room in which there is no line-of-sight to the non-line-of-sight gesture-recognition system as behind the objects relative to the non-line-of-sight gesture-recognition system.

5. The computer-implemented method of claim 1, wherein determining the set of transmission paths iteratively provides the radar transmissions and correlating reflections from the iteratively provided radar transmissions.

6. The computer-implemented method of claim 1, further comprising:

providing the reflected radar field within the volume;

receiving a reflection of a gesture interaction made within the volume; and

determining, based on the reflection of the gesture interaction, a gesture made within the volume.

7. A computer-implemented method comprising:

providing, through a non-line-of-sight gesture-recognition system, a radar field within a room, the radar field including direct radar fields and reflected radar fields, the reflected radar fields provided in one or more volumes in the room to which direct line-of-sight from the non-line-of-sight gesture-recognition system is not available;

receiving a reflection of a gesture interaction made within the radar field within the room, the reflection enabling capture of data about the gesture interaction;

determining a gesture based on captured data associated with the reflection of the gesture interaction; and

providing the gesture to a device or application effective to control the device or application.

8. The computer-implemented method of claim 7, wherein the reflection of the gesture interaction is made within one of the reflected radar fields.

9. The computer-implemented method of claim 7, wherein providing the radar field within the room further provides penetration radar fields within portions of the one or more volumes of the room in which the direct line-of-sight from the non-line-of-sight gesture-recognition system is not available.

10. The computer-implemented method of claim 9, wherein the reflection of the gesture interaction is made within one of the penetration radar fields.

11. A computer-implemented method comprising:

providing a radar field;

sensing reflections from an interaction that is within the radar field;

recognizing, at a smart device associated with the radar field and based on the reflections from the interaction that is within the radar field, a gesture;

determining that the gesture is associated with a non-line-of-sight smart device; and

passing the gesture from the smart device to the non-line-of-sight smart device effective to enable the gesture to control the non-line-of-sight smart device or establish communication with the non-line-of-sight smart device.

12. The computer-implemented method of claim 11, wherein providing the radar field provides a direct radar field and reflected radar field, the reflected radar field provided in one or more volumes in the room to which direct line-of-sight from the non-line-of-sight gesture-recognition system is not available, the interaction is within the reflected radar field, and sensing the reflections is from the interaction within the reflected radar field.

13. The computer-implemented method of claim 12, wherein sensing the reflections from the interaction within the reflection radar field are received at an antenna of the non-line-of-sight gesture-recognition system and reflected from a surface of the room.

14. The computer-implemented method of claim 11, wherein providing the radar field provides a direct radar field and penetration radar field, the penetration radar field provided in one or more volumes in the room to which direct line-of-sight from the non-line-of-sight gesture-recognition system is not available, the interaction is within the penetration radar field, and sensing the reflections is from the interaction within the penetration radar field.

15. The computer-implemented method of claim 14, wherein sensing the reflections from the interaction within the penetration radar field are received at an antenna of the non-line-of-sight gesture-recognition system and through an object occluding the interaction within the penetration radar field.

16. The computer-implemented method of claim 11, further comprising receiving a second gesture from a second non-line-of-sight gesture-recognition system of the non-line-of-sight smart device, the second gesture associated with the smart device and effective to enable the second gesture to control the smart device.

17. The computer-implemented method of claim 11, wherein determining that the gesture is associated with the non-line-of-sight smart device maps the gesture to a pre-configured control gesture associated with a control input for an application associated with the non-line-of-sight smart device.

18. The computer-implemented method of claim 11, wherein determining that the gesture is associated with the non-line-of-sight smart device maps the gesture to a pre-configured communication-establishment gesture associated the non-line-of-sight smart device.

19. The computer-implemented method of claim 11, wherein the gesture establishes communication with the non-line-of-sight smart device effective to enable control through a future gesture determined at the smart device and sensed through the non-line-of-sight gesture-recognition system of the smart device.

20. The computer-implemented method of claim 11, wherein determining that the gesture is associated with the non-line-of-sight smart device maps the gesture to a unique gesture of a set of unique gestures unique to each of a set of smart devices.