Smartphone-based radar system for facilitating awareness of user presence and orientation

• US 10,770,035 B2
• Filed: 08/22/2018
• Issued: 09/08/2020
• Est. Priority Date: 08/22/2018
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google LLC (Alphabet Inc.)
• Inventors: Giusti, Leonardo, Poupyrev, Ivan, Barbello, Brandon, Amihood, Patrick M.
• Primary Examiner(s): Rayan, Mihir K
• Application Number:
  US16/109,534
• Publication Number:
  US 20200066236A1
• Time in Patent Office: 748 Days
• CPC Class Codes: G01S 13/42 Simultaneous measurement of distance and other co-ordinates indirect measurement G01S13/46
• CPC Class Codes: G01S 7/41 using analysis of echo signal for target characterisation; Target signature; Target cross-section
• CPC Class Codes: G01S 7/415 Identification of targets based on measurements of movement associated with the target
• CPC Class Codes: G01S 7/417 involving the use of neural networks
• CPC Class Codes: G06F 1/3203 Power management, i.e. event-based initiation of a power-saving mode
• CPC Class Codes: G06F 1/3231 Monitoring the presence, absence or movement of users
• CPC Class Codes: G06F 2203/041 Indexing scheme relating to G06F3/041 - G06F3/045
• CPC Class Codes: G06F 2203/04101 2.5D-digitiser, i.e. digitiser detecting the X/Y position of the input means, finger or stylus, also when it does not touch, but is proximate to the digitiser's interaction surface and also measures the distance of the input means within a short range in the Z direction, possibly with a separate measurement setup
• CPC Class Codes: G06F 2203/04108 Touchless 2D- digitiser, i.e. digitiser detecting the X/Y position of the input means, finger or stylus, also when it does not touch, but is proximate to the digitiser's interaction surface without distance measurement in the Z direction
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0346 with detection of the device orientation or free movement in a 3D space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors
• CPC Class Codes: G06V 10/143 Sensing or illuminating at different wavelengths
• CPC Class Codes: G06V 10/242 by image rotation, e.g. by 90 degrees
• CPC Class Codes: G09G 2340/0492 Change of orientation of the displayed image, e.g. upside-down, mirrored
• CPC Class Codes: G09G 2354/00 Aspects of interface with display user
• CPC Class Codes: G09G 5/38 with means for controlling the display position
• CPC Class Codes: H04M 1/724 User interfaces specially adapted for cordless or mobile telephones
• CPC Class Codes: H04M 1/72454 according to context-related or environment-related conditions

Abstract

This document describes techniques and systems that enable a smartphone-based radar system for facilitating awareness of user presence and orientation. The techniques and systems use a radar field to accurately determine a user'"'"'s location and physical orientation with respect to an electronic device, such as a smartphone. The radar field also enables the device to receive 3D gestures from the user to interact with the device. The techniques allow the device to provide functionality based on the user'"'"'s presence and orientation, and to appropriately adjust the timing, content, and format of the device'"'"'s interactions with the user.

Claims

1. A smartphone, comprising:

a display;

a radar system, implemented at least partially in hardware, configured to;

provide a radar field;

sense reflections from an object in the radar field;

analyze the reflections from the object in the radar field; and

provide, based on the analysis of the reflections, radar data;

one or more computer processors; and

one or more computer-readable media having instructions stored thereon that, responsive to execution by the one or more computer processors, perform operations comprising;

determining, based on a first subset of the radar data, an orientation of the smartphone with reference to the object;

responsive to determining the orientation of the smartphone, entering a do-not-disturb (DND) mode when the smartphone is determined to be in a first orientation with reference to the object;

determining, based on a second subset of the radar data, a change in the orientation of the smartphone with reference to the object, the change in the orientation comprising a rotation of the smartphone within a plane that is substantially parallel to a viewing surface of the display, the rotation effective to cause the smartphone to be in a second orientation with reference to the object; and

responsive to the change in orientation, exiting the DND mode.

2. The smartphone of claim 1, wherein:

the object is a user;

the first orientation with reference to the user is a landscape orientation; and

the second orientation with reference to the user is a portrait orientation.

3. The smartphone of claim 1, wherein the rotation of the smartphone within a plane that is substantially parallel to a viewing surface of the display does not include lifting or tilting the smartphone.

4. The smartphone of claim 1, wherein the DND mode is a mode in which:

at least one of reminders, notifications, or other device-initiated communications are silenced and;

at least one user interface element is visible on the display.

5. The smartphone of claim 4, wherein the operations further comprise:

responsive to exiting the DND mode, providing reminders, notifications, or other device-initiated communications that were silenced while the smartphone was in the DND mode.

6. The smartphone of claim 1, wherein the rotation of the smartphone within a plane that is substantially parallel to the viewing surface of the display is effective to cause the smartphone to be in the second orientation with reference to the object when the rotation exceeds a threshold rotational distance.

7. The smartphone of claim 6, wherein the threshold rotational distance is one of approximately 25, approximately 45, approximately 90, or approximately 180 degrees.

8. The smartphone of claim 6, wherein the threshold rotational distance is:

predefined;

user-selectable;

or

determined via a machine learning module that is included or associated with the radar system.

9. The smartphone of claim 1, wherein the radar system further comprises a digital beamformer and an angle estimator, and the radar system is configured to monitor angles in a field of view between approximately −

90 degrees and approximately 90 degrees.

10. A smartphone, comprising:

a display;

a radar system, implemented at least partially in hardware, configured to;

provide a radar field;

sense reflections from at least a first user and a second user in the radar field;

analyze the reflections from at least the first user and the second user in the radar field; and

provide, based on the analysis of the reflections, radar data;

one or more computer processors; and

one or more computer-readable media having instructions stored thereon that, responsive to execution by the one or more computer processors, perform operations comprising;

determining, based on a first subset of the radar data, an orientation of the smartphone with reference to the first user;

responsive to determining the orientation of the smartphone, presenting content on the display in a first-user-facing orientation;

determining, based on a second subset of the radar data, a change in the orientation of the smartphone with reference to the first user, the change in the orientation of the smartphone with reference to the first user comprising a displacement of the smartphone, the displacement effective to cause the smartphone to become farther from the first user and closer to the second user; and

responsive to the change in orientation, presenting content on the display in a second-user-facing orientation.

11. The smartphone of claim 10, wherein the displacement of the smartphone, effective to cause the smartphone to become farther from the first user and closer to the second user, does not include lifting or rotating the smartphone.

12. The smartphone of claim 10, wherein the operations further comprise:

presenting content on the display in a second-user-facing orientation when the displacement exceeds a threshold distance.

13. The smartphone of claim 12, wherein the threshold distance is one of approximately six, approximately twelve, or approximately eighteen inches.

14. The smartphone of claim 12, wherein the threshold distance is:

predefined;

user-selectable;

or

determined via a machine learning module that is included or associated with the radar system.

15. The smartphone of claim 10, wherein the radar system further comprises a digital beamformer and an angle estimator, and the radar system is configured to monitor angles in a field of view between approximately −

90 degrees and approximately 90 degrees.

16. A method implemented in an electronic device that includes a radar system and a radar-based application, the method comprising:

providing, by the radar system, a radar field;

sensing, by the radar system, reflections from an object in the radar field;

analyzing the reflections from the object in the radar field;

providing, based on the analysis of the reflections, radar data;

determining, based on a first subset of the radar data, an orientation of the smartphone with reference to the object;

responsive to determining the orientation of the smartphone, entering a do-not-disturb (DND) mode when the smartphone is determined to be in a first orientation with reference to the object;

determining, based on a second subset of the radar data, a change in the orientation of the smartphone with reference to the object, the change in the orientation comprising a rotation of the smartphone within a plane that is substantially parallel to a viewing surface of the display, the rotation effective to cause the smartphone to be in a second orientation with reference to the object; and

responsive to the change in orientation, exiting the DND mode.

17. The method of claim 16, wherein:

the object is a user;

the first orientation with reference to the user is a landscape orientation; and

the second orientation with reference to the user is a portrait orientation.

18. The method of claim 16, wherein the rotation of the smartphone within a plane that is substantially parallel to a viewing surface of the display does not include lifting or tilting the smartphone.

19. The method of claim 16, wherein the DND mode is a mode in which:

at least one of reminders, notifications, or other device-initiated communications are silenced; and

at least one user interface element is visible on the display.

20. The method of claim 19, wherein the operations further comprise:

responsive to exiting the DND mode, providing reminders, notifications, or other device-initiated communications that were silenced while the smartphone was in the DND mode.