Determining relative positions of user devices

• US 10,655,951 B1
• Filed: 06/25/2015
• Issued: 05/19/2020
• Est. Priority Date: 06/25/2015
• Status: Active Grant

• Current Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Original Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Inventors: Froment, Arnaud Marie, Kiraly, Kenneth Paul, Dolbakian, Levon
• Primary Examiner(s): Perungavoor, Sath V
• Assistant Examiner(s): Dobbs, Kristin
• Application Number:
  US14/750,266
• Time in Patent Office: 1,790 Days
• Field of Search: 348158
• CPC Class Codes: G01B 11/14 for measuring distance or clearance between spaced objects or spaced apertures G01B11/26 takes precedence; rangefinders G01C3/00
• CPC Class Codes: G01B 11/26 for measuring angles or tapers; for testing the alignment of axes
• CPC Class Codes: G06T 11/20 Drawing from basic elements, e.g. lines or circles
• CPC Class Codes: G06T 7/70 Determining position or orientation of objects or cameras camera calibration G06T7/80
• CPC Class Codes: G06V 20/20 in augmented reality scenes

Abstract

Techniques for determining positions of devices within an environment are described herein. In some instances, an environment, such as a home or office of a user, may include an array of devices, some or all of which may couple to a network or to other devices via short-range wireless connections (e.g., Bluetooth®, Zigbee®, etc.). These devices may capture an array of data for providing to a central service, which is configured to analyze the data and, based on this analysis, determine a location of the devices relative to one another. That is, the central service may analyze the data to determine relative distances and orientations between the identified devices within the environment.

Claims

1. A system comprising:

one or more processors; and

one or more computer-readable media storing computer-executable instructions that, when executed on the one or more processors, cause the one or more processors to perform acts comprising;

receiving, from a first device located at a first position within an environment that is remote from the one or more processors of the system, an identifier of the first device and wireless-signal-strength data indicating a wireless signal strength between the first device and a second device located at a second position in the environment;

receiving, from the second device, an identifier of the second device and time-difference-of-arrival (TDOA) data indicative of a TDOA between a first time at which audio was received at a first microphone of the second device and a second time at which the audio was received at a second microphone of the second device, the audio having been output by a third device located at a third position within the environment;

receiving, from the third device within the environment, an identifier of the third device and one or more images of the environment captured by a camera of the third device;

generating information related to the third position of the third device based, at least in part, on the TDOA data; and

generating environment data based at least in part on the identifier of the first device, the wireless-signal-strength data, the identifier of the second device, the TDOA data, the identifier of the third device, and the one or more images, the environment data indicating the first position, the second position, and the third position within the environment relative to one another.

2. The system as recited in claim 1, wherein the wireless-signal-strength data comprises first wireless-signal-strength data, and the acts further comprising:

receiving, from the first device, second wireless-signal-strength data indicating a wireless signal strength between the first device and the third device;

and wherein the generating of the environment data further comprises correlating;

(i) the first wireless-signal-strength data with a distance between the first device and the second device in the environment, and (ii) the second wireless-signal-strength data with a distance between the first device and the third device in the environment.

3. The system as recited in claim 1, wherein:

the one or more images include at least one image that includes the first device;

the acts further comprise;

determining a size, shape, and color of the first device;

identifying an object within the one or more images;

determining that a size, shape, and color of the object within the one or more images corresponds to the size, shape, and color of the first device;

determining that the object comprises the first device;

estimating a distance between the third device and the first device based at least in part on a size of the first device within the one or more images; and

estimating an orientation of the third device relative to the first device based at least in part one or more visual characteristics of the first device within the one or more images; and

the generating is further based at least in part on the distance between the third device and the first device and the orientation of the third device relative to the first device.

4. The system as recited in claim 1, the acts further comprising:

determining, from a device identifier of the first device, that the first device is associated with an account of a user;

storing an indication that the first device is associated with the account of the user such that the first device is available to present content associated with the account of the user;

determining, from a device identifier of the second device, that the second device is not associated with the account of the user; and

storing an indication that the second device is not associated with the account of the user such that the second device is not available to present content associated with the account of the user.

5. A method comprising:

receiving, from a first device located at a first position in an environment, first sensor data generated by the first device, the first sensor data identifying at least a second device located at a second position within the environment;

receiving, from the second device, second sensor data identifying at least a third device located at a third position within the environment, wherein the second sensor data includes time-difference-of-arrival (TDOA) data indicative of a TDOA between audio received at a first microphone of the second device and the audio received at a second microphone of the second device, the audio having been output by the third device;

analyzing the first sensor data and the second sensor data to determine first information related to the first position of the first device relative to the second position of the second device and the third position of the third device;

storing first position data indicative of the first position of the first device;

analyzing the first sensor data and the second sensor data to determine second information related to the second position of the second device relative to the first device and the third device;

storing second position data indicative of the second position of the second device;

analyzing at least one of the first sensor data or the second sensor data to determine third information related to the third position of the third device relative to the first device and the second device; and

storing third position data indicative of the third position of the third device.

6. The method as recited in claim 5, wherein:

the storing of the first position data comprises storing first position data that is indicative of;

(i) a distance between the first device and the second device, (ii) a distance between the first device and third device, and (iii) at least one of an orientation of the first device relative to the third device or an orientation of the first device relative to the second device.

7. The method as recited in claim 5, wherein:

the storing of the first position data comprises storing first position data that is indicative of a first X, Y, and Z position of the first device within the environment;

the storing of the second position data comprises storing second position data that is indicative of a second X, Y, and Z position of the second device within the environment; and

the storing of the third position data comprises storing third position data that is indicative of a third X, Y, and Z position of the third device within the environment.

8. The method as recited in claim 5, wherein the receiving the first sensor data comprises receiving data that indicates a wireless signal strength between the first device and the second device.

9. The method as recited in claim 5, wherein the receiving the first sensor data comprises receiving one or more images captured by a camera of the first device, the one or more images including at least one image that includes the second device.

10. The method as recited in claim 5, wherein the receiving the first sensor data comprises receiving data indicative of the audio output by the second device, and further comprising:

identifying the audio;

determining that the audio matches audio being output in the environment by the second device; and

identifying the second device.

11. The method as recited in claim 5, wherein the receiving the first device comprises receiving data indicative of a barometric pressure measured at the first device.

12. The method as recited in claim 5, further comprising:

determining, from at least one of the first sensor data or the second sensor data, that the first device is moving within the environment;

storing an indication that the first device is mobile;

determining, from at least one of the first sensor data or the second sensor data, that the second device is not moving within the environment; and

storing an indication that the second device is stationary.

13. The method as recited in claim 5, further comprising:

receiving, from the first device, an indication that a mobile device associated with a user account is proximate to the first device; and

sending an instruction to at least one of the first, second, or third devices to present content.

14. The method as recited in claim 5, further comprising:

receiving, from the first device, a first indication that a mobile device associated with a user account is proximate to the first device;

sending a first instruction to the first device to present content;

receiving, from at least one of the first or the second device, a second indication that the mobile device is closer to the second device than the first device; and

sending a second instruction to at least one of the first device or the second device, the second instruction instructing the second device to present the content.

15. The method as recited in claim 5, further comprising:

receiving an audio signal generated by a device within the environment;

performing speech-recognition on the audio signal to identify a voice command of a user, the voice command requesting that content be presented within the environment;

determining, from the audio signal and at least one of the first position data, the second position data, or the third position data, that the user is nearer the first device than the second device; and

presenting the content on the first device.

16. The method as recited in claim 5, further comprising:

receiving an audio signal generated by a device within the environment;

performing speech-recognition on the audio signal to identify a voice command of a user;

determining, from the audio signal and at least one of the first position data, the second position data, or the third position data, that the user is nearer the first device than the second device; and

determining an operation to perform in response to the voice command based at least in part on the user being nearer the first device than the second device.

20. The method as recited in claim 5, further comprising generating environment data based at least in part on the first sensor data and the second sensor data, wherein the environment data indicates the first position of the first device, the second position of the second device, and the third position of the third device relative to one another.

17. A system comprising:

one or more processors; and

one or more computer-readable media storing computer-executable instructions that, when executed on the one or more processors, cause the one or more processors to perform acts comprising;

receiving, from a first device located at a first position in an environment that is remote from the one or more processors of the system, first sensor data identifying at least a second device located at a second position within the environment;

receiving, from the second device, second sensor data identifying at least a third device located at a third position within the environment, wherein the second sensor data includes time-difference-of-arrival (TDOA) data indicative of a TDOA between audio received at a first microphone of the second device and the audio received at a second microphone of the second device, the audio having been output by the third device;

analyzing at least a portion of at least one of the first sensor data or the second sensor data to determine first information related to the first position of the first device relative to the second device and the third device;

storing first position data indicative of the first position of the first device;

analyzing at least a portion of at least one of the first sensor data or the second sensor data to determine second information related to the second position of the second device relative to the first device and the third device;

storing second position data indicative of the second position of the second device;

analyzing the first sensor data and the second sensor to determine third information related to the third position of the third device relative to the first device and the second device; and

storing third position data indicative of the third position of the third device.

18. The system as recited in claim 17, wherein the acts further comprising:

determining, from at least one of the first sensor data or the second sensor data, that the first device is moving within the environment;

storing an indication that the first device is mobile;

determining, from at least one of the first sensor data or the second sensor data, that the second device is not moving within the environment; and

storing an indication that the third device is stationary.

19. The system as recited in claim 17, the acts further comprising:

receiving an audio signal generated by a device within the environment;

performing speech-recognition on the audio signal to identify a voice command of a user;

determining, from the audio signal and at least one of the first position data, the second position data, or the third position data, that the user is nearer the first device than the second device; and

determining an operation to perform in response to the voice command based at least in part on the user being nearer the first device than the second device.