Proximity detection via audio

• US 9,148,742 B1
• Filed: 02/29/2012
• Issued: 09/29/2015
• Est. Priority Date: 07/29/2011
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Koulomzin, Daniel G., Wren, Christopher R.
• Primary Examiner(s): Lu, Shirley
• Application Number:
  US13/408,308
• Time in Patent Office: 1,308 Days
• Field of Search: 340/540, 455/456.1, 455/414.3
• US Class Current: 1/1
• CPC Class Codes: G01S 11/14 using ultrasonic, sonic, or infrasonic waves
• CPC Class Codes: G06Q 10/00 Administration; Management
• CPC Class Codes: G06Q 2220/00 Business processing using cryptography
• CPC Class Codes: G08B 1/00 Systems for signalling characterised solely by the form of transmission of the signal
• CPC Class Codes: G10L 19/018 Audio watermarking, i.e. embedding inaudible data in the audio signal
• CPC Class Codes: H04W 4/00 Services specially adapted for wireless communication networks; Facilities therefor
• CPC Class Codes: H04W 4/023 using mutual or relative location information between multiple location based services [LBS] targets or of distance thresholds
• CPC Class Codes: H04W 4/21 for social networking applications

Abstract

Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for detecting ambient audio, generating audio data based on the ambient audio, generating a first audio fingerprint based on the audio data, providing the first audio fingerprint to a fingerprint comparison engine, the fingerprint comparison engine providing a comparison between the first audio fingerprint and a second audio fingerprint, and receiving a proximity signal based on the comparison.

Claims

1. A computer-implemented method comprising:

receiving, by a first mobile device, a request from a first user of the first mobile device to activate a first proximity detection service of the first mobile device;

identifying a first coarse location associated with the first mobile device, and generating a first coarse location signal based on the first coarse location;

receiving, by a second mobile device, a request from a second user of the second mobile device to activate a second proximity detection service of the second mobile device;

identifying a second coarse location associated with the second mobile device, and generating a second coarse location signal based on the second coarse location;

comparing the first and the second coarse location signals, and in response, determining that the first and the second mobile devices are within a predetermined coarse geographic proximity;

in response to determining that the first and the second mobile devices are within a predetermined coarse geographic proximity, performing a proximity detection between the first and the second mobile devices, including;

capturing, by a first audio sensor of the first mobile device, first audio signals;

generating, by the first mobile device, first audio data based on the first audio signals;

generating one or more first audio fingerprints based upon the first audio data;

capturing, by a second audio sensor of the second mobile device, second audio signals;

generating, by the second mobile device, second audio data based on the second audio signals;

generating one or more second audio fingerprints based upon the second audio data;

generating, using the one or more first and second audio fingerprints, a similarity score that indicates a similarity between the first audio signals and the second audio signals;

determining that the similarity score exceeds a threshold similarity score;

based on the determining, generating a proximity signal usable to indicate that the first mobile device is proximate to the second mobile device; and

providing, by the first mobile device, the second mobile device, or both the first and second mobile devices, an indication that the first user is proximate to the second user.

2. The computer-implemented method of claim 1, further including generating, by the second mobile device, the first audio signals.

3. The computer-implemented method of claim 1, further including decoding, by the first mobile device, the first audio signals to determine an identifier associated with the second mobile device.

4. A computing device, comprising:

a non-transitory computer-readable medium coupled to one or more computers having instructions stored thereon which, when executed by the one or more computers, cause the one or more computers to perform operations comprising;

identifying a first coarse location of the computing device, and generating a first coarse location signal based on the first coarse location;

providing the first coarse location signal to a comparison engine, the comparison engine providing a comparison between the first coarse location and a second coarse location to determine that the first and the second coarse locations are within a predetermined coarse geographic proximity;

receiving a coarse location signal indicating that the first coarse location and the second coarse location are within the predetermined coarse geographic proximity, and in response, performing a proximity detection, including;

detecting ambient audio;

generating audio data based on the ambient audio;

generating a first audio fingerprint based on the audio data;

providing the first audio fingerprint to the comparison engine, the comparison engine providing a comparison between the first audio fingerprint and a second audio fingerprint; and

receiving a proximity signal based on the comparison.

5. The computing device of claim 4, wherein the proximity signal indicates that another computing device is proximate to the computing device.

6. The computing device of claim 5, wherein the another computing device is a source of the second audio fingerprint.

7. The computing device of claim 4, wherein the ambient audio comprises audio generated by another computing device, the another computing device being a source of the second audio fingerprint.

8. The computing device of claim 7, wherein operations further comprise decoding the audio generated by the another computing device to determine an identifier associated with the another computing device.

9. The computing device of claim 7, wherein the audio generated by the another computing device is sub-sonic audio.

10. The computing device of claim 4, wherein the comparison engine is executed using a server system and the first audio fingerprint is provided to the server system by transmitting the first audio fingerprint to the server system over a network.

11. The computing device of claim 4, wherein the first audio fingerprint is generated using a fingerprinting engine, the fingerprinting engine being executed on the computing device.

12. The computing device of claim 4, wherein the comparison engine is executed on the computing device.

13. The computing device of claim 4, wherein the comparison between the first audio fingerprint and the second audio fingerprint comprises:

determining a similarity score based on the first audio fingerprint and the second audio fingerprint;

comparing the similarity score to a threshold similarity score; and

generating the proximity signal based on comparing the similarity score to a threshold similarity score.

14. The computing device of claim 13, wherein the proximity signal indicates that the computing device is located proximate to another computing device when the similarity score is greater than the threshold similarity score, the another computing device being a source of the second audio fingerprint.

15. The computing device of claim 4, wherein the coarse location signal comprises at least one of a global positioning system (GPS) signal, a WiFi signal and a cellular triangulation signal.

16. The computing device of claim 4, wherein generating the first coarse location signal comprises executing a location-aware computer program application on the computing device, the location-aware computer program application generating the coarse location signal.

17. The computing device of claim 4, wherein the computing device is a mobile computing device.

18. A computer storage medium encoded with a computer program, the program comprising instructions that when executed by one or more computers cause the one or more computers to perform operations comprising:

identifying a first coarse location of the computing device, and generating a first coarse location signal based on the first coarse location;

providing the first coarse location signal to a comparison engine, the comparison engine providing a comparison between the first coarse location and a second coarse location to determine that the first and the second coarse locations are within a predetermined coarse geographic proximity;

receiving a coarse location signal indicating that the first coarse location and the second coarse location are within the predetermined coarse geographic proximity, and in response, performing a proximity detection, including;

detecting ambient audio;

generating audio data based on the ambient audio;

generating a first audio fingerprint based on the audio data;

providing the first audio fingerprint to a fingerprint comparison engine, the fingerprint comparison engine providing a comparison between the first audio fingerprint and a second audio fingerprint; and

receiving a proximity signal based on the comparison.