Audiolocation system combining use of audio fingerprinting and audio watermarking

• US 9,332,522 B2
• Filed: 05/20/2014
• Issued: 05/03/2016
• Est. Priority Date: 05/20/2014
• Status: Active Grant

• Current Assignee: Disney Enterprises Incorporated (The Walt Disney Company)
• Original Assignee: Disney Enterprises Incorporated (The Walt Disney Company)
• Inventors: Corbellini, Giorgio, Mangold, Stefan, Vukadinovic, Vladimir
• Primary Examiner(s): Reinier, Barbara
• Application Number:
  US14/282,062
• Publication Number:
  US 20150341890A1
• Time in Patent Office: 714 Days
• Field of Search: 382/100, 382/232, 380/210, 380/287, 380/54, 713/176, 480/460, 386/94, 399/366, 283/902, 358/1.14, 358/3.28, 726/26, 726/31
• US Class Current: 1/1
• CPC Class Codes: G01S 1/00 Beacons or beacon systems transmitting signals having a characteristic or characteristics capable of being detected by non-directional receivers and defining directions, positions, or position lines fixed relatively to the beacon transmitters; Receivers co-operating therewith position fixing by co-ordinating a plurality of determinations of direction or position lines G01S5/00
• CPC Class Codes: G01S 5/18 using ultrasonic, sonic, or infrasonic waves
• CPC Class Codes: H04W 4/02 Services making use of location information
• CPC Class Codes: H04W 4/021 Services related to particular areas, e.g. point of interest [POI] services, venue services or geofences
• CPC Class Codes: H04W 4/024 Guidance services
• CPC Class Codes: H04W 4/029 Location-based management or tracking services

Abstract

An audio-based navigation system for determining a present location or position of a mobile device such as a smartphone within a venue. The system includes a sound system with a first speaker playing first audio content and a second speaker playing second audio content. The first speaker is positioned in a first space in a venue, and the second speaker is positioned in a second space of the venue. The navigation system also includes a mobile device with memory storing a database of fingerprints of a plurality of differing audio content. The mobile device includes a microphone for receiving an audio signal associated with the first audio content or with the second audio content. The mobile device includes a processor(s) selectively running software to determine the mobile device location in the first or second space using either fingerprinting or watermarking module by processing the received audio signal.

Claims

1. A system for audio-based navigation or localization, comprising:

a sound system with a first speaker playing first audio content and a second speaker playing second audio content, wherein the first speaker is positioned in a first space in a venue and the second speaker is positioned in a second space of the venue; and

a mobile device with memory storing a database of fingerprints of a plurality of differing audio content, with a microphone for receiving an audio signal associated with the first audio content or the second audio content, and with a processor running at least one of a fingerprinting module and a watermarking module to determine a location of the mobile device by processing the received audio signal,

wherein the processor runs the fingerprinting module and the watermarking module in parallel to process the audio signal, and

wherein the location is provided by the watermarking module when a watermark is decoded from the audio signal by the watermarking module and provided by the fingerprinting module when decoding by the watermarking module fails.

2. The system of claim 1, wherein the received audio signal comprises a watermark identifying the first speaker or the second speaker as a source of the received audio signal and wherein the watermarking module decodes the watermark to provide the location of the mobile device relative to the first or second speaker.

3. The system of claim 1, wherein the fingerprinting module determines the location by generating a fingerprint based on the audio signal, comparing the generated fingerprint with the fingerprints in the database to find a best matching fingerprint, and finding a location cross-referenced to the best matching fingerprint.

4. A system for audio-based navigation or localization, comprising:

a sound system with a first speaker playing first audio content and a second speaker playing second audio content, wherein the first speaker is positioned in a first space in a venue and the second speaker is positioned in a second space of the venue; and

a mobile device with memory storing a database of fingerprints of a plurality of differing audio content, with a microphone for receiving an audio signal associated with the first audio content or the second audio content, and with a processor running at least one of a fingerprinting module and a watermarking module to determine a location of the mobile device by processing the received audio signal,

wherein the processor initially only runs the fingerprinting module to provide the location and runs the watermarking module to provide the location after the audio signal is locked based on processing by the fingerprinting module,

wherein the processor switches the fingerprinting module off when the audio signal is locked, and

wherein the processor resumes use of the fingerprinting module and switches the watermarking module off when the watermarking is unable to decode a watermark in the received audio signal.

5. The system of claim 4, wherein the audio signal is locked when a fingerprint match achieved by the fingerprinting module has a confidence level greater than a predefined minimum confidence level.

6. The system of claim 4, wherein the received audio signal comprises a watermark identifying the first speaker or the second speaker as a source of the received audio signal and wherein the watermarking module decodes the watermark to provide the location of the mobile device relative to the first or second speaker.

7. The system of claim 4, wherein the fingerprinting module determines the location by generating a fingerprint based on the audio signal, comparing the generated fingerprint with the fingerprints in the database to find a best matching fingerprint, and finding a location cross-referenced to the best matching fingerprint.

8. A system for audio-based navigation or localization, comprising:

a sound system with a first speaker playing first audio content and a second speaker playing second audio content, wherein the first speaker is positioned in a first space in a venue and the second speaker is positioned in a second space of the venue; and

a mobile device with memory storing a database of fingerprints of a plurality of differing audio content, with a microphone for receiving an audio signal associated with the first audio content or the second audio content, and with a processor running at least one of a fingerprinting module and a watermarking module to determine a location of the mobile device by processing the received audio signal,

wherein the processor initially only runs the fingerprinting module to provide the location and runs the watermarking module to provide the location after the audio signal is locked based on processing by the fingerprinting module, and

wherein the processor switches the watermarking module off after a first watermark is decoded in the received audio signal and switches the fingerprinting module on to provide the location on a periodic basis.

9. The system of claim 8, wherein the fingerprinting module performs fingerprinting at an increased frequency upon a loss of signal lock.

10. The system of claim 8, wherein the received audio signal comprises a watermark identifying the first speaker or the second speaker as a source of the received audio signal and wherein the watermarking module decodes the watermark to provide the location of the mobile device relative to the first or second speaker.

11. The system of claim 8, wherein the fingerprinting module determines the location by generating a fingerprint based on the audio signal, comparing the generated fingerprint with the fingerprints in the database to find a best matching fingerprint, and finding a location cross-referenced to the best matching fingerprint.

12. The system of claim 8, wherein the audio signal is locked when a fingerprint match achieved by the fingerprinting module has a confidence level greater than a predefined minimum confidence level.

13. A system for audio-based localization, comprising:

a sound system with a first speaker playing first audio content and a second speaker playing second audio content, wherein the first speaker is positioned in a first space in a venue and the second speaker is positioned in a second space of the venue; and

a mobile device with memory storing a database of fingerprints of a plurality of differing audio content, with a microphone for receiving an audio signal associated with the first audio content or the second audio content, and with a processor selectively running a fingerprinting module and a watermarking module to determine a location of the mobile device by processing the received audio signal,

wherein the processor first runs the fingerprinting module to provide the location and second runs the watermarking to provide the location after the audio signal is locked based on processing by the fingerprinting module,

wherein the processor switches the fingerprinting module off when the audio signal is locked, and

wherein the processor resumes use of the fingerprinting module and switches the watermarking module off when the watermarking is unable to decode a watermark in the received audio signal.

14. The system of claim 13, wherein the audio signal is locked when a fingerprint match achieved by the fingerprinting module has a confidence level greater than a predefined minimum confidence level.

15. The system of claim 13, wherein the processor switches the watermarking module off after a first watermark is decoded in the received audio signal and switches the fingerprinting module on to provide the location on a periodic basis.

16. The system of claim 15, wherein the fingerprinting module performs fingerprinting at an increased frequency upon a loss of signal lock.

17. A system for audio-based navigation or localization, comprising:

a sound system with a first speaker playing first audio content and a second speaker playing second audio content, wherein the first speaker is positioned in a first space in a venue and the second speaker is positioned in a second space of the venue; and

a mobile device with a microphone for receiving an audio signal associated with the first audio content or the second audio content,

wherein the mobile device further includes a processor running a fingerprinting module and a watermarking module to determine a location of the mobile device by processing the received audio signal,

wherein the processor runs the fingerprinting module and the watermarking module in parallel to process the audio signal, and

wherein the location is provided by the watermarking module when a watermark is decoded from the audio signal by the watermarking module and provided by the fingerprinting module when decoding by the watermarking module fails.

18. A system for audio-based navigation or localization, comprising:

a sound system with a first speaker playing first audio content and a second speaker playing second audio content, wherein the first speaker is positioned in a first space in a venue and the second speaker is positioned in a second space of the venue; and

a mobile device with a microphone for receiving an audio signal associated with the first audio content or the second audio content,

wherein the mobile device further includes a processor running a fingerprinting module and a watermarking module to determine a location of the mobile device by processing the received audio signal,

wherein the processor initially only runs the fingerprinting module to provide the location and runs the watermarking to provide the location after the audio signal is locked based on processing by the fingerprinting module,

wherein the processor switches the fingerprinting module off when the audio signal is locked, and

wherein the processor resumes use of the fingerprinting module and switches the watermarking module off when the watermarking is unable to decode a watermark in the received audio signal.

19. The system of claim 18, wherein the audio signal is locked when a fingerprint match achieved by the fingerprinting module has a confidence level greater than a predefined minimum confidence level.

20. A system for audio-based navigation or localization, comprising:

a sound system with a first speaker playing first audio content and a second speaker playing second audio content, wherein the first speaker is positioned in a first space in a venue and the second speaker is positioned in a second space of the venue; and

a mobile device with a microphone for receiving an audio signal associated with the first audio content or the second audio content,

wherein the mobile device further includes a processor running a fingerprinting module and a watermarking module to determine a location of the mobile device by processing the received audio signal,

wherein the processor initially only runs the fingerprinting module to provide the location and runs the watermarking to provide the location after the audio signal is locked based on processing by the fingerprinting module,

wherein the processor switches the watermarking module off after a first watermark is decoded in the received audio signal and switches the fingerprinting module on to provide the location on a periodic basis, and

wherein the fingerprinting module performs fingerprinting at an increased frequency upon a loss of signal lock.

21. A system for audio-based localization, comprising:

a sound system with a first speaker playing first audio content and a second speaker playing second audio content, wherein the first speaker is positioned in a first space in a venue and the second speaker is positioned in a second space of the venue; and

a mobile device with memory storing a database of fingerprints of a plurality of differing audio content, with a microphone for receiving an audio signal associated with the first audio content or the second audio content, and with a processor selectively running a fingerprinting module and a watermarking module to determine a location of the mobile device by processing the received audio signal,

wherein the processor first runs the fingerprinting module to provide the location and second runs the watermarking to provide the location after the audio signal is locked based on processing by the fingerprinting module, and

wherein the processor switches the watermarking module off after a first watermark is decoded in the received audio signal and switches the fingerprinting module on to provide the location on a periodic basis.

22. The system of claim 21, wherein the audio signal is locked when a fingerprint match achieved by the fingerprinting module has a confidence level greater than a predefined minimum confidence level.

23. The system of claim 21, wherein the fingerprinting module performs fingerprinting at an increased frequency upon a loss of signal lock.