AUDIOLOCATION METHOD AND SYSTEM COMBINING USE OF AUDIO FINGERPRINTING AND AUDIO WATERMARKING

• US 20150341890A1
• Filed: 05/20/2014
• Published: 11/26/2015
• Est. Priority Date: 05/20/2014
• Status: Active Grant

• Current Assignee: Disney Enterprises Incorporated (The Walt Disney Company)
• Original Assignee: Disney Enterprises Incorporated (The Walt Disney Company)
• Inventors: Corbellini, Giorgio, Mangold, Stefan, Vukadinovic, Vladimir
• Granted Patent: US 9,332,522 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G01S 1/00 Beacons or beacon systems transmitting signals having a characteristic or characteristics capable of being detected by non-directional receivers and defining directions, positions, or position lines fixed relatively to the beacon transmitters; Receivers co-operating therewith position fixing by co-ordinating a plurality of determinations of direction or position lines G01S5/00
• CPC Class Codes: G01S 5/18 using ultrasonic, sonic, or infrasonic waves
• CPC Class Codes: H04W 4/02 Services making use of location information
• CPC Class Codes: H04W 4/021 Services related to particular areas, e.g. point of interest [POI] services, venue services or geofences
• CPC Class Codes: H04W 4/024 Guidance services
• CPC Class Codes: H04W 4/029 Location-based management or tracking services

Abstract

An audio-based navigation system for determining a present location or position of a mobile device such as a smartphone within a venue. The system includes a sound system with a first speaker playing first audio content and a second speaker playing second audio content. The first speaker is positioned in a first space in a venue, and the second speaker is positioned in a second space of the venue. The navigation system also includes a mobile device with memory storing a database of fingerprints of a plurality of differing audio content. The mobile device includes a microphone for receiving an audio signal associated with the first audio content or with the second audio content. The mobile device includes a processor(s) selectively running software to determine the mobile device location in the first or second space using either fingerprinting or watermarking module by processing the received audio signal.

Claims

1. A system for audio-based navigation or localization, comprising:

a sound system with a first speaker playing first audio content and a second speaker playing second audio content, wherein the first speaker is positioned in a first space in a venue and the second speaker is positioned in a second space of the venue; and

a mobile device with memory storing a database of fingerprints of a plurality of differing audio content, with a microphone for receiving an audio signal associated with the first audio content or the second audio content, and with a processor running at least one of a fingerprinting module and a watermarking module to determine a location of the mobile device by processing the received audio signal.

2. The system of claim 1, wherein the received audio signal comprises a watermark identifying the first speaker or the second speaker as a source of the received audio signal and wherein the watermarking module decodes the watermark to provide the location of the mobile device relative to the first or second speaker.

3. The system of claim 1, wherein the fingerprinting module determines the location by generating a fingerprint based on the audio signal, comparing the generated fingerprint with the fingerprints in the database to find a best matching fingerprint, and finding a location cross-referenced to the best matching fingerprint.

4. The system of claim 1, wherein the processor runs the fingerprinting module and the watermarking module in parallel to process the audio signal and wherein the location is provided by the watermarking module when a watermark is decoded from the audio signal by the watermarking module and provided by the fingerprinting module when decoding by the watermarking module fails.

5. The system of claim 1, wherein the processor initially only runs the fingerprinting module to provide the location and runs the watermarking to provide the location after the audio signal is locked based on processing by the fingerprinting module.

6. The system of claim 5, wherein the audio signal is locked when a fingerprint match achieved by the fingerprinting module has a confidence level greater than a predefined minimum confidence level.

7. The system of claim 5, wherein the processor switches the fingerprinting module off when the audio signal is locked and wherein the processor resumes use of the fingerprinting and switches the watermarking module off when the watermarking is unable to decode a watermark in the received audio signal.

8. The system of claim 5, wherein the processor switches the watermarking module off after a first watermark is decoded in the received audio signal and switches the fingerprinting module on to provide the location on a periodic basis.

9. The system of claim 8, wherein the fingerprinting is performed at an increased frequency upon a loss of signal lock.

10. An audio-based localization method for use on a mobile device, comprising:

receiving a first audio signal;

processing the first audio signal using fingerprinting to determine a location of the mobile device relative to a source of the audio signal;

when the first audio signal produces a lock, receiving a second audio signal and processing the second audio signal using watermarking to determine a location of the mobile device; and

when the first audio signal fails to produce the lock, receiving the second audio signal and processing the second audio signal using fingerprinting to determine a location of the mobile device.

11. The method of claim 10, wherein the audio signal produces the lock when the fingerprinting provides match results in excess of a predefined minimum confidence level.

12. The method of claim 10, further comprising, after the first audio signal produces the lock, continuing to receive and process audio signals using watermarking to provide locations of the mobile device until the lock is lost.

13. The method of claim 12, further comprising, after the lock is lost, receiving audio signals and processing the audio signals using fingerprinting until a new signal lock is achieved.

14. The method of claim 10, further comprising, after the first audio signal produces the lock, continuing to receive and process audio signals using watermarking to provide locations until a location beacon is received and decoded and then switching to use of fingerprinting to process received audio signals to provide locations for the mobile device.

15. The method of claim 14, wherein the use of the fingerprinting after the switching is performed at a reduced frequency compared to the processing of the first audio signal.

16. A mobile device adapted for localization, comprising:

a processor;

a memory device storing a database of fingerprints each associated with a different audio file;

a microphone receiving audio signals output from speakers of a venue;

a fingerprinting module run by the processor to process the audio signals to determine a location of the mobile device by creating fingerprints of a first subset of the received audio signals and comparing the created fingerprints for matches in the database of fingerprints in the memory device; and

a watermarking module run by the processor to determine a location of the mobile device by processing a second subset of the audio signals to identify and decode watermarks each providing an identification of one of the speakers.

17. The mobile device of claim 16, wherein the processor selectively runs only one of the fingerprinting module and watermarking module to determine the location of the mobile device.

18. The mobile device of claim 16, wherein the processor runs the fingerprinting module to provide a lock on one of the audio signals for which a fingerprint match confidence level exceeds a predefined minimum confidence level.

19. The mobile device of claim 16, wherein the processor runs the watermarking while a signal lock is maintained.

20. The mobile device of claim 16, wherein the processor runs the watermarking module until a location beacon is received and decoded and then switches to running the fingerprinting module in an intermittent manner until a new location is determined by the fingerprinting module.