PRE-PROCESSING MEDIA FOR AUDIO-BASED SYNCHRONIZATION

• US 20110208333A1
• Filed: 04/28/2011
• Published: 08/25/2011
• Est. Priority Date: 05/27/2009
• Status: Abandoned Application

• Current Assignee: Spot411 Technologies, Inc.
• Original Assignee: Spot411 Technologies, Inc.
• Inventors: Shah, Ajay, Glitsch, Hans M.
• Application Number:
  US13/096,943
• Publication Number:
  US 20110208333A1
• Time in Patent Office: Days
• US Class Current: 700/94
• CPC Class Codes: G06F 16/433 using audio data
• CPC Class Codes: G06F 16/4393 Multimedia presentations, e.g. slide shows, multimedia albums
• CPC Class Codes: H04N 21/2407 Monitoring of transmitted content, e.g. distribution time, number of downloads arrangements for monitoring programmes for broadcast or for distribution combined with broadcast H04H20/14
• CPC Class Codes: H04N 21/4394 involving operations for analysing the audio stream, e.g. detecting features or characteristics in audio streams arrangements characterised by components specially adapted for monitoring, identification or recognition of audio in broadcast systems H04H60/58

Abstract

An audio channel of a time-based media presentation provides a basis for synchronizing to the presentation across a variety of platforms independent of when and where the presentation is being viewed. By pre-processing the media into a series of non-unique hashes, and similarly processing an audio stream of the media captured at a client device, a comparison can be made that yields an accurate time offset within the presentation. The comparison may usefully be performed over a data network using a server that hosts data from the pre-processed media, and a variety of applications may be deployed on the client device based on the resulting synchronization.

Claims

1. A method comprising:

receiving an audio portion of a time-based media presentation with a microphone of a client device as a plurality of digital samples;

processing the plurality of digital samples to obtain a plurality of hashes, each one of the plurality of hashes non-uniquely corresponding to a portion of the time-based media presentation, and each one of the plurality of hashes having a relative time offset to each other one of the plurality of hashes; and

analyzing the plurality of hashes to determine a time offset within the time-based media presentation most closely corresponding to the plurality of hashes.

2. The method of claim 1 wherein analyzing the plurality of hashes includes transmitting the plurality of hashes to a server over a data network.

3. The method of claim 2 further comprising receiving from the server the time offset within the time-based media presentation.

4. The method of claim 2 further comprising receiving from the server one or more candidate offsets for each one of the plurality of hashes and locally processing the one or more candidate offsets to determine the time offset within the time-based media presentation.

5. The method of claim 2 further comprising transmitting supplemental information to the server including an identifier that uniquely identifies the time-based media presentation.

6. The method of claim 2 further comprising transmitting supplemental information to the server including a hash sequence number that identifies an order of each one of the plurality of hashes relative to one or more other ones of the plurality of hashes.

7. The method of claim 1 wherein processing the plurality of digital samples includes processing more than one of the plurality of digital samples to create each one of the plurality of hashes.

8. The method of claim 1 wherein processing the plurality of digital samples includes processing more than one of the plurality of digital samples to provide a calculated hash and determining a number of bitwise variations to the calculated hash to provide two or more of the plurality of hashes.

9. The method of claim 6 wherein the number of bitwise variations to the calculated hash includes each value with no more bitwise variations from the calculated hash than an allowable error rate.

10. The method of claim 1 wherein the client device includes at least one of a computer, a cellular phone, and a portable digital device.

11. The method of claim 1 wherein the time-based media presentation includes a live television broadcast.

12. The method of claim 1 wherein the time-based media presentation includes a time-shifted replay of a television broadcast.

13. The method of claim 1 wherein the time-based media presentation includes pre-recorded media from one or more of a CD, a DVD, a Blu-ray Disc, and an HDDVD.

14. The method of claim 1 wherein processing the plurality of digital samples includes filtering the plurality of digital samples with a low pass filter and transforming the resulting data with a discrete Fourier transform.

15. The method of claim 1 wherein processing the plurality of digital sample includes normalizing a magnitude of the plurality of digital samples.

16. The method of claim 1 further comprising synchronizing to the time-based media presentation and rendering supplemental content on the client device that is time-synchronized to the time-based media presentation.

17. The method of claim 16 wherein synchronizing includes synchronizing an application executing on the client device.

18. The method of claim 17 further comprising displaying an indication of synchronization status on the client device.

19. The method of claim 17 further comprising conditionally synchronizing the application to the time-based media presentation only when the time-offset can be determined with a predetermined certainty.

20. The method of claim 16 wherein the supplemental content includes an advertisement.

21. The method of claim 16 wherein the supplemental content includes content retrieved from a remote site through a data network.

22. A computer program product for synchronizing to media, the computer program product comprising computer executable code embodied on a non-transitory computer readable medium that, when executing on one or more computing devices, performs the steps of:

receiving an audio portion of a time-based media presentation with a microphone of a client device as a plurality of digital samples;

processing the plurality of digital samples to obtain a plurality of hashes, each one of the plurality of hashes non-uniquely corresponding to a portion of the time-based media presentation, and each one of the plurality of hashes having a relative time offset to each other one of the plurality of hashes; and

analyzing the plurality of hashes to determine a time offset within the time-based media presentation most closely corresponding to the plurality of hashes.