DEVICE WITH AUDIO-BASED MEDIA SYNCHRONIZATION

• US 20110202156A1
• Filed: 04/28/2011
• Published: 08/18/2011
• Est. Priority Date: 05/27/2009
• Status: Abandoned Application

• Current Assignee: Spot411 Technologies, Inc.
• Original Assignee: Spot411 Technologies, Inc.
• Inventors: Shah, Ajay, Glitsch, Hans M.
• Application Number:
  US13/096,920
• Publication Number:
  US 20110202156A1
• Time in Patent Office: Days
• US Class Current: 700/94
• CPC Class Codes: G06F 16/433 using audio data
• CPC Class Codes: G06F 16/4393 Multimedia presentations, e.g. slide shows, multimedia albums
• CPC Class Codes: H04N 21/2407 Monitoring of transmitted content, e.g. distribution time, number of downloads arrangements for monitoring programmes for broadcast or for distribution combined with broadcast H04H20/14
• CPC Class Codes: H04N 21/4394 involving operations for analysing the audio stream, e.g. detecting features or characteristics in audio streams arrangements characterised by components specially adapted for monitoring, identification or recognition of audio in broadcast systems H04H60/58

Abstract

An audio channel of a time-based media presentation provides a basis for synchronizing to the presentation across a variety of platforms independent of when and where the presentation is being viewed. By pre-processing the media into a series of non-unique hashes, and similarly processing an audio stream of the media captured at a client device, a comparison can be made that yields an accurate time offset within the presentation. The comparison may usefully be performed over a data network using a server that hosts data from the pre-processed media, and a variety of applications may be deployed on the client device based on the resulting synchronization.

Claims

1. A device comprising:

a microphone that receives an audio portion of a time-based media presentation and converts the audio portion into electrical signals;

an analog-to-digital converter coupled to the microphone that receives the electrical signals and provides a sequence of digital samples of the audio portion;

a network interface for communicating over a data network;

a processor coupled to the network interface and the analog-to-digital converter, the processor including processing circuitry configured to perform the steps of processing the sequence of digital samples to provide a plurality of hashes, each one of the plurality of hashes including a plurality of bits, each one of the plurality of hashes providing a non-unique representation of a segment of the audio portion, and each one of the plurality of hashes having a known relative time offset to each other one of the plurality of hashes, and the processor further configured to transmit the plurality of hashes and a unique identifier for the time-based media presentation to a server through the network interface and to receive from the server a time offset indicative of a current time offset within the time-based media presentation; and

a display under control of the processor that renders an output synchronized to the time-based media presentation according to the time offset.

2. The device of claim 1 wherein the microphone, the analog-to-digital converter, the processor, the network interface, and the display are integrated into a housing for at least one of a mobile device, a cellular phone, a laptop computer, a notebook computer, and a netbook.

3. The device of claim 1 wherein the time-based media presentation includes one or more of a movie, a sports event, and a television broadcast.

4. The device of claim 1 wherein adjacent ones of the plurality of hashes are calculated from overlapping windows of the sequence of digital samples.

5. The device of claim 1 wherein processing includes downsampling the sequence of digital samples to about five thousand five hundred Hertz.

6. The device of claim 1 wherein processing includes filtering the sequence of digital samples with a low pass filter to provide a filtered output and transforming the filtered output with a discrete Fourier transform.

7. The device of claim 1 wherein processing includes normalizing a magnitude of the sequence of digital samples.

8. The device of claim 1 wherein the server is configured to perform the steps of:

determining an allowable bit error for the plurality of hashes;

identifying for each one of the plurality of hashes a set of candidate hashes with a number of bitwise variations from the one of the plurality of hashes no greater than the allowable bit error;

locating any candidate time offsets in the time-based media presentation corresponding to each set of candidate hashes for each one of the plurality of hashes;

updating scores for the candidate time offsets corresponding to each set of candidate hashes;

selecting one of the candidate time offsets having a best one of the scores as the time offset within the time-based media presentation; and

responding to the network interface with the time offset.

9. The device of claim 8 wherein the allowable bit error is a variable bit error received by the server from the network interface.

10. The device of claim 9 wherein each one of the plurality of hashes consists of thirty two bits and the allowable bit error is eight bits.

11. The device of claim 8 wherein identifying a set of candidate hashes includes providing a binary tree of all possible values for the hash and traversing the binary tree in a manner that excludes branches for binary values that exceed the allowable bit error for the hash.

12. The device of claim 8 wherein transmitting the time offset includes conditionally transmitting the time offset only when the best one of the scores exceeds a predetermined threshold.

13. The device of claim 8 wherein the processor is further configured to transmit supplemental information to the server including a hash sequence number that identifies an order of the plurality of hashes relative to one another.

14. The device of claim 8 wherein synchronizing includes displaying an indicator on the client device that indicates a synchronization with the time-based media presentation.

15. The device of claim 8 wherein the processor is further configured to render additional content on the display under control of the processor, the additional content synchronized to the time-based media presentation.

16. The device of claim 15 wherein the additional content includes one or more of a supplemental video stream, contextual information, advertising, and interactive content.

17. A device comprising:

a microphone that receives an audio portion of a time-based media presentation and converts the audio portion into electrical signals;

an analog-to-digital converter coupled to the microphone that receives the electrical signals and provides a sequence of digital samples of the audio portion;

a processor coupled to the analog-to-digital converter, the processor including processing circuitry configured to perform the steps of processing the sequence of digital samples to obtain a plurality of hashes, each one of the plurality of hashes non-uniquely corresponding to a portion of the time-based media presentation, and each one of the plurality of hashes having a relative time offset to each other one of the plurality of hashes, and analyzing the plurality of hashes to determine a time offset within the time-based media presentation most closely corresponding to the plurality of hashes, and to provide an output synchronized to the time offset within the time-based media; and

a display that renders the output.

18. The device of claim 17 further comprising a data network interface, wherein the processing circuitry is further configured to transmit the plurality of hashes to a remote server through the data network interface and to receive through the data network interface from the server data indicative of the time offset.