SYNCHRONIZING SECONDARY CONTENT TO A MULTIMEDIA PRESENTATION

• US 20110063503A1
• Filed: 07/06/2010
• Published: 03/17/2011
• Est. Priority Date: 07/06/2009
• Status: Abandoned Application

• Current Assignee: Sidecastr Co.
• Original Assignee: Sidecastr Co.
• Inventors: BRAND, STEVEN M., Gilbert, Andrew
• Application Number:
  US12/830,655
• Publication Number:
  US 20110063503A1
• Time in Patent Office: Days
• US Class Current: 348/500
• CPC Class Codes: H04N 21/23109 by placing content in organized collections, e.g. EPG data repository details of retrieval of video data and associated meta data in video databases G06F16/739
• CPC Class Codes: H04N 21/23424 involving splicing one content stream with another content stream, e.g. for inserting or substituting an advertisement
• CPC Class Codes: H04N 21/242 Synchronization processes, e.g. processing of PCR [Program Clock References] arrangements for synchronising broadcast or distribution via plural systems in broadcast distribution systems H04H20/18
• CPC Class Codes: H04N 21/426 Internal components of the client ; Characteristics thereof H04N5/44 takes precedence
• CPC Class Codes: H04N 21/4305 Synchronising client clock from received content stream, e.g. locking decoder clock with encoder clock, extraction of the PCR packets arrangements for synchronising receiver with transmitter by comparing receiver clock with transmitter clock H04L7/0012; arrangements for synchronising receiver with transmitter wherein the receiver takes measures against momentary loss of synchronisation H04L7/0083
• CPC Class Codes: H04N 21/43074 of additional data with content streams on the same device, e.g. of EPG data or interactive icon with a TV program
• CPC Class Codes: H04N 21/4788 communicating with other users, e.g. chatting arrangements for providing for computer conferences, e.g. chat rooms, to substation in data switching networks H04L12/1813; distributed application using peer-to-peer [P2P] networks H04L67/104
• CPC Class Codes: H04N 21/8106 involving special audio data, e.g. different tracks for different languages

Abstract

In various embodiments, secondary content synchronized to a multimedia presentation is delivered. An audio signal is sampled with a local application and transmitted to a remote server. The remote server determines secondary content associated with the audio sample and transmits the secondary content to the local application for display thereat.

Claims

1. A method for providing secondary content synchronized to a remotely-experienced multimedia presentation, the method comprising:

receiving, from a remote location, an audio sample of the multimedia presentation;

determining a temporal location, within the multimedia presentation, of the audio sample;

identifying secondary content based on the temporal location; and

causing delivery of the secondary content to the remote location, the secondary content being synchronized to the multimedia presentation.

2. The method of claim 1, further comprising identifying the multimedia presentation based at least in part on the audio sample.

3. The method of claim 2, wherein identifying the multimedia presentation comprises comparing the audio sample to a database of audio features.

4. The method of claim 1, wherein the audio sample is received from a device located where the multimedia presentation is experienced.

5. The method of claim 1, wherein the temporal location is determined based on an analysis of the audio sample.

6. The method of claim 1, wherein the multimedia presentation comprises a live TV program.

7. The method of claim 1, wherein the multimedia presentation comprises a time-shifted TV program.

8. The method of claim 1, further comprising analyzing the multimedia presentation, prior to the step of determining the temporal location, to facilitate locating of the audio sample within the multimedia presentation.

9. The method of claim 8, further comprising storing results of the analysis of the multimedia presentation in an audio features database.

10. The method of claim 8, wherein analyzing the multimedia presentation comprises at least one of feature extraction and indexing.

11. The method of claim 10, wherein feature extraction comprises at least one of pre-emphasizing audio content of the multimedia presentation, creating frames of samples of audio content of the multimedia presentation, extracting features of audio content of the multimedia presentation in a time domain, or extracting features of audio content of the multimedia presentation in a frequency domain.

12. The method of claim 1, wherein determining the temporal location comprises matching a pattern in the audio sample with a pattern in the multimedia presentation.

13. The method of claim 1, wherein the audio sample is received at a periodic interval, on an ad-hoc basis, or at a request from a user.

14. The method of claim 1, wherein identifying secondary content comprises querying a database of secondary content with the temporal location.

15. The method of claim 1, wherein the secondary content comprises at least one of live user-generated content and stored user-generated content.

16. A system for providing secondary content synchronized to a multimedia presentation, the system comprising:

computer memory for storing an audio sample of the multimedia presentation;

an audio-processing module for determining a temporal location, within the multimedia presentation, of the audio sample;

a content-processing module for identifying secondary content based on the temporal location; and

a transmitter for transmitting the secondary content to a remote location, the secondary content being synchronized to the multimedia presentation.

17. The system of claim 16, wherein the audio-processing module comprises at least one of a feature-extractor module or a time-indexing module.

18. The system of claim 17, wherein the feature-extractor module comprises at least one of a pre-emphasis filter, a window frame-builder module, a time-domain feature extractor, or a frequency-domain feature extractor.

19. The system of claim 16, further comprising a secondary-content server for hosting a database of secondary content, the database serving secondary content based on the determined temporal location.

20. The system of claim 16, wherein the interface module is hosted on at least one of a notebook computer, netbook computer, desktop computer, personal digital assistant, cellular phone, or handheld media player.

21. The system of claim 16, wherein the secondary content comprises at least one of live user-generated content and stored user-generated content.

22. A method for delivering secondary content synchronized to a multimedia presentation to a user, the method comprising:

sampling an audio portion of the multimedia presentation, thereby creating an audio sample;

transmitting the audio sample to a remote server;

receiving secondary content synchronized to the multimedia presentation, the secondary content based at least in part on the temporal location of the audio sample in the multimedia presentation;

delivering, via a user interface, the secondary content to the user.

23. The method of claim 22, wherein delivering the secondary content comprises one of displaying visual data or playing back audio data.

24. The method of claim 22, further comprising pre-processing the audio sample prior to transmission.

25. The method of claim 23, wherein pre-processing the audio sample comprises at least one of normalization or initial-feature extraction.

26. The method of claim 22, wherein the secondary content is delivered based at least in part on a user preference, a location of the user interface, or a screen size of the user interface.

27. The method of claim 22, further comprising varying the length of the audio sample.

28. The method of claim 22, wherein the secondary content comprises at least one of live user-generated content and stored user-generated content.

29. An article of manufacture comprising computer-readable instructions thereon for delivering secondary content to a user, the secondary content synchronized to a multimedia presentation, the article of manufacture comprising:

instructions to sample an audio portion of the multimedia presentation, thereby creating an audio sample;

instructions to transmit the audio sample to a remote server;

instructions to receive secondary content synchronized to the multimedia presentation, the secondary content based at least in part on the temporal location of the audio sample in the multimedia presentation;

instructions to deliver the secondary content to the user.

30. The article of claim 29, wherein delivering the secondary content comprises one of displaying visual data or playing back audio data.

31. The article of claim 29, further comprising instructions for pre-processing the audio sample prior to transmission.

32. The article of claim 31, wherein pre-processing the audio sample comprises at least one of normalization or initial-feature extraction.

33. The article of claim 29, wherein the secondary content is delivered based at least in part on a user preference, a location of the user interface, or a screen size of the user interface.

34. The article of claim 29, further comprising instructions for varying the length of the audio sample.

35. The method of claim 29, wherein the secondary content comprises at least one of live user-generated content and stored user-generated content.

36. A method for delivering secondary content synchronized to a multimedia presentation to a user, the method comprising:

sampling an audio portion of the multimedia presentation, thereby creating an audio sample;

determining a temporal location, within the multimedia presentation, of the audio sample;

identifying secondary content based on the temporal location; and

delivering, via a user interface, the secondary content to the user.

37. The method of claim 36, further comprising:

receiving, from a remote location, at least one of audio features corresponding to the multimedia presentation and secondary content corresponding to the multimedia presentation; and

storing at least one of the audio features and secondary content in a local database.

38. A system for providing secondary content synchronized to a multimedia presentation, the system comprising:

computer memory for storing an audio sample of the multimedia presentation;

a pre-process module for determining a temporal location, within the multimedia presentation, of the audio sample; and

a user interface for delivering secondary content corresponding to the temporal location to a user.

39. The system of claim 38, further comprising a local database for storing the secondary content.