SYNCHRONIZING SECONDARY CONTENT TO A MULTIMEDIA PRESENTATION

• US 20160073141A1
• Filed: 09/14/2015
• Published: 03/10/2016
• Est. Priority Date: 07/06/2009
• Status: Abandoned Application

• Current Assignee: Sidecastr Co.
• Original Assignee: Sidecastr Co.
• Inventors: Gilbert, Andrew, Brand, Steven M.
• Application Number:
  US14/853,375
• Publication Number:
  US 20160073141A1
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: H04N 21/23109 by placing content in organized collections, e.g. EPG data repository details of retrieval of video data and associated meta data in video databases G06F16/739
• CPC Class Codes: H04N 21/23424 involving splicing one content stream with another content stream, e.g. for inserting or substituting an advertisement
• CPC Class Codes: H04N 21/242 Synchronization processes, e.g. processing of PCR [Program Clock References] arrangements for synchronising broadcast or distribution via plural systems in broadcast distribution systems H04H20/18
• CPC Class Codes: H04N 21/426 Internal components of the client ; Characteristics thereof H04N5/44 takes precedence
• CPC Class Codes: H04N 21/4305 Synchronising client clock from received content stream, e.g. locking decoder clock with encoder clock, extraction of the PCR packets arrangements for synchronising receiver with transmitter by comparing receiver clock with transmitter clock H04L7/0012; arrangements for synchronising receiver with transmitter wherein the receiver takes measures against momentary loss of synchronisation H04L7/0083
• CPC Class Codes: H04N 21/43074 of additional data with content streams on the same device, e.g. of EPG data or interactive icon with a TV program
• CPC Class Codes: H04N 21/4788 communicating with other users, e.g. chatting arrangements for providing for computer conferences, e.g. chat rooms, to substation in data switching networks H04L12/1813; distributed application using peer-to-peer [P2P] networks H04L67/104
• CPC Class Codes: H04N 21/8106 involving special audio data, e.g. different tracks for different languages

Abstract

In various embodiments, secondary content synchronized to a multimedia presentation is delivered. An audio signal is sampled with a local application and transmitted to a remote server. The remote server determines secondary content associated with the audio sample and transmits the secondary content to the local application for display thereat.

Claims

1-39. -39. (canceled)

40. A method of synchronizing delivery of secondary content compatible with a plurality of multimedia and social network sources to a content display device , the secondary content associated with and synchronized with at least one of an original transmission and a previous transmission of a multimedia presentation, the method comprising:

sampling and extracting, in real-time and at a first temporal location, an audio sample from the multimedia presentation;

determining, based at least in part on the audio sample, a second temporal location within the at least one of the original transmission and the previous transmission, the second temporal location corresponding to the first temporal location of the multimedia presentation;

identifying secondary content referenced to the second temporal location of the at least one of the original transmission and the previous transmission;

synchronizing, in real-time and at the first temporal location, the identified secondary content associated with the at least one of the original transmission and the previous transmission to the multimedia presentation; and

transmitting the identified secondary content to the content display device over a network such that the identified secondary content associated with the at least one of the original transmission and the previous transmission can be viewed on the content display device in synchronization with the multimedia presentation.

41. The method of claim 40 further comprising referencing, during at least one of the original presentation and the previous presentation, secondary content to the second temporal location in the multimedia presentation, based at least in part on an audio portion of the multimedia presentation corresponding to the second temporal location.

42. The method of claim 40, wherein sampling and extracting comprises transmitting the audio sample to a remote server.

43. The method of claim 40, wherein transmitting comprises selectively transmitting the identified secondary content to the content display device of discrete users.

44. The method of claim 43 further comprising, based on a request from a discrete user, transmitting less of identified secondary content.

45. The method of claim 43 further comprising, based on a request from a discrete user, transmitting only desired media from the identified secondary content.

46. The method of claim 43 further comprising, based on a request from a discrete user, transmitting only a desired media from the identified secondary content.

47. The method of claim 40 further comprising selectively transmitting secondary content based at least in part on at least one of a user profile, user preferences, and a user account.

48. The method of claim 40 further comprising analyzing some portion of an entire media presentation before sampling to facilitate extracting the extracted audio content from the multimedia presentation.

49. The method of claim 48, wherein sampling is only performed on an analyzed portion of the multimedia presentation.

50. The method of claim 40 further comprising updating the secondary content by adding input from a time-shift user.

51. The method of claim 40, wherein determining the second temporal location comprises matching the audio sample to a multiplicity of stored audio features for the at least one of original transmission and previous transmission of the multimedia presentation.

52. The method of claim 40, wherein the content display device displays the multimedia presentation and the identified secondary content.

53. The method of claim 40, wherein the content display device displays the identified secondary content and a second display device display the multimedia presentation.

54. The method of claim 40 further comprising time-indexing each second temporal location to provide a length of time elapsed from a beginning of the multimedia presentation.

55. The method of claim 40, wherein determining the second temporal location comprises identifying, based at least in part on the audio sample, at least one of the original transmission and the previous transmission of the multimedia presentation.

56. The method of claim 40 further comprising adjusting transmission of the synchronized secondary content after an interruption in displaying the multimedia presentation.

57. The method of claim 40, wherein sampling and extracting an audio sample comprises performing a Fast Fourier Transform on the audio sample to generate a digital audio fingerprint of the audio sample.

58. The method of claim 57, wherein determining a second temporal location comprises matching the digital audio fingerprint of an audio feature of at least one of the original transmission and the previous transmission of the multimedia presentation.

59. A system for synchronizing deliver of content to a content delivery device during a multimedia presentation, the system comprising:

a local server for sampling and extracting in real-time and at a first temporal location, an audio sample from the multimedia presentation;

computer memory for receiving and storing the audio sample of the multimedia presentation;

an audio-processing module for identifying, based on the audio sample, the multimedia presentation and for determining a second temporal location of the audio sample in at least one of an original transmission and a previous transmission of the multimedia presentation;

a content-processing module for identifying, based on the second temporal location of the audio sample, secondary content referenced to the second temporal location of the at least one of the original transmission and the previous transmission; and

a transmitter for transmitting the identified secondary content to the local server, the identified secondary content being synchronized to the multimedia presentation.

60. The system of claim 59, wherein at least one of the audio-processing module and the content-processing module is disposed in a server remote from the local server.

61. The system of claim 60, wherein the remote server comprises at least one of an audio-feature database comprising a multiplicity of audio features selected from a multiplicity of original multimedia presentations and a secondary-content database comprising a multiplicity of secondary content, each of the secondary content referenced to at least one of the audio features.

62. The system of claim 59 further comprising a user interface to enable a user to request selective desired media from the identified secondary content.

63. The system of claim 59 further comprising a display device in communication with the local server.

65. The system of claim 63 further comprising at least one filter to only display secondary content in accordance with expressed preferences of the user.

66. The system of claim 59, wherein the secondary content comprises at least one of audio, video, text messages, advertisements, and images.

67. The system of claim 59, wherein the local server transmits the audio sample to the computer memory via at least one of the Internet, a wired network, a wireless network and a cellular phone network.

68. The system of claim 59, wherein, based on a bandwidth of the local server, the local server is adapted to request adjusting at least one of a sampling rate and a frame rate.

69. The system of claim 68, further comprising a pre-processing module executing on the content display device for extracting the extracted audio content from the audio sample of the multimedia presentation.

70. The system of claim 69, wherein the pre-processing module further comprises feature-extractor module, which further comprises at least one of a pre-empt filter, a window frame-builder module, a time-domain feature extractor, or a frequency-domain feature extractor.

71. The system of claim 59 further comprising a secondary-content server for hosting a database of secondary content, the database serving secondary content based on the determined temporal location.

72. The system of claim 59, wherein the content display device comprises one of a notebook computer, netbook computer, desktop computer, personal digital assistant, cellular phone, or handheld media player.

73. The system of claim 59, wherein the identified secondary content comprises at least one of live user-generated content and stored user-generated content.

74. The system of claim 59, wherein sampling and extracting an audio sample comprises performing a Fast Fourier Transform on the audio sample to generate a digital audio fingerprint of the audio sample.

75. The system of claim 74, wherein determining a second temporal location comprises matching the digital audio fingerprint of an audio feature of at least one of the original transmission and the previous transmission of the multimedia presentation.

64. The system of claim 64, wherein the display device displays the multimedia presentation and the content display device displays the identified secondary content.