Context Relevant Interactive Television

• US 20130139209A1
• Filed: 05/18/2012
• Published: 05/30/2013
• Est. Priority Date: 11/28/2011
• Status: Active Grant

• Current Assignee: R2 Solutions LLC (Acacia Research Corporation)
• Original Assignee: Yahoo! Inc. (Apollo Global Management, Inc.)
• Inventors: Urrabazo, Roger, La, John, Rice, Dave, Glidden, Todd R., Raman, Vishwas
• Granted Patent: US 9,866,915 B2
• Time in Patent Office: Days
• US Class Current: 725/109
• CPC Class Codes: H04N 21/43072 of multiple content streams on the same device
• CPC Class Codes: H04N 21/4394 involving operations for analysing the audio stream, e.g. detecting features or characteristics in audio streams arrangements characterised by components specially adapted for monitoring, identification or recognition of audio in broadcast systems H04H60/58
• CPC Class Codes: H04N 21/4828 for searching program descriptors retrieval of video data G06F16/739

Abstract

Methods, systems and computer readable medium with program instructions for providing contextually relevant information for multimedia content streaming through a rendering device, includes detecting a selection of the multimedia content for rendering on the rendering device. An audio signal of the multimedia content is retrieved and analyzed to generate an audio fingerprint. The generated audio fingerprint is used to identify the multimedia content that is selected for rendering. Contextually relevant content for a portion of the multimedia content being rendered on the rendering device is dynamically retrieved from one or more sources and mapped to data associated with the portion of the multimedia content. The mapping results in the rendering of the relevant content during the rendering of the portion of the multimedia content and provides additional information that is contextually relevant to the portion of the multimedia content that is currently being rendered on the rendering device.

Claims

1. A method for providing contextually relevant information for multimedia content streaming through a rendering device, comprising:

detecting a selection of the multimedia content for rendering on the rendering device;

retrieving an audio signal of the multimedia content, the audio signal analyzed to generate an audio fingerprint;

identifying the multimedia content using the audio fingerprint;

dynamically retrieving content that is contextually relevant to a portion of the multimedia content currently being rendered on the rendering device, from one or more sources; and

mapping the content retrieved from the one or more sources to data associated with the portion of the multimedia content currently being rendered on the rendering device, the mapping results in rendering of the content during the rendering of the portion of the multimedia content, wherein the content retrieved from the one or more sources provides additional information that is contextually relevant to data related to the portion of the multimedia content that is currently rendered on the rendering device.

2. The method of claim 1, wherein identifying the multimedia content further includes,

partitioning the audio signal into a plurality of segments of small intervals;

analyzing a particular segment to identify acoustic modulations in the particular segment, the analysis generating a distinct vector for the particular segment based on the acoustic modulation, the vector defining the audio fingerprint that is unique for the particular segment of the audio signal; and

querying a content database on a server using the vector of the particular segment of audio signal to obtain content information for multimedia content that matches the fingerprint of the particular segment, the content information obtained from a content provider providing details related to the multimedia content that matches the audio signal received for rendering.

3. The method of claim 1, wherein mapping further includes,

defining a trigger event for the multimedia content based on the context of the multimedia content;

providing a video offset at the portion of the multimedia content where the trigger event occurs based on the context, wherein the video offset is defined by temporally tagging the portion of the multimedia content;

retrieving the content from the one or more sources based on the context of the multimedia content, the retrieved content collected and stored in a content repository for subsequent retrieval and rendering; and

mapping the content from the one or more sources to the video offset at the portion of the multimedia content.

4. The method of claim 3, further includes storing the retrieved content from the one or more sources in a content repository for subsequent retrieval and rendering.

5. The method of claim 3, further includes,

providing an indicator for the content from the one or more sources, the indicator indicative of presence of additional contextually relevant information for the multimedia content currently being rendered at the rendering device.

6. The method of claim 3, wherein defining the trigger event further includes,

analyzing the multimedia content being rendered on the rendering device to determine current context of the multimedia content based on previous context associated with the multimedia content;

and

generating an event trigger based on the current context as related to the previous context.

7. The method of claim 6, wherein content retrieved from the repository further includes,

analyzing the content to determine freshness factor, the freshness factor defined using a threshold value and is indicative of age of the content;

when the freshness factor exceeds the threshold value,

discarding the content from one or more sources; and

updating content from the one or more sources related to the multimedia content based on the context of the multimedia content.

8. The method of claim 1, wherein when the multimedia content does not include any content from the one or more sources, rendering a default content for the multimedia content based on the context of the multimedia content.

9. The method of claim 1, further includes,

detecting a switch in the multimedia content rendered on the rendering device;

automatically discarding content retrieved from the one or more sources for the multimedia content previously rendered on the rendering device; and

dynamically retrieving content from the one or more sources for the multimedia content based on context and audio fingerprint generated for newly switched multimedia content.

10. A method for providing contextually relevant information for multimedia content streaming through a rendering device, comprising:

detecting a selection of the multimedia content for rendering on the rendering device, the multimedia content selected is a pre-recorded content;

retrieving an audio signal of the multimedia content, the audio signal analyzed to generate an audio fingerprint;

identifying the multimedia content using the audio fingerprint;

retrieving content that is contextually relevant to a portion of the multimedia content currently rendered on the rendering device, from a content repository, wherein the content repository includes content collected from a plurality of sources for the multimedia content and mapped to different portions of the multimedia content based on context of the multimedia content; and

rendering the content retrieved from the content repository during rendering of the multimedia content, wherein the content retrieved from the content repository provides additional information that is contextually relevant to the portion of the multimedia content that is currently rendered on the rendering device.

11. The method of claim 10, wherein identifying the multimedia content further includes,

partitioning the audio signal into a plurality of segments of small intervals;

analyzing a particular segment to identify acoustic modulations in the particular segment, the analysis generating a distinct vector for the particular segment based on the acoustic modulation, the vector defining the audio fingerprint that is unique for the particular segment of the audio signal; and

querying a content database on a server using the vector of the particular segment of audio signal to obtain content information for multimedia content that matches the fingerprint of the particular segment, the content information obtained from a content provider providing details related to the multimedia content matching the audio signal.

12. The method of claim 10, wherein mapping further includes,

identifying a trigger event at the portion of the multimedia content based on the context of the multimedia content, the trigger event identified using a video offset defined by a temporal tag at the portion of the multimedia content that is currently rendered on the rendering device;

and

retrieving the content mapped to the portion of the multimedia content from the content repository based on the video offset.

13. The method of claim 12, wherein content retrieved from the content repository further includes,

analyzing the content to determine freshness factor, the freshness factor is indicative of age of the content;

when the freshness factor exceeds a threshold value,

updating the content associated with the portion of the multimedia content in the content repository, from the one or more sources based on the context of the multimedia content.

14. The method of claim 12, further includes,

dynamically retrieving content that is contextually relevant to a portion of the multimedia content currently being rendered on the rendering device, from one or more sources;

adding the content retrieved from the sources to the content retrieved from the content repository to obtain composite content;

rendering the composite content on the rendering device during the rendering of the multimedia content, the composite content providing updated information related to the trigger event at the portion of the multimedia content.

15. The method of claim 14, wherein the content retrieved from the one or more sources includes any one or combination of video, audio, image, or promotional media related to the trigger event.

16. A method for providing contextually relevant information for multimedia content streaming through a rendering device, comprising:

detecting a selection of the multimedia content for rendering on the rendering device, the multimedia content selected is a pre-recorded content;

retrieving an audio signal of the multimedia content, the audio signal analyzed to generate an audio fingerprint;

identifying the multimedia content using the audio fingerprint;

retrieving content that is contextually relevant to a portion of the multimedia content currently rendered on the rendering device, from a content repository and dynamically from one or more sources, wherein the content repository includes content collected from a plurality of sources during prior rendering of the multimedia content and mapped to different portions of the multimedia content based on context of the multimedia content; and

rendering the retrieved content during rendering of the portion of the multimedia content, wherein the retrieved content provides additional information that is contextually relevant to the portion of the multimedia content that is currently rendered on the rendering device.

17. The method of claim 16, further includes,

mapping the retrieved content from the one or more sources to data associated with the portion of the multimedia content currently being rendered on the rendering device, the mapping results in updating the content stored in the content repository related to the portion of the multimedia content with the retrieved content.