System and method for automatically authoring interactive television content

• US 8,646,006 B2
• Filed: 10/10/2011
• Issued: 02/04/2014
• Est. Priority Date: 12/27/2002
• Status: Expired due to Fees

• Current Assignee: AT&T Corporation (AT&T, Inc.)
• Original Assignee: AT&T Intellectual Property I LP (AT&T, Inc.)
• Inventors: Gibbon, David Crawford, Liu, Zhu, Renger, Bernard Simon, Shahraray, Behzad, Weinraub, Mitchell J., Morrison, Stephanie A., Begeja, Lee
• Primary Examiner(s): Chug, Jason J
• Assistant Examiner(s): Parra, Omar S
• Application Number:
  US13/270,059
• Publication Number:
  US 20120030713A1
• Time in Patent Office: 848 Days
• Field of Search: None
• US Class Current: 725/53
• CPC Class Codes: G11B 27/034 on discs G11B27/036, G11B27/038 take precedence
• CPC Class Codes: G11B 27/28 by using information signals recorded by the same method as the main recording G11B27/22 takes precedence
• CPC Class Codes: H04N 21/233 Processing of audio elementary streams monitoring, identification or recognition of audio in broadcast systems H04H60/58
• CPC Class Codes: H04N 21/23418 involving operations for analysing video streams, e.g. detecting features or characteristics television picture signal circuitry for scene change detection H04N5/147; filtering for image enhancement G06T5/00; methods or arrangements for recognising scenes G06V20/00; arrangements characterised by components specially adapted for monitoring, identification or recognition of video in broadcast systems H04H60/59
• CPC Class Codes: H04N 21/235 Processing of additional data, e.g. scrambling of additional data or processing content descriptors
• CPC Class Codes: H04N 21/4722 for requesting additional data associated with the content
• CPC Class Codes: H04N 21/8545 for generating interactive applications
• CPC Class Codes: H04N 21/858 Linking data to content, e.g. by linking an URL to a video object, by creating a hotspot

Abstract

A system and method is provided to automatically generate content for ITV products and services by processing primary media sources. In one embodiment of the invention, keywords are automatically extracted from the primary media sources using one or more of a variety of techniques directed to video, audio and/or textual content of the multimodal source. In some embodiments, keywords are then processed according to one or more disclosed algorithms to narrow the quantity of downstream processing that is necessary to associate secondary sources (reference items) with the primary video source. Embodiments of the invention also provide automatic searching methods for the identification of reference items based on the processed keywords in order to maximize the value added by the association of reference items to the video source.

Claims

1. A method for generating interactive television content, comprising:

determining, by a processor, a first plurality of keywords from a portion of a media source, wherein the first plurality of keywords is generated from a language analysis process on words collected from the portion of the media source;

selecting, by the processor, a second plurality of keywords from the first plurality of keywords, wherein the selecting the second plurality of keywords comprises scoring each of the first plurality of keywords based on a type of collection process that determined each of the corresponding first plurality of keywords, wherein a first type of collection process is always scored higher than a second type of collection process, wherein the second plurality of keywords is determined based upon one or more of the first plurality of keywords that have a score above a threshold;

searching, by the processor, a memory to identify a plurality of reference items related to the media source based upon the second plurality of keywords, wherein the plurality of reference items is external to the media source;

embedding, by the processor, the plurality of reference items into the media source; and

delivering, by the processor, the media source embedded with the plurality of reference items to a customer premises.

2. The method of claim 1, wherein the determining the first plurality of keywords comprises:

determining whether the media source includes a closed caption text;

decoding the closed caption text when the media source includes the closed caption text; and

extracting an audio signal and performing a speech-to-text conversion on the audio signal that is extracted when the media source does not include the closed caption text.

3. The method of claim 1, wherein the determining the first plurality of keywords comprises performing an on-screen text recognition.

4. The method of claim 1, wherein the determining the first plurality of keywords comprises performing a shape analysis on a video frame of the media source.

5. The method of claim 1, wherein the determining the first plurality of keywords comprises:

detecting a face on a video frame of the media source; and

performing a face recognition on the face.

6. The method of claim 1, wherein the searching the memory is performed for each of the first plurality of keywords.

7. The method of claim 6, wherein the searching the memory comprises:

querying a news clip directory where a keyword from the first plurality of keywords is related to a current event; and

querying an encyclopedia where the keyword from the first plurality of keywords is not related to a current event.

8. The method of claim 6, wherein the searching the memory comprises:

querying a primary reference list; and

querying a secondary reference list when a quantity of the plurality of reference items does not exceed a threshold value.

9. The method of claim 1, wherein the searching the memory is performed for each of the second plurality of keywords.

10. The method of claim 9, wherein the searching the memory comprises:

querying a news clip directory where a keyword from the second plurality of keywords is related to a current event; and

querying an encyclopedia where the keyword from the second plurality of keywords is not related to a current event.

11. The method of claim 9, wherein the searching the memory comprises:

querying a primary reference list; and

querying a secondary reference list when a quantity of the plurality of reference items does not exceed a threshold value.

12. An apparatus for generating interactive television content, comprising:

a processor; and

a computer-readable medium storing a plurality of instructions which, when executed by the processor, cause the processor to perform operations, the operations comprising;

determining a first plurality of keywords from a portion of a media source, wherein the first plurality of keywords is generated from a language analysis process on words collected from the portion of the media source;

selecting a second plurality of keywords from the first plurality of keywords, wherein the selecting the second plurality of keywords comprises scoring each of the first plurality of keywords based on a type of collection process that determined each of the corresponding first plurality of keywords, wherein a first type of collection process is always scored higher than a second type of collection process, wherein the second plurality of keywords is determined based upon one or more of the first plurality of keywords that have a score above a threshold;

searching a memory to identify a plurality of reference items related to the media source based upon the second plurality of keywords, wherein the plurality of reference items are external to the media source;

embedding the plurality of reference items into the media source; and

delivering the media source embedded with the plurality of reference items to a customer premises.

13. The apparatus for generating interactive television content of claim 12, wherein the determining the first plurality of keywords determines whether the media source includes a closed caption text, decodes the closed caption text when the media source includes the closed caption text and extracts an audio signal and performing a speech-to-text conversion on the audio signal that is extracted when the media source does not include the closed caption text.

14. The apparatus for generating interactive television content of claim 12, wherein the determining the first plurality of keywords comprises performing an on-screen text recognition.

15. The apparatus for generating interactive television content of claim 12, wherein the determining the first plurality of keywords comprises performing a shape analysis on a video frame of the media source.

16. The apparatus for generating interactive television content of claim 12, wherein the determining the first plurality of keywords detects a face on a video frame of the media source and performs a face recognition on the face.

17. The apparatus for generating interactive television content of claim 12, wherein the searching the memory queries a news clip directory where a keyword from the first plurality of keywords is related to a current event, and queries an encyclopedia where the keyword from the first plurality of keywords is not related to a current event.

18. A system for generating interactive television content, comprising:

an interface to a media source;

an interface to a database, the database having a list of reference items, wherein the reference items are external to the media source; and

an application server in communication with the interface to the database and the interface to the media source, the application server for determining a plurality of first key words from a portion of the media source, wherein the first plurality of keywords is generated from a language analysis process on words collected from the portion of the media source, for selecting a second plurality of keywords from the first plurality of keywords, wherein the selecting the second plurality of keywords comprises scoring each of the first plurality of keywords based on a type of collection process that determined each of the corresponding first plurality of keywords, wherein a first type of collection process is always scored higher than a second type of collection process, wherein the second plurality of keywords is determined based upon one or more of the first plurality of keywords that have a score above a threshold, for searching the database to identify a plurality of reference items related to the media source based upon the second plurality of keywords, for embedding the plurality of reference items into the media source, and for delivering the media source embedded with the plurality of reference items to a customer premises.

19. The system of claim 18, wherein the media source includes video information.

20. The system of claim 18, wherein the application server is further for searching a reference item using the plurality of second keywords.