Augmentation of streaming media

• US 20100121973A1
• Filed: 11/10/2009
• Published: 05/13/2010
• Est. Priority Date: 11/12/2008
• Status: Abandoned Application

• Current Assignee: Ramp Holdings, Inc. (Massachusetts)
• Original Assignee: Ramp Holdings, Inc. (Massachusetts)
• Inventors: Meteer, Marie, Lobacheva, Yuliya, Zinovieva, Nina
• Application Number:
  US12/590,533
• Publication Number:
  US 20100121973A1
• Time in Patent Office: Days
• US Class Current: 709/231
• CPC Class Codes: G06F 40/279 Recognition of textual entities
• CPC Class Codes: G06F 40/40 Processing or translation of natural language natural language analysis G06F40/20; semantic analysis G06F40/30
• CPC Class Codes: G06Q 30/00 Commerce
• CPC Class Codes: G06Q 30/0251 Targeted advertisements
• CPC Class Codes: G06Q 30/0256 User search
• CPC Class Codes: G06Q 30/0277 Online advertisement
• CPC Class Codes: G10L 15/183 using context dependencies, e.g. language models
• CPC Class Codes: G10L 15/26 Speech to text systems G10L15/08 takes precedence
• CPC Class Codes: G10L 19/0018 Speech coding using phonetic or linguistical decoding of the source; Reconstruction using text-to-speech synthesis
• CPC Class Codes: G10L 19/167 Audio streaming, i.e. formatting and decoding of an encoded audio signal representation into a data stream for transmission or storage purposes
• CPC Class Codes: H04L 65/70 Media network packetisation

Abstract

Methods and apparatus, including computer program products, for augmentation of streaming media. A method includes receiving streaming media, applying a speech-to-text recognizer to the received streaming media, identifying keywords, determining topics, and augmenting speech elements with one or more content items. The one or more content items cab be placed temporally to coincide with speech elements. The method can also include converting the speech elements into text and generating a text-searchable representation of the streaming media.

Claims

1. A method comprising:

receiving streaming media;

applying a speech-to-text recognizer to the received streaming media;

identifying keywords;

determining topics; and

augmenting speech elements with one or more content items.

2. The method of claim 1 wherein the one or more content items are placed temporally to coincide with non-speech elements.

3. The method of claim 1 wherein the one or more content items are selected based on topics.

4. The method of claim 1 wherein the one or more content items are selected based on one or more of the identified keywords.

5. The method of claim 1 wherein determining topics is based on one or more of the identified keywords.

6. The method of claim 1 wherein determining topics is based on derivation from a statistical categorization into a known taxonomy of topics, on derivation from a rules-based categorization into a known taxonomy of topics, on filtering from a list of keywords, or on composition from a list of keywords.

7. The method of claim 1 wherein the identified keywords are processed with keywords in editorial metadata associated with the received streaming media.

8. The method of claim 1 wherein determining topics is done in conjunction with editorial metadata associated with the received streaming media.

9. The method of claim 7 wherein the editorial metadata includes one or more of a title and description.

10. The method of claim 1 wherein the identified keywords are assigned a confidence score.

11. The method of claim 1 wherein the speech-to-text recognizer is a keyword spotter.

12. The method of claim 1 wherein identifying keywords comprises applying natural language processing (NLP) to closed captioning/editorial transcripts.

13. The method claim 1 wherein identifying keywords comprises applying one of statistical natural language processing (NLP), rules-based NLP, simple editorial keyword list processing, or statistical keyword list processing.

14. The method of claim 1 wherein augmenting is performed while the streaming media is playing or prior to the streaming media playing.

15. The method of claim 1 wherein augmenting comprises an insertion of the one or more content items into the streaming media or spliced into the streaming media by a video/audio player.

16. The method of claim 1 wherein the streaming media comprises a radio broadcast or Internet-streamed audio or video.

17. The method of claim 1 further comprising:

converting the speech elements into text; and

generating a text-searchable representation of the streaming media.

18. The method of claim 1 further comprising limiting the augmentation to a maximum number of content items.

19. The method of claim 18 wherein the maximum number of content items is one.

20. The method of claim 1 wherein the one or more content items are placed within the streaming media at a minimum temporal displacement from the speech elements on which selection of the content items is based.

21. The method of claim 1 wherein the content items comprise advertisements.

22. The method of claim 21 wherein advertisements are inserted within the streaming media itself or shown along side the streaming media on a Web page.

23. The method of claim 21 wherein the advertisements reside on an external source.

24. The method of claim 21 wherein the advertisements are selected by providing the topics as metadata to one or more external engines.

25. The method of claim 1 further comprising streaming the augmented media.

26. The method of claim 1 further comprising providing the keywords with the streamed augmented media.

27. A system comprising:

a media server configured to receive streaming media;

a speech processor for segmenting the streaming media into speech elements and non-speech audio elements, identifying keywords within the speech audio elements, and determining a topic based on one or more of the identified keywords; and

an augmentation server for augmenting the streaming media with one or more content items.

28. The method of claim 27 wherein the one or more content items are selected based on the topic and placed temporally to coincide with non-speech elements.

29. The system of claim 27 further comprising a database server for storing the content elements.

30. The system of claim 27 wherein the media server is further configured to transmit the augmented streaming media.

31. A method comprising:

receiving streaming media;

selecting a segment of the streaming media;

separating the selected segment into speech elements and non-speech audio elements;

identifying keywords within each of the speech elements;

determining a topic based on one or more of the identified keywords; and

augmenting the selected segment with one or more content items selected based on the topic and placed temporally

32. The method of claim 31 wherein the identified keywords are filtered using keywords in editorial metadata associated with the received streaming media.

33. The method of claim 32 wherein the editorial metadata includes one or more of a title and description.

34. The method of claim 31 wherein identifying keywords comprises applying full continuous speech-to-text processing.

35. The method of claim 31 wherein identifying keywords comprises applying a keyword spotter.

36. The method of claim 31 wherein identifying keywords comprises applying natural language processing (NLP) to closed captioning/editorial transcripts.

37. The method claim 31 wherein identifying keywords comprises applying one of statistical natural language processing (NLP), rules-based NLP, simple editorial keyword list processing, or statistical keyword list processing.

38. The method of claim 31 wherein augmenting is performed while the streaming media is playing or prior to the streaming media playing.

39. The method of claim 31 wherein the augmenting comprises an insertion of the one or more content items into the streaming media or spliced into the streaming media by a video/audio player.

40. The method of claim 31 wherein the non-speech audio elements comprise one or more of silence, applause, music, laughter, and background noise.

41. The method of claim 31 further comprising:

converting the speech elements into text; and

generating a text-searchable representation of the streaming media.

42. The method of claim 31 wherein the content items comprise advertisements.

43. The method of claim 42 wherein advertisements are inserted within the streaming media itself or shown along side the streaming media on a Web page.