MULTIMEDIA ACCESS

• US 20100274667A1
• Filed: 04/24/2009
• Published: 10/28/2010
• Est. Priority Date: 04/24/2009
• Status: Abandoned Application

• Current Assignee: Nexidia Incorporated (Nice Ltd)
• Original Assignee: Nexidia Incorporated (Nice Ltd)
• Inventors: Lanham, Drew, Willcutts, John, Gavalda, Marsal, Edwards, Gordon
• Application Number:
  US12/429,218
• Publication Number:
  US 20100274667A1
• Time in Patent Office: Days
• US Class Current: 705/14.490
• CPC Class Codes: G06F 16/433 using audio data
• CPC Class Codes: G06F 16/48 Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
• CPC Class Codes: G06Q 30/02 Marketing; Price estimation or determination; Fundraising
• CPC Class Codes: G06Q 30/0251 Targeted advertisements
• CPC Class Codes: G10L 15/26 Speech to text systems G10L15/08 takes precedence

Abstract

A computer-implemented method provides access to multimedia content, which include units of content that include audio components. Meta data for the units of content is formed to an association of key phrases detected in the audio components and the units. In some examples, forming the meta data includes determining a candidate set of key phrases associated with the unit of multimedia and searching for the presence of the candidate key phrases in the audio components. Forming the meta data then includes forming data representing the presence of key phrases in the audio components.

Claims

1. A computer-implemented method for providing access to multimedia content, the method comprising:

accepting audio components associated with a plurality of units of multimedia content, each unit of content including one of the audio components; and

for each unit of the plurality of units of multimedia content, forming meta data associated with said unit, the meta data for the units including an association of key phrases detected in the audio components and the units;

wherein forming the meta data comprises

determining a candidate set of key phrases associated with the unit of multimedia;

searching for presence of the candidate key phrases in the audio components, and

forming data representing the presence of key phrases in the audio components.

2. The method of claim 1 wherein the set of key phrases associated with the unit comprises a universal set of key phrases selected independently of the unit.

3. The method of claim 1 further comprising adding the units of multimedia content to a library of content and wherein the accepting of the audio components of a unit of content is at the time the unit is added to the library, and wherein at least some of the forming of the meta data for the unit is performed at the time the unit is added to the library.

4. The method of claim 1 wherein determining a candidate set of key phrases associated with the unit of multimedia comprises:

accepting the candidate key phrases in conjunction with receiving of the audio component.

5. The method of claim 4 wherein the candidate key phrases are accepted from a party providing the unit of multimedia.

6. The method of claim 1 wherein determining a candidate set of key phrases associated with the unit of multimedia comprises:

identifying text associated with the units of multimedia; and

selecting key phrases according to the identified text.

7. The method of claim 6 wherein identifying the text associated with a unit of the multimedia comprises:

accepting the text in conjunction with receiving of the audio component.

8. The method of claim 7 wherein the text is accepted from a party providing the unit of multimedia.

9. The method of claim 6 wherein identifying the text associated with a unit of the multimedia comprises:

identifying a text component that is linked to the unit of multimedia.

10. The method of claim 9 wherein the text component is linked to the unit of multimedia via a hyperlink.

11. The method of claim 6 wherein identifying the text associated with a unit of the multimedia comprises:

identifying text embedded in a video component of the unit of multimedia.

12. The method of claim 1 wherein determining a candidate set of key phrases associated with the unit of multimedia comprises:

determining one or more classes associated with the unit; and

selecting key phrases associated with the determined one or more classes.

13. The method of claim 11 wherein the classes comprise topics.

14. The method of claim 12 wherein determining the one or more classes associated with the unit is performed at the time that the unit is added to a library, and wherein selecting key phrases associated with the determined classes, searching for the presence of the selected key phrases, and updating the meta data according to the searching is repeated multiple times subsequent to determining the classes.

15. The method of claim 11 wherein determining the one or more classes associated with the unit comprises:

accessing a data representation of a plurality of classes and relationships between the classes;

identifying a first set of candidate classes from the representations;

for each candidate class in the first set, searching for presence of the key phrases associated with said candidate class in the audio component of the unit of content, and determining an association of the unit with said class according to determined presence of the key phrases in the audio component.

16. The method of claim 15 wherein determining the one or more classes associated with the unit further comprises, repeating at least once:

identifying a further set of candidate classes according to the determined association of the unit with classes, searching for presence of the key phrases associated with said further classes, and determining an association of the unit with said classes according to determined presence of the key phrases.

17. The method of claim 1 further comprising, for each of the units of content:

processing the audio components to a preprocessed form suitable for searching for presence of key phrases; and

storing the preprocessed form of the audio in association with the unit of content.

18. The method of claim 17 wherein processing the audio components to the preprocessed form comprises forming phonetically-based representations of the audio components.

19. The method of claim 17 wherein subsequent searching for key phrases in the audio component comprises searching the preprocessed form for the key phrases.

20. The method of claim 17 wherein storing the preprocessed form comprises forming an integrated data representation of the unit of content that includes the audio component and the preprocessed form of the audio component.

21. The method of claim 17 wherein storing the preprocessed form comprises storing the preprocessed form at an address determined according to an address of the unit of content.

22. The method of claim 17 comprising providing the preprocessed form in conjunction with the unit of content.

23. The method of claim 22 comprising providing an integrated transport stream comprising the preprocessed form and the unit of content.

24. The method of claim 1 further comprising:

providing access to the multimedia content, including accepting a query for content, and determining units of content matching the query according to the meta data for the units.

25. The method of claim 24 wherein determining units matching the query includes searching the audio components according to query terms not represented in the meta data for the units.

26. The method of claim 25 further comprising augmenting the meta data for units according to the searching of the query terms.

27. The method of claim 25 wherein the query is accepted from a user computer, and searching the audio components includes distributing preprocessed forms of the audio components to the user computer, and receiving results of searching for the query terms from the user computers.

28. A computer-implemented method for searching a multimedia source, comprising:

receiving a query;

detecting a presence of the query in a meta data associated with the multimedia source, the metadata including a respective description of each of one or more segments of the multimedia source; and

based on the detection result, locating a segment of the multimedia content that corresponds to the query.

29. The computer-implemented method of claim 28, wherein the metadata includes a product of processing audio of the multimedia source.

30. The computer-implemented method of claim 29, wherein detecting the presence of the query includes applying a phonetically based word spotting procedure to locate the query in the metadata.

31. The computer-implemented method of claim 28, wherein the query includes a text query.

32. The computer-implemented method of claim 31, wherein the detecting the presence of the query in the metadata includes:

processing the text query to identify whether one or more components of the text query is represented in the metadata associated with the multimedia source.

33. The computer-implemented method of claim 28, wherein the query includes an audio query.

34. The computer-implemented method of claim 33, further comprising:

translating the audio query into a text query; and

processing the text query to identify whether one or more components of the text query is represented in the metadata associated with the multimedia source.

35. The computer-implemented method of claim 28, wherein the multimedia source is represented in a file format that enables the metadata to be embedded within the multimedia source.

36. The computer-implemented method of claim 28, wherein the respective description of each segment of the multimedia source includes a time location of the segment, or a link from a portion of the metadata that contains the respective description to the corresponding segment of the multimedia source.

37. A computer-implemented method for tagging a multimedia source, comprising:

receiving a query from a first user;

detecting a presence of the query in a text source associated with the multimedia source;

based on a result of the detection, locating a segment of the multimedia source that corresponds to the query; and

creating a tag associating the query to a time location of the segment of the multimedia source, the tag being accessible to a second and subsequent user.

38. The computer-implemented method of claim 37, further comprising:

distributing the tag to the second user through a centralized server.

39. The computer-implemented method of claim 37, further comprising:

distributing the tag to the second user in a peer-to-peer fashion.

40. The computer-implemented method of claim 37, further comprising:

generating an index of the multimedia source; and

updating the index based on the newly-created tag.

41. The computer-implemented method of claim 37, further comprising:

storing a representation of the tag within the multimedia source.

42. A computer-implemented method for validating a user-supplied annotation for a multimedia source, comprising:

identifying components of the user-supplied annotation;

accessing a text source associated with the multimedia source;

detecting whether the text source contains a representation of one or more of the identified components of the user-supplied annotation; and

based on a result of the detection, creating a tag associating the user-supplied annotation to the multimedia source.

43. The computer-implemented method of claim 42, wherein the multimedia source includes a plurality of segments that have been pre-processed to align with corresponding portions of the text source.

44. The computer-implemented method of claim 43, wherein the tag is configured to link the user-supplied annotation to a segment of the multimedia source that is aligned to the portion of the text source that contains the representation of the one or more of the indentified components of the user-supplied annotation.

45. The computer-implemented method of claim 42, wherein the text source includes at least one of the following:

a text news article that is related to the multimedia source, a web page that includes or links to the multimedia source, producer notes that are used to prepare the multimedia source, a transcript aligned to the multimedia source, optical character recognition (OCR) of text in the multimedia source, meta tags, keywords, an abstract or a summary provided by an author or editor of the multimedia source, and closed captioning of the multimedia source in the form of a TV broadcast.

46. A computer-implemented method for presenting advertisements to users of a multimedia source, comprising:

identifying one or more search terms associated with the multimedia source;

accessing a mapping that associates a plurality of search terms with a plurality of advertisements,

based on the identified search terms and the mapping, selecting an advertisement to be presented to users of the multimedia source.

47. The computer-implemented method of claim 46, wherein the search term includes at least one of a key word or a phrase present in a metadata of the multimedia source, and a key word or a phrase present in an audio portion of the multimedia source.

48. The computer-implemented method of claim 46, wherein the selection of the advertisement is conditioned on a score computed using the identified search terms.

49. The computer-implemented method of claim 46, wherein the search term includes a topic term.

50. The computer-implemented method of claim 49, wherein the mapping includes a predefined topic tree being aligned to the plurality of advertisements, the topic tree having a plurality of nodes each representing a classification of topic terms.

51. The computer-implemented method of claim 46, further comprising:

synchronizing a presentation of the selected advertisement with a presentation of the multimedia source.