Method and apparatus for organizing segments of media assets and determining relevance of segments to a query

• US 8,713,016 B2
• Filed: 12/24/2008
• Issued: 04/29/2014
• Est. Priority Date: 12/24/2008
• Status: Active Grant

• Current Assignee: Comcast Interactive Media LLC (Comcast Corporation)
• Original Assignee: Comcast Interactive Media LLC (Comcast Corporation)
• Inventors: Chipman, Leslie Eugene, Tzoukermann, Evelyne, Davis, Anthony R., Houghton, David F., Farrell, Ryan M., Zhou, Hongzhong, Jojic, Oliver, Kronrod, Vladimir, Shevade, Bageshree, Ambwani, Geetu
• Primary Examiner(s): TRAN, ANHTAI V
• Application Number:
  US12/343,790
• Publication Number:
  US 20100161580A1
• Time in Patent Office: 1,952 Days
• Field of Search: None
• US Class Current: 707/737
• CPC Class Codes: G06F 16/24578 using ranking
• CPC Class Codes: G06F 16/438 Presentation of query results
• CPC Class Codes: G06F 16/44 Browsing; Visualisation therefor
• CPC Class Codes: G06F 16/48 Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually

Abstract

The invention pertains to methods, systems, and apparatus for identifying media items relevant to a selected subject matter, the method comprising determining the subject matter of a first media item, the first media item comprising at least one of audio content and video content, determining the classification within an ontology of the subject matter of the first media item, analyzing the ontology to identify other subject matter related to the subject matter of the first media item, and performing a search for other media items relevant to the subject matter of the first media item as a function of at least the other related subject matter according to the ontology.

Claims

1. A method comprising:

analyzing, by one or more computing devices, audio or video of an audiovisual program, resulting in subject matter data of the audiovisual program;

deriving one or more first words from the subject matter data;

locating the one or more first words in an ontology to identify a location in the ontology corresponding to the one or more first words;

determining one or more second words at least in part by analyzing one or more relationships defined at the location of the ontology, wherein the one or more relationships associate the location of the ontology to one or more locations of the ontology corresponding to the one or more second words;

formulating a search query that comprises at least one of the one or more first words and at least one of the one or more second words;

assigning weights to the search query, which comprises assigning a first of the weights to the at least one of the one or more first words and assigning a second of the weights to the at least one of the one or more second words, wherein the first of the weights is different from the second of the weights; and

performing a search for audiovisual program segments that uses the search query and is performed in accordance with the weights.

2. The method of claim 1 wherein performing the search comprises searching for one or more matching audiovisual program segments, and wherein each of the one or more matching audiovisual program segments comprises a segment of media at a sub-asset level, each of the one or more matching audiovisual program segments having a common subject matter.

3. The method of claim 1 wherein a first relationship of the one or more relationships is a parent-child relationship between a particular first word of the one or more first words and a particular second word of the one or more second words, wherein the difference between the first of the weights and the second of the weights is dependent on the parent-child relationship.

4. The method of claim 1 wherein a first relationship of the one or more relationships is a sibling relationship between a particular first word of the one or more first words and a particular second word of the one or more second words.

5. The method of claim 1 wherein the audiovisual program comprises multimedia assets.

6. The method of claim 1 wherein the audiovisual program comprises television programming.

7. The method of claim 1 wherein the analyzing of the audio or the video of the audiovisual program comprises at least one of the following:

analyzing audio within the audiovisual program for non-speech subject matter data, performing speech recognition on audio within the audiovisual program, performing optical character recognition on video of the audiovisual program, or performing video analytics on video of the audiovisual program.

8. The method of claim 1, wherein the subject matter data comprises a plurality of attribute-value pairs that describe content of the audiovisual program, and wherein deriving one or more first words from the subject matter data comprises identifying the one or more first words from the plurality of attribute-value pairs.

9. The method of claim 1, further comprising:

submitting the search query to a search engine; and

presenting results of the search to a user.

10. The method of claim 1, wherein the analyzing of the audio or the video of the audiovisual program is performed while a user is viewing the audiovisual program.

11. The method of claim 1, wherein the steps of deriving the one or more first words, locating the one or more first words in the ontology, determining the one or more second words, formulating the search query, assigning the weights to the search query, and performing the search are part of an automated process that is performed by the one or more computing devices.

12. The method of claim 1, further comprising:

determining, from the performing of the search for audiovisual program segments, a first audiovisual program segment corresponding to the subject matter data, wherein determining the first audiovisual program segment comprises determining a beginning and an end of the first audiovisual program segment; and

indexing the first audiovisual program segment to the location of the ontology.

13. The method of claim 1, wherein the first of the weights is greater than the second of the weights.

14. The method of claim 1, further comprising:

storing a plurality of attribute sets, each attribute set of the plurality of attribute sets comprising one or more attributes, and each attribute set of the plurality of attribute sets corresponding to one of a plurality of knowledge domains;

determining a particular knowledge domain of the plurality of knowledge domains for the audiovisual program;

selecting a specific process for determining the subject matter data as a function of the particular knowledge domain for the audiovisual program; and

selecting a set of attributes from the plurality of attribute sets as a function of the particular knowledge domain for the audiovisual program;

wherein the analyzing of the audio or video of the audiovisual program comprises performing the specific process for the determining of the subject matter data and determining values for the set of attributes based on the audio or video of the audiovisual program.

15. The method of claim 14 wherein determining the particular knowledge domain for the audiovisual program comprises determining, by the one or more computing devices, the particular knowledge domain for the audiovisual program based on a title of the audiovisual program.

16. A computer-readable memory storing executable instructions that, when executed, cause an apparatus to:

analyze audio or video of an audiovisual program, resulting in subject matter data of the audiovisual program;

derive one or more first words from the subject matter data;

locate the one or more first words in an ontology to identify a location in the ontology corresponding to the one or more first words;

determine one or more second words at least in part by analyzing one or more relationships defined at the location of the ontology, wherein the one or more relationships associate the location of the ontology to one or more locations of the ontology corresponding to the one or words;

formulate a search query that comprises at least one of the one or more first words and at least one of the one or more second words;

assign weights to the search query, which comprises assigning a first of the weights to the at least one of the one or more first words and assigning a second of the weights to the at least one of the one or more second words, wherein the first of the weights is different from the second of the weights; and

perform a search for audiovisual program segments that uses the search query and is performed in accordance with the weights.

17. The computer-readable memory of claim 16 wherein the ontology has different portions specifically adapted to different knowledge domains and the audiovisual program comprises an audiovisual program segment, and the audiovisual program segment is classified within a portion of the ontology corresponding to a knowledge domain of the audiovisual program.

18. The computer-readable memory of claim 16 further storing executable instructions that, when executed, cause the apparatus to:

index a plurality of audiovisual program segments within the ontology as a function of their subject matter; and

search for one or more matching audiovisual program segments indexed under the one or more second words within the ontology.

19. The computer-readable memory of claim 16 wherein the audiovisual program comprises television programming.

20. The computer-readable memory of claim 16 wherein a first relationship of the one or more relationships is a parent-child relationship or a sibling relationship, the first relationship being between a particular first word of the one or more first words and a particular second word of the one or more second words wherein the difference between the first of the weights and the second of the weights is dependent on the parent-child relationship.

21. The computer-readable memory of claim 16 further storing executable instructions that, when executed, cause the apparatus to:

store a plurality of attribute sets, each attribute set of the plurality of attribute sets comprising one or more attributes, and each attribute set of the plurality of attribute sets corresponding to one of a plurality of knowledge domains;

determine a particular knowledge domain of the plurality of knowledge domains for the audiovisual program;

select a specific process for determining the subject matter data as a function of the particular knowledge domain for the audiovisual program; and

select a set of attributes from the plurality of attribute sets as a function of the particular knowledge domain for the audiovisual program;

wherein the analyzing of the audio or video of the audiovisual program comprises performing the specific process for the determining of the subject matter data and determining values for the set of attributes based on the audio or video of the audiovisual program.

22. The computer-readable memory of claim 21 wherein determining the particular knowledge domain for the audiovisual program comprises determining the particular knowledge domain for the audiovisual program based on a title of the audiovisual program.