IDENTIFICATION OF SEGMENTS WITHIN AUDIO, VIDEO, AND MULTIMEDIA ITEMS

• US 20100158470A1
• Filed: 12/24/2008
• Published: 06/24/2010
• Est. Priority Date: 12/24/2008
• Status: Active Grant

• Current Assignee: TiVo Corporation (Adeia Inc.)
• Original Assignee: Comcast Interactive Media LLC (Comcast Corporation)
• Inventors: JOJIC, Oliver, HOUGHTON, David F., ZHOU, Hongzhong, Ambwani, Geetu, DAVIS, Anthony R., TZOUKERMANN, Evelyne, CHIPMAN, Leslie Eugene, SHEVADE, Bageshree, KRONROD, Vladimir, FARRELL, Ryan M.
• Granted Patent: US 9,442,933 B2
• Time in Patent Office: Days
• US Class Current: 386/241
• CPC Class Codes: G06F 16/48 Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually

Abstract

The invention pertains to methods, systems, and apparatus for identifying segments within a media item, the media segment including at least one of audio content and video content, comprising segmenting the media item into a plurality of segments as a function of subject matter, storing data identifying each segment and its subject matter, and organizing each segment within an ontology based on its subject matter.

Claims

1. A method of identifying sub-segments within a media item, the media segment comprising at least one of audio content and video content, the method comprising the steps of:

segmenting a media item into a plurality of segments as a function of subject matter;

classifying each segment within an ontology based on its subject matter; and

storing data identifying each segment and its subject matter.

2. The method of claim 1 wherein the segmenting step comprises:

determining subject matter information of a plurality of portions of the media item; and

segmenting the media item into a plurality of segments, each segment having a cohesive subject matter.

3. The method of claim 1 wherein the subject matter of each media item relates to a particular knowledge domain and wherein the segmenting step comprises determining subject matter with respect to at least one category of information, wherein the category is selected as a function of a knowledge domain of each media item.

4. The method of claim 3 wherein the at least one category and corresponding subject matter comprises at least one attribute/value pair.

5. The method of claim 1 wherein the step of classifying each segment comprises classifying each segment according to an ontology wherein the ontology has different portions specifically-adapted to different knowledge domains and each segment is classified within a portion of the ontology corresponding to a knowledge domain of the media item of which the segment forms a part.

6. The method of claim 3 wherein the step of classifying each segment comprises classifying each segment according to an ontology wherein the ontology has different portions specifically adapted to different knowledge domains and each segment is classified within a portion of the ontology corresponding to a knowledge domain of the media item of which each segment forms a part.

7. The method of claim 1 wherein the segmenting step comprises selecting a specific process for segmenting as a function of a knowledge domain of the media item.

8. The method of claim 3 wherein the segmenting step comprises selecting a specific process for determining subject matter information as a function of a knowledge domain of the media item.

9. The method of claim 6 wherein the segmenting step comprises selecting a specific process for determining subject matter information as a function of a knowledge domain of the media item.

10. The method of claim 7 further comprising the step of:

determining the knowledge domain of the media item.

11. The method of claim 1 further comprising the step of:

providing a search engine for searching for segments based on their subject matters, wherein the search engine utilizes the ontology to determine relevance of segments to a search query.

12. The method of claim 1 wherein the segmenting step comprises at least one of analyzing a closed captioning stream within the media items, analyzing audio within the media items for non-speech subject matter information, performing speech recognition on the audio within the media items, performing optical character recognition on the video of the media items, performing video analytics on video of the media items, and analyzing metadata within the media items for subject matter information.

13. The method of claim 12 wherein the segmenting step further comprises:

obtaining external subject matter information about the media items.

14. The method of claim 1 wherein the media items comprise multimedia assets and the segments are at the sub-asset level.

15. The method of claim 14 wherein the media items comprise television programs.

19. The computer program product of claim 15 wherein the subject matter of each media segment relates to a particular knowledge domain and wherein the computer executable instructions for segmenting comprises computer executable instructions for determining subject matter with respect to at least one category of information, wherein the category is selected as a function of a knowledge domain of each media item.

20. The computer program product of claim 19 wherein the at least one category and corresponding subject matter comprises at least one attribute/value pair.

21. The computer program product of claim 19 wherein the computer executable instructions for classifying each segment comprises computer executable instructions for classifying each segment according to an ontology wherein the ontology has different portions specifically adapted to different knowledge domains and each segment is classified to a portion of the ontology corresponding to a knowledge domain of the media item of which each segment forms a part.

22. The computer program product of claim 21 wherein the computer executable instructions for segmenting comprises computer executable instructions for selecting a specific process for segmenting as a function of a knowledge domain of the media item.

16. A computer program product for identifying segments within a media item having a cohesive subject matter, the media item comprising at least one of audio content and video content, the content relating to a particular knowledge domain comprising:

(1) computer executable instructions for segmenting the media segment into a plurality of segments, as a function of subject matter;

(2) computer executable instructions for storing data identifying each segment and its subject matter; and

(3) computer executable instructions for classifying each segment within an ontology based on its subject matter.

17. A method of claim 16 wherein the computer executable instruction for segmenting comprise:

(1) computer executable instructions for determining the subject matter information of a plurality of portions of the media item; and

(2) computer executable instructions for segmenting the media segment into a plurality of segments, each segment having a cohesive subject matter.

18. The computer program product of claim 16 wherein the computer executable instructions for segmenting comprises computer executable instructions for at least one of analyzing a closed captioning stream within the media items, analyzing audio within the media segments for non-speech subject matter information, performing speech recognition on the audio within the media segments, performing optical character recognition on the video of the media segments, performing video analytics on video of the media segments, and analyzing metadata within the media segments for subject matter information.

23. A data structure storing data defining an ontology, the ontology comprising different portions specifically adapted to different knowledge domains and each segment is classified within a portion of the ontology corresponding to a knowledge domain of the media item of which the segment forms a part.

24. A data structure storing data defining a plurality of segments of a plurality of media assets and including for each segment data defining subject matter information of the segment.

25. A data structure of claim 24 wherein the subject matter information for each segment is stored as a set of attribute/value pairs.

26. A data structure of claim 25 wherein each media asset relates to a particular knowledge domain and wherein the set of attribute/value pairs for each segment differs as a function of the knowledge domain of the media item of which it is a part.

27. A data structure of claim 24 wherein the subject matter information includes classification information for each sub-segment within an ontology having different portions specifically adapted to different knowledge domains and each segment is classified within a portion of the ontology corresponding to a knowledge domain of the media item of which the segment forms a part.

28. A data structure of claim 24 wherein the media items include multimedia assets and the segments are at the sub-asset level.

29. A data structure of claim 28 wherein the media items include television programs.