Method and system for indexing and searching timed media information based upon relevance intervals

• US 8,117,206 B2
• Filed: 01/07/2009
• Issued: 02/14/2012
• Est. Priority Date: 07/06/2000
• Status: Expired due to Term

• Current Assignee: Comcast Cable Communications Management LLC (Comcast Corporation)
• Original Assignee: Streamray Incorporated (FriendFinder Networks Inc.)
• Inventors: Sibley, Tim V., Morton, Michael Scott, Unger, Noam Carl, Rubinoff, Robert, Davis, Anthony Ruiz, Aveni-Deforge, Kyle
• Primary Examiner(s): LEWIS, CHERYL RENEA
• Application Number:
  US12/349,934
• Publication Number:
  US 20090125534A1
• Time in Patent Office: 1,133 Days
• Field of Search: 707/805, 707/746
• US Class Current: 707/746
• CPC Class Codes: G06F 16/245 Query processing
• CPC Class Codes: G06F 16/31 Indexing; Data structures therefor; Storage structures
• CPC Class Codes: G06F 16/3344 using natural language analysis
• CPC Class Codes: G06F 16/41 Indexing; Data structures therefor; Storage structures
• CPC Class Codes: G06F 16/435 Filtering based on additional data, e.g. user or group profiles
• CPC Class Codes: G06F 16/447 Temporal browsing, e.g. timeline
• CPC Class Codes: G06F 16/7844 using original textual content or text extracted from visual content or transcript of audio data
• CPC Class Codes: G06F 40/205 Parsing
• CPC Class Codes: G06F 40/253 Grammatical analysis; Style critique
• CPC Class Codes: G06F 40/263 Language identification
• CPC Class Codes: Y10S 707/913 Multimedia
• CPC Class Codes: Y10S 707/99931 Database or file accessing
• CPC Class Codes: Y10S 707/99935 Query augmenting and refining, e.g. inexact access

Abstract

A method and system for indexing, searching, and retrieving information from timed media files based upon relevance intervals. The method and system for indexing, searching, and retrieving this information is based upon relevance intervals so that a portion of a timed media file is returned, which is selected specifically to be relevant to the given information representations, thereby eliminating the need for a manual determination of the relevance and avoiding missing relevant portions. The timed media includes streaming audio, streaming video, timed HTML, animations such as vector-based graphics, slide shows, other timed media, and combinations thereof.

Claims

1. A computer-implemented method, comprising:

determining by a computer, for each of a plurality of information representations associated with a content of a timed media file, a segment of the timed media file based on the information representation, including determining, for each of the segments, a start time of the segment and an end time of the segment both depending upon the information representation for the segment; and

storing, for each of the plurality of information representations, data that associates the information representation with the determined segment.

2. The computer-implemented method of claim 1, wherein at least two of the determined segments overlap with each other.

3. The computer-implemented method of claim 1, further comprising performing speech recognition of the content of the timed media file, and wherein determining comprises, for each of the information representations, the segment based on a result of the speech recognition.

4. The computer-implemented method of claim 1, further comprising performing optical character recognition of the content of the timed media file, and wherein determining comprises, for each of the information representations, the segment based on a result of the optical character recognition.

5. The computer-implemented method of claim 1, wherein determining comprises, for a first one of the segments:

determining a plurality of occurrences of the associated information representation in the timed media file; and

determining the first one of the segments such that the plurality of occurrences are within the start time and the end time of the first one of the segments.

6. The computer-implemented method of claim 1, wherein determining comprises, for a first one of the segments:

determining a plurality of occurrences of the associated information representation in the timed media file;

determining a first subset and a different second subset of the plurality of occurrences; and

determining the first one of the segments such that the first subset of the plurality of occurrences, and not the second subset of the plurality of occurrences, are within the start time and the end time of the first one of the segments.

7. The computer-implemented method of claim 6, wherein determining the first and second subsets comprises determining the first and second subsets based on a temporal distribution of the occurrences.

8. The computer-implemented method of claim 1, wherein determining comprises determining each of the segments to have at least a minimum length.

9. The computer-implemented method of claim 1, wherein storing comprises storing the data in a non-transitory computer-readable medium.

10. A computer-implemented method, comprising:

determining by a computer, for each of a plurality of information representations associated with a content of a timed media file, a segment of the timed media file based on the information representation;

storing, for each of the plurality of information representations, data that associates the information representation with the determined segment;

determining, for each of the information representations, a magnitude of relevance of the segment to the information representation; and

storing data representing each of the magnitudes of relevance.

11. A non-transitory computer-readable medium storing computer-executable instructions for performing a method, the method comprising determining, for each of a plurality of information representations associated with a timed media file, a segment of the timed media file based on the information representation, including determining, for each of the segments, a start time of the segment and an end time of the segment both depending upon the information representation for the segment.

12. The non-transitory computer-readable medium of claim 11, wherein at least two of the determined segments overlap with each other.

13. The non-transitory computer-readable medium of claim 11, wherein the method further comprises performing optical character recognition of the content of the timed media file, and wherein determining comprises, for each of the information representations, the segment based on a result of the optical character recognition.

14. The non-transitory computer-readable medium of claim 11, wherein the method further comprises determining, for each of the information representations, a magnitude of relevance of the segment to the information representation.

15. The non-transitory computer-readable medium of claim 11, wherein determining comprises, for a first one of the segments:

determining a plurality of occurrences of the associated information representation in the timed media file; and

determining the first one of the segments such that the plurality of occurrences are within the start time and the end time of the first one of the segments.

16. The non-transitory computer-readable medium of claim 11, wherein determining comprises, for a first one of the segments:

determining a plurality of occurrences of the associated information representation in the timed media file;

determining a first subset and a different second subset of the plurality of occurrences; and

determining the first one of the segments such that the first subset of the plurality of occurrences, and not the second subset of the plurality of occurrences, are within the start time and the end time of the first one of the segments.

17. The non-transitory computer-readable medium of claim 16, wherein determining the first and second subsets comprises determining the first and second subsets based on a temporal distribution of the occurrences.

18. The non-transitory computer-readable medium of claim 11, wherein determining comprises determining each of the segments to have at least a minimum length.