User-defined indexing of multimedia content

• US 7,984,089 B2
• Filed: 02/13/2004
• Issued: 07/19/2011
• Est. Priority Date: 02/13/2004
• Status: Expired due to Fees

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Bradstreet, John, McCullough, Luke Wabaunsee, Gates, Matthijs A., Evans, Glenn F., Weiss, Rebecca C.
• Primary Examiner(s): Lu; Kuen S
• Assistant Examiner(s): Liu; Hexing
• Application Number:
  US10/778,769
• Publication Number:
  US 20050193005A1
• Time in Patent Office: 2,713 Days
• Field of Search: 707/830, 707/793, 707/796
• US Class Current: 707/830
• CPC Class Codes: G06F 16/41 Indexing; Data structures therefor; Storage structures
• CPC Class Codes: G06F 16/435 Filtering based on additional data, e.g. user or group profiles
• CPC Class Codes: G06F 16/489 using time information

Abstract

Methods, systems, and computer program products for arbitrarily indexing multimedia content. In accordance with example embodiments, a multimedia content stream that comprises one or more samples is received. At least one of the samples is passed to one or more multimedia filters for analysis based on criteria that define an index type for the multimedia filter. Once it is determined that the sample meets the criteria for an arbitrary filter and index type, an index entry that identifies both the sample and the index type of the arbitrary filter is created. The index entry is adapted in accordance with a common, extensible indexing scheme to coexist with other index entries of other or the same index type for the sample. The created index entry is stored for subsequent access.

Claims

1. For a system configured to process multimedia content, a computer program product comprising one or more computer readable storage media carrying computer executable instructions that implement a method of interfacing with a plurality of code samples for interacting with an arbitrary index to the multimedia content such that the arbitrary index is capable of coexisting with one or more other arbitrary indexes, the method comprising acts of:

defining at least four separate programming interfaces configured to communicate with each other, the at least three separate interfaces including a get-index programming interface, a get-index-from-position programming interface, a get-index-position programming interface, and a get-entry-from-time programming interface, wherein each such interface is defined such that;

the get-index programming interface retrieves one of a plurality of indexes for the multimedia content, the plurality of indexes including an existing index corresponding to the multimedia content and at least one other index, the get-index programming interface comprising at least three different parameters, including;

(i) a parameter to identify the one index to retrieve, (ii) a parameter to identify the retrieved one index, and (iii) a return value to indicate whether or not a particular call to the get-index programming interface succeeds;

the get-index-from-position programming interface retrieves one of the plurality of indexes corresponding to the multimedia content based on a position of the retrieved index within the plurality of available indexes, the get-index-from-position programming interface comprising at least three different parameters, including;

(i) a parameter to identify the position of the one index to retrieve, (ii) a parameter to identify the retrieved one index, and (iii) a return value to indicate whether or not a particular call to the get-index-from-position programming interface succeeds;

the get-index-position programming interface retrieves a position of one of the plurality of indexes within the plurality of indexes for the multimedia content, the get-index-position programming interface comprising at least three different parameters, including;

(i) a parameter to identify the one index to retrieve, (ii) the position of the one index within the plurality of indexes, and (iii) a return value to indicate whether or not a particular call to the get-index-position programming interface succeeds; and

the get-entry-from-time programming interface retrieves an index entry based on a time to search for the index entry within the multimedia content, wherein the get-entry-from-time programming interface includes an optional parameter to identify a search direction relative to the time to search, wherein the optional parameter is configured to specify any of whether the index entry should appear before, at, after, and as close as possible to the time to search;

combining the existing index with at least one of the plurality of other indexes into a combined index, the combined index comprising a plurality of indexes which are applied to the multimedia content in parallel, in series or in a combination of parallel and series, such that one index can adjust another index, the multimedia content being accessible in a user-customized manner using the combined index, wherein each of the plurality of indexes is configured to adjust the existing index such that inputs directed to the existing index are modified in real time to provide the multimedia content at a position determined to be most preferable to the user;

receiving user input indicating that at least a portion of the multimedia content is to be accessed using the combined index including at least a portion of the existing index and at least a portion of one or more of the plurality of indexes;

based on the received user input, determining a first multimedia content position using the existing index;

adjusting the first multimedia content position with a second multimedia content position determined by at least one of the plurality of indexes to be preferable to the first multimedia content position;

accessing the multimedia content at the second position using the combined index including at least the portion of the existing index and at least one of the get-index programming interface, the get-index-from-position programming interface, and the get-index-position programming interfaces; and

displaying the indexed multimedia content on a display, wherein the multimedia content is navigable by a user using the plurality of indexes.

2. The computer program product as recited in claim 1, wherein the method further comprises an act of:

defining a get-count programming interface that retrieves a count of the number of indexes available for the multimedia content, the get-count programming interface comprising (i) a parameter to identify the count of the number of indexes available for the multimedia content, and (ii) a return value to indicate whether or not a particular call to the get-count programming interface succeeds.

3. The computer program product as recited in claim 1, wherein the method further comprises an act of:

defining a get-entry-from-position programming interface that retrieves an index entry based on a position of the index entry within an index, the get-entry-from-position programming interface comprises;

(i) a parameter to identify the position of the index entry within the index, (ii) a parameter to identify a time that corresponds to the index entry, (iii) a parameter for data corresponding to the index entry, (iv) a parameter that specifies on input a data size available for data corresponding to the index entry, and on output specifies an actual data size for data corresponding to the index entry, and (v) a return value to indicate whether or not a particular call to the get-entry-from-position programming interface succeeds.

4. The computer program product as recited in claim 1, wherein:

the get-entry-from-time programming interface comprises;

(i) a parameter to identify the time to search for the index entry within the multimedia content, (ii) a parameter to identify a position of the index entry within the index, (iii) a parameter for data corresponding to the index entry, (iv) a parameter that specifies on input a data size available for data corresponding to the index entry, and on output specifies an actual data size for data corresponding to the index entry, and (v) a return value to indicate whether or not a particular call to the get-entry-from-time programming interface succeeds.

5. The computer program product as recited in claim 1, wherein the method further comprises an act of:

defining a get-count programming interface that retrieves a count of index entries within an index, the get-count programming interface comprising (i) a parameter to identify the count of the index entries within the index, and (ii) a return value to indicate whether or not a particular call to the get-count programming interface that retrieves the count of index entries within an index succeeds.

6. The computer program product as recited in claim 1, wherein the method further comprises an act of:

defining a get-descriptor programming interface that retrieves a descriptor of an index, the get-descriptor programming interface comprising (i) a parameter to identify the descriptor of the index, (ii) a parameter to identify on input a data size available for the descriptor, and on output to identify an actual data size of the descriptor, and (iii) a return value to indicate whether or not a particular call to the get-descriptor programming interface succeeds.

7. The computer program product as recited in claim 1, further comprising an act of receiving user input defining a user-defined index which indicates one or more navigation points that are contrary to those indicated in the existing index.

8. The computer program product as recited in claim 7, wherein the user navigates to a first navigation point of the multimedia content using the existing index.

9. The computer program product as recited in claim 8, wherein the user navigates to a second navigation point of the multimedia content using at least one of the plurality of indexes.

12. The method of claim 1, wherein the existing, combined and plurality of other indexes are applied in real-time.

13. The method of claim 1, further comprising applying the combined index to at least two different data streams of the multimedia content.

14. The method of claim 1, further comprising applying both the existing index and at least one of the plurality of other indexes to each multimedia stream in the multimedia content.

15. The method of claim 13, wherein the at least two different streams comprise video and audio streams.

16. The method of claim 1, wherein the plurality of indexes is hierarchical, such that at least one of the plurality references at least one other previously generated index.

10. At an end-user computer system capable of processing multimedia content, a method for interfacing with a plurality of code samples for navigating a portion of multimedia content including an existing index and one or more user-defined indexes, such that the user-defined indexes are capable of coexisting with and are concurrently accessible with the existing index and the other user-defined indexes, the method comprising acts of:

receiving user input defining a plurality of different programming interfaces, including;

user input defining a get-index programming interface that retrieves one of a plurality of indexes for the multimedia content, the plurality of indexes including a existing index corresponding to the multimedia content and at least one user-defined index, the get-index programming interface comprising at least three different parameters, including;

(i) a parameter to identify the one index to retrieve, (ii) a parameter to identify the retrieved one index, and (iii) a return value to indicate whether or not a particular call to the get-index programming interface succeeds;

user input defining a get-index-from-position programming interface that retrieves one of the plurality of indexes corresponding to the multimedia content based on a position of the retrieved index within the plurality of available indexes, the get-index-from-position programming interface comprising at least three different parameters, including;

(i) a parameter to identify the position of the one index to retrieve, (ii) a parameter to identify the retrieved one index, and (iii) a return value to indicate whether or not a particular call to the get-index-from-position programming interface succeeds;

user input defining a get-index-position programming interface that retrieves a position of one of the plurality of indexes within the plurality of indexes for the multimedia content, the get-index-position programming interface comprising at least three different parameters, including;

(i) a parameter to identify the one index to retrieve, (ii) the position of the one index within the plurality of indexes, and (iii) a return value to indicate whether or not a particular call to the get-index-position programming interface succeeds; and

user input defining a get-entry-from-time programming interface that retrieves an index entry based on a time to search for the index entry within the multimedia content, wherein the get-entry-from-time programming interface includes an optional parameter to identify a search direction relative to the time to search, wherein the optional parameter is configured to specify any of whether the index entry should appear before, at, after, and as close as possible to the time to search;

combining the existing index with at least one of the plurality of other indexes into a combined index, the combined index comprising a plurality of indexes which are applied to the multimedia content in parallel, in series or in a combination of parallel and series, such that one index can adjust another index, the multimedia content being accessible in a user-customized manner using the combined index, wherein each of the plurality of indexes is configured to adjust the existing index such that inputs directed to the existing index are modified in real time to provide the multimedia content at a position determined to be most preferable to the user;

receiving user navigation input indicating that at least a portion of the multimedia content is to be accessed using the combined index including at least a portion of the existing index and at least a portion of the one or more of the plurality of indexes;

based on the received user input, determining a first multimedia content position using the existing index;

adjusting the first multimedia content position with a second multimedia content position determined by at least one of the plurality of indexes to be preferable to the first multimedia content position;

accessing the multimedia content at the second position using the combined index including at least the portion of the existing index and at least one of the get-index programming interface, the get-index-from-position programming interface, and the get-index-position programming interfaces to navigate to the portion indicated in the received user navigation input;

displaying the accessed multimedia content on a display according to the portion accessed as a result of the user input;

receiving user navigation input indicating that at least a portion of the multimedia content is to be accessed using one or more of the user-defined indexes;

accessing the multimedia content using at least one of the user-defined get-index programming interface, the user-defined get-index-from-position programming interface, and the user-defined get-index-position programming interfaces to navigate to the portion indicated in the received user navigation input;

displaying the accessed multimedia content on the display according to the portion accessed as a result of the user input.

11. At a computer system configured to process multimedia content, a method for interfacing with a plurality of code samples for interacting with an arbitrary index to the multimedia content such that the arbitrary index is capable of coexisting with one or more other arbitrary indexes, the method comprising acts of:

defining a get-index programming interface that retrieves one of a plurality of indexes for the multimedia content, the plurality of indexes including an existing index corresponding to the multimedia content and at least one other index, the get-index programming interface comprising (i) a parameter to identify the one index to retrieve, (ii) a parameter to identify the retrieved one index, and (iii) a return value to indicate whether or not a particular call to the get-index programming interface succeeds;

defining a get-index-from-position programming interface that retrieves one of the plurality of indexes corresponding to the multimedia content based on a position of the retrieved index within the plurality of available indexes, the get-index-from-position programming interface comprising (i) a parameter to identify the position of the one index to retrieve, (ii) a parameter to identify the retrieved one index, and (iii) a return value to indicate whether or not a particular call to the get-index-from-position programming interface succeeds;

defining a get-index-position programming interface that retrieves a position of one of the plurality of indexes within the plurality of indexes for the multimedia content, the get-index-position programming interface comprising (i) a parameter to identify the one index to retrieve, (ii) the position of the one index within the plurality of indexes, and (iii) a return value to indicate whether or not a particular call to the get-index-position programming interface succeeds;

defining a get-entry-from-position programming interface that retrieves an index entry based on a position of the index entry within an index;

defining a get-entry-from time programming interface that retrieves an index entry based on a time to search for the index entry within the multimedia content, wherein the get-entry-from-time programming interface includes a parameter to identify a search direction relative to the time to search, wherein the parameter is configured to specify any of whether the index entry should appear before, at, after, and as close as possible to the time to search;

combining the existing index with at least one of the plurality of other indexes into a combined index, the combined index comprising a plurality of indexes which are applied to the multimedia content in parallel, in series or in a combination of parallel and series, such that one index can adjust another index, the multimedia content being accessible in a user-customized manner using the combined index, wherein each of the plurality of indexes is configured to adjust the existing index such that inputs directed to the existing index are modified in real time to provide the multimedia content at a position determined to be most preferable to the user;

receiving user input indicating that at least a portion of the multimedia content is to be accessed using the combined index including at least a portion of the existing index and at least a portion of one or more of the plurality of indexes;

based on the received user input, determining a first multimedia content position using the existing index;

adjusting the first multimedia content position with a second multimedia content position determined by at least one of the plurality of indexes to be preferable to the first multimedia content position;

accessing the multimedia content at the second position using the combined index including at least the portion of the existing index and at least one of the get-index programming interface, the get-index-from-position programming interface, and the get-index-position programming interfaces; and

displaying the indexed multimedia content in two different content portion orderings on two different displays for the at least two different users, wherein the multimedia content is separately navigable by each user using the combined index.