System and method for processing multimedia data streams using filter graphs

• US 5,913,038 A
• Filed: 12/13/1996
• Issued: 06/15/1999
• Est. Priority Date: 12/13/1996
• Status: Expired due to Term

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Griffiths, Laurence Kelvin
• Primary Examiner(s): Rinehart, Mark H.
• Application Number:
  US08/766,795
• Time in Patent Office: 914 Days
• Field of Search: 395/200.61, 395/200.66, 395/200.49, 395/551, 395/650, 395/835
• US Class Current: 709/231
• CPC Class Codes: H04N 19/00 Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
• CPC Class Codes: H04N 21/4143 embedded in a Personal Computer [PC]

Abstract

Multimedia data processing accomplished by automated assembly of a filter graph comprising filters operative to conduct processing functions on data streams. The filter graph can be assembled by selecting appropriate filters that can handle the data processing requirements for the desired data stream(s). For example, a graph can be constructed by (1) selecting a set of filters, including an appropriate file reader compatible with the media type of the data stream(s), a demultiplexer for separating multiplexed data, a decoder for decoding encoded data, and a renderer to display or sound the data, and (2) combining these filters within the architecture of a filter graph to efficiently process the multimedia data.

Claims

1. A computer-implemented method for rendering a data stream of a source having a media type, comprising the steps of:

Reading a table of filter characteristics;

In response to reading the filter characteristics listed in the table, identifying a plurality of filters available for operation in a filter graph;

Creating an instance of a class of filters from the identified filters appropriate for rendering the data stream, each filter operative to conduct a processing operation and having at least one input pin, at least one output pin, or a combination of input and output pins; and

Connecting the pins of the filters to assemble the filter graph comprising connected filters, wherein the first filter in the filter graph accepts the data stream and the final filter in the filter graph renders the data stream.

7. The computer-implemented method of claim 1 further comprising the step of selecting the first filter, prior to adding the first filter to the filter graph, by searching the filters in the filter graph to locate one of the filters that is compatible with the media type for the data stream.

8. The computer-implemented method of claim 7 further comprising the step of selecting the first filter, prior to adding the first filter to the filter graph, by searching a table comprising characteristics of a plurality of available ones of the filters to locate one of the available filters that is (a) compatible with the media type for the data stream (b) operative to perform the processing operation needed for its position within the filter graph, and (c) is loaded for operation with a computer.

9. The computer-implemented method of claim 7 further comprising the step of selecting each intermediate filter and the final filter, prior to adding each intermediate and the final filter to the filter graph, by searching a table to locate a corresponding available filter that is (a) compatible with the media type for the data stream and (b) operative to perform the processing operation needed for its position within the filter graph.

10. The computer-implemented method of claim 7 further comprising the step of selecting each intermediate filter and the final filter, prior to adding each intermediate and the final filter to the filter graph, by searching a listing to locate a corresponding available filter that is:

(a) compatible with the media type for the data stream,

(b) operative to perform the processing operation needed for its position within the filter graph, and

(c) is loaded for operation with a computer.

11. The computer-implemented method of claim 10, wherein the listing contains filters which have been loaded for operation with the computer and have not yet been added to the filter graph.

12. The computer-implemented method of claim 7 further comprising the step of selecting each intermediate filter and the final filter, prior to adding each intermediate and the final filter to the filter graph, by:

searching a listing to locate a corresponding available filter that is

(a) compatible with the media type for the data stream,

(b) operative to perform the processing operation needed for its position within the filter graph, and

(c) is loaded for operation with a computer; and

in the event that the corresponding available filter is not identified in the listing, then searching a table comprising characteristics of available filters, including a merit figure assigned to each available filter for identifying a priority usage ranking, to locate a corresponding available filter that is

(d) compatible with the media type for the data stream,

(e) operative to perform the processing operation needed for its position within the filter graph, and

(f) has the highest merit figure.

13. The computer-implemented method of claim 1 further comprising the step of storing instructions for connecting the filters within the filter graph.

2. A method for automatically connecting filters to perform multimedia processing operations on a source file containing at least one data stream, comprising the steps of:

1) Selecting a source filter that can accept as an input a selected one of the data streams of the source file, and loading the source filter;

2) In the event that the source filter has an available output, selecting one of the outputs of the source filter;

3) For the selected output of the source filter, selecting and connecting another filter that can accept the selected output from the source filter as an input to the newly selected filter;

4) If the connection of the source filter to the selected filter is successful, then proceeding to step 5;

otherwise disconnecting the selected filter and repeating step 3 for another filter;

5) In the event that the selected filter has an output, then selecting the output of the selected filter;

6) For the selected output of the selected filter, selecting and connecting a new filter that can accept the selected output from the selected filter as an input to the new filter;

7) If the connection of the selected filter to the new filter is successful, then proceeding to step 8;

otherwise disconnecting the new filter and repeating step 6 for another filter;

8) Repeating steps 5-7 until the selected data stream is rendered by a chain formed by the connected filters; and

9) Repeating steps 2-8 for each remaining output of the source filter to render each remaining data stream of the source file.

14. The method of claim 2, wherein step 8 further comprises repeating steps 5-7 until the selected data stream is rendered by a leg of a chain of connected filters;

and

further comprising, after step 8, the steps of;

in the event that the selected filter has a second output, then selecting the second output of the selected filter;

repeating steps 6-7 until the selected data stream is rendered by a second leg of the chain including filters connected to the selected second output; and

determining one of the first leg and the second leg as a best leg based upon an efficiency of performing multimedia processing operations.

15. The method of claim 14 further comprising the step, prior to step 9, of storing instructions for building the best leg.

3. A system for automatically rendering each data stream maintained in a source file by constructing a graph of filters, comprising:

a filter graph manager for (1) determining the filters available that can accept a data type corresponding to each data stream and (2) controlling interconnection of selected ones of the available filters to assemble the filter graph, comprising interconnected filters, which accepts and renders each data stream; and

communication interfaces for supporting communication between the filter graph manager and a program module that can make direct calls to the filter graph manager to control a selected data stream or to retrieve filter-related events.

4. The system recited in claim 3, wherein the filter graph manager comprises a filter mapper for reading a table and, in response to filter characteristics listed in the table, determining the available filters.

16. The system of claim 4, wherein the filter mapper is further operative to read the filter graph to determine if one of the filters loaded and incorporated within the filter graph may be used as one of the available filters.

17. The system of claim 4, wherein the filter mapper is further operative to read a listing comprising filter identifiers to determine the available filters.

18. The system of claim 17, wherein the identifiers listed in the listing indicate the filters loaded but not incorporated within the graph of filters.

19. The system of claim 4, wherein the characteristics listed in the table include a merit figure associated with each the filters for defining an effectiveness for working with the data type.

20. The system of claim 19, wherein the filter graph manager is operative for determining the available filters according to a priority based upon the merit figure.

21. The system of claim 3, wherein the communication interfaces comprise a direct interface for supporting communication directly between the program module and the interconnected filters, which are created by the filter graph manager.

22. The system of claim 3 further comprising instructions stored in memory for assembling the interconnected filters of the filter graph.

5. A method for automatically connecting filters to perform processing operations on a source file containing at least one multimedia data stream characterized by a media type, comprising the steps of:

determining available filters that match a media type for a selected data stream of the source file, each available filter having a merit figure defining its effectiveness for working with the media type;

selecting available filters having a highest merit figure defining the suitability of the corresponding filter for a particular processing task; and

linking filters having the highest merit figure until the selected data stream is rendered by a filter graph constructed from the linked filters.

23. The method of claim 5, wherein the determining step further comprises searching a table comprising characteristics of the available filters to locate the available filters that match the media type for the selected data stream of the source file.

24. The method of claim 5 further comprising the step of storing instructions for constructing the filter graph from the linked filters.

25. The method of claim 5, wherein the linking step further comprises linking filters having the highest merit figure until the selected data stream is rendered by a first leg of filters of the filter graph, the first leg constructed from the linked filters wherein one of the linked filters has a first output linked within the first leg and a second output not linked within the first leg;

and

further comprising the step of repeating the determining, the selecting, and the linking steps until the selected data stream is rendered by a second leg of filters of the filter graph, the second output being linked within the second leg.

26. The method of claim 25 further comprising the steps of:

determining a best leg of the filter graph by comparing processing efficiencies of the first leg and the second leg; and

storing instructions for constructing the best leg of the filter graph.

6. A method for automatically connecting filters to perform filtering operations on a source file containing at least one data stream characterized by a media type, comprising the steps of:

1) Examining the source file to determine (1) a source of the source file and (2) the media type of a selected one of the media streams;

2) Searching a table containing a listing of characteristics of available ones of the filters, including a merit figure assigned to each filter to reflect a priority level for using the corresponding filter, and selecting a source filter based on the media type of the selected data stream and availability of the source file, wherein the source filter can accept as an input the selected data stream;

3) Loading the selected source filter;

4) Determining whether the selected source filter has an available output and, if so, selecting one of the available outputs of the source filter;

5) For the selected output of the source filter, examining the table and select, load, and connecting a new available filter that can accept the selected output from the source filter, wherein the selected new filter has the highest merit figure of the available filters compatible with the media type of the selected data stream and listed in the table;

6) If the connection of the source filter to the selected new filter is successful, then proceeding to step 7;

other wise disconnecting the selected new filter and returning to step 5 for another filter having the next highest merit figure and compatible with the media type of the selected data stream,

7) In the event that the selected new filter has an output, then selecting the output of the selected new filter;

otherwise, disconnecting the selected new filter and returning to step 5 for another filter having the next highest merit figure of the available filters compatible with the media type of the selected data stream and listed in the table unless the selected new filter renders the selected data stream, then proceeding to step 11;

8) For the selected output of the selected new filter, selecting, loading and connecting another available filter that can accept the selected output from the selected filter, wherein the present selected filter has the highest merit figure of the available filters compatible with the media type of the selected data stream and listed in the table;

9) If the connection of the prior selected new filter to the present selected filter is successful, then proceeding to step 10;

otherwise disconnecting the present selected filter and returning to step 8 for another available filter;

10) Repeating steps 7-9 until the selected data stream is rendered;

11) Repeating steps 2-10 for each remaining output of the source filter to render the remaining data streams of the source file.

27. The method of claim 6, wherein step 6 further comprises maintaining the disconnected new filter within a listing of loaded filters which are disconnected before returning to step 5;

and

wherein step 9 further comprises maintaining the disconnected selected filter within the listing of loaded filters which are disconnected before returning to step 8.

28. The method of claim 6, wherein step 10 renders the selected data stream with a first leg of connected filters;

and

further comprising the step, after step 10, of maintaining a score associated with the first leg of connected filters, the score based upon a processing efficiency of the selected data stream when processed by the first leg of connected filters.

29. The method of claim 28, wherein the score is based upon a fraction of the data streams in the source file successfully rendered.

30. The method of claim 28, wherein the score is based upon a number of the filters within the chain.

31. The method of claim 28 further comprising, after step 10, the steps of:

in the event that the selected new filter has a second output, then selecting a second output of the selected new filter;

repeating steps 8-9 until the selected data stream is rendered by a second leg of connected filters, the second leg connected to the selected second output;

determining a best one of the first leg and the second leg as a best leg based upon which of the first leg and the second leg has the higher score; and

storing instructions for building the best leg.