Methods and systems for managing multiple inputs and methods and systems for processing media content

• US 20020097980A1
• Filed: 12/06/2000
• Published: 07/25/2002
• Est. Priority Date: 12/06/2000
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Rudolph, Eric H.
• Granted Patent: US 6,954,581 B2
• Time in Patent Office: Days
• US Class Current: 386/281
• CPC Class Codes: G11B 27/034 on discs G11B27/036, G11B27/038 take precedence
• CPC Class Codes: G11B 27/34 Indicating arrangements indicating means incorporated in magazine or cassette G11B23/046 and G11B23/0875; indicating measured values in general G01D

Abstract

Methods and systems for managing multiple inputs that are capable of competing or contending for a particular or primary output are described. In one embodiment, the multiple inputs are managed through the use of a software-implemented matrix switch object, and an associated data structure that is used to program the switch object and resolve contention issues between the inputs. The matrix switch object can process the multiple inputs to provide a primary output. One implementation of the switch object uses virtual input and output pins to receive and provide data streams. One specific embodiment is used in connection with multi-media editing software that enables users build or define their own editing multi-media editing projects that incorporate multiple different user-selected clips, in the form of digital data streams, into an integrated project. Each clip can be defined in terms of one or more digital data streams, e.g. video and audio streams. One implementation method defines a first data structure that represents an editing project and processes the data structure to provide a second data structure that contains data that can be used to program the matrix switch object so that multiple switch inputs are routed to multiple switch outputs and contentions between the inputs for a primary output are resolved.

Claims

1. A method of managing multiple inputs that can contend for a single output comprising:

providing a matrix switch object that can process multiple inputs and provide one or more outputs, one of the outputs comprising a primary output for which contention can take place between the multiple inputs,; and

programming the matrix switch object using a data structure, said programming resolving contention between the multiple inputs for the primary output.

2. The method of claim 1, wherein the matrix switch object is configured for use with inputs that comprise data streams.

3. The method of claim 1, wherein the matrix switch object is configured for use with inputs that comprise data streams, said programming enabling at least portions of at least some of the data streams to be combined.

4. The method of claim 1, wherein the matrix switch object is configured for use with a multi-media editing project in which the inputs comprise data streams at least portions of which are to be combined.

5. The method of claim 4, wherein said data streams comprise video data streams.

6. The method of claim 4, wherein said data streams comprise audio data streams.

7. The method of claim 4, wherein said data streams comprise both video and audio data streams.

8. The method of claim 1, wherein said providing comprises providing a software-implemented matrix switch object.

9. One or more computer-readable media having computer-readable instructions thereon which, when executed by a computer, implement the method of claim 1.

10. A method of managing multiple inputs that can compete for a single output comprising:

defining a software-implemented matrix switch having multiple input pins and multiple output pins, the input pins being individually associated with inputs that can compete, during a common time period, for a particular output pin that is associated with the matrix switch; and

programming the matrix switch using a data structure, said programming providing a routing scheme for routing input pins to output pins such that at any given time, only one input pin is routed to the particular output pin.

11. The method of claim 10, wherein the matrix switch is configured for use with inputs that comprise data streams.

12. The method of claim 10, wherein the matrix switch is configured for use with inputs that comprise data streams, said programming of the matrix switch providing for the combination of at least portions of at least some of the data streams.

13. The method of claim 10, wherein the matrix switch is configured for use with a multi-media editing project in which the inputs comprise data streams at least portions of which are to be combined.

14. The method of claim 13, wherein said data streams comprise video data streams.

15. The method of claim 13, wherein said data streams comprise audio data streams.

16. The method of claim 13, wherein said data streams comprise both video and audio data streams.

17. One or more computer-readable media having computer-readable instructions thereon which, when executed by a computer, implement the method of claim 10.

19. The multi-media editing method of claim 18, wherein said providing of the software-implemented matrix switch comprises providing a switch having virtual input pins and virtual output pins, the virtual input pins being configured to receive individual data streams, the virtual output pins being configured to provide individual data streams.

20. The multi-media editing method of claim 18, wherein said processing of the first data structure comprises configuring the matrix switch so that it receives at least one source stream at one of its inputs.

21. The multi-media editing method of claim 20, wherein said configuring comprises building a collection of associated objects operably associated with the matrix switch and that are configured to process digital data to provide a source stream at one or more of the inputs.

22. The multi-media editing method of claim 21, wherein at least some of the associated objects comprise filters.

23. The multi-media editing method of claim 18, wherein said processing of the first data structure comprises providing at least one object configured to receive at least one output stream from the matrix switch, process the output stream to provide an input stream, and provide the input stream to an input of the switch.

24. The multi-media editing method of claim 23, wherein said object comprises a transition object that is configured to transition between at least one source stream and at least one other source stream.

25. The multi-media editing method of claim 23, wherein said object comprises an effect object that is configured to operate on a source stream to produce a different source stream.

26. The multi-media editing method of claim 23, wherein said object comprises a mix object that is configured to mix multiple audio or video source streams.

27. The multi-media editing method of claim 18, wherein said defining of the first data structure comprises defining a hierarchical tree structure.

28. The multi-media editing method of claim 27, wherein the hierarchical tree structure comprises nodes that represent tracks, each track being associated with one or more data stream sources.

29. The multi-media editing method of claim 28, wherein the data stream sources with which one or more of the tracks are associated comprise video data stream sources.

30. The multi-media editing method of claim 28, wherein the data stream sources with which one or more of the tracks are associated comprise audio data stream sources.

31. The multi-media editing method of claim 28, wherein the data stream sources with which one or more of the tracks are associated comprise both video and audio data stream sources.

32. The multi-media editing method of claim 18, wherein said processing of the first data structure to provide the second data structure comprises processing the first data structure to provide a grid structure,

the grid structure containing multiple rows, individual rows representing inputs of the matrix switch and being associated with individual data stream sources,

each row containing at least one value that is associated with an output associated with said single output of the matrix switch.

33. The multi-media editing method of claim 32, wherein said operating on the second data structure comprises changing at least one value in at least one of the rows of the grid structure.

34. The multi-media editing method of claim 18, wherein said processing of the first data structure comprises configuring the matrix switch so that it receives multiple source streams at multiple respective inputs at multiple times.

35. The multi-media editing method of claim 34, wherein said configuring comprises building one or more filter graphs that are individually configured to process digital data to provide multiple respective source streams, and associating one or more filter graphs with individual inputs of the matrix switch.

36. The multi-media editing method of claim 35, further comprising building multiple sub-graphs inside one or more of the filter graphs.

37. One or more computer-readable media having computer-readable instructions thereon which, when executed by a computer, implement the method of claim 18.

39. The computer-readable media of claim 38, wherein the multi-media editing project operates on one or more multiple video data streams.

40. The computer-readable media of claim 39, wherein the multi-media editing project operates on the one or more multiple video data streams by combining two or more video streams.

41. The computer-readable media of claim 38, wherein the multi-media editing project operates on one or more audio data streams.

42. The computer-readable media of claim 41, wherein the multi-media editing project operates on multiple audio data streams by combining two or more audio data streams.

43. The computer-readable media of claim 38, wherein the multi-media editing project operates on one or more video and audio data streams.

44. The computer-readable media of claim 43, wherein the multi-media editing project operates on multiple video and audio data streams by combining two or more video data streams and combining two or more audio data streams.

45. The computer-readable media of claim 38, wherein the multi-media editing project comprises multiple tracks, each of which is associated with at least one data stream source.

46. The computer-readable media of claim 45, wherein the instructions cause the computer to use the tree structure to prioritize the tracks.

47. The computer-readable media of claim 46, wherein the instructions cause the computer to process the tree structure to provide the grid structure by traversing the tree structure in a predetermined manner.

48. The computer-readable media of claim 38, wherein the instructions cause the computer to change at least one data value within the grid structure to ensure that only one stream is routed to a primary input of the matrix switch at a time.

50. The computer-readable media of claim 49, wherein the matrix switch has a single output that provides a data stream that represents the editing project.

51. The computer-readable media of claim 49, wherein the matrix switch has at least one feedback loop coupled between an output and an input.

52. The computer-readable media of claim 51, wherein the feedback loop contains a transition element having two or more inputs and a single output, the transition element being configured to transition between multiple data streams.

53. The computer-readable media of claim 51, wherein the feedback loop contains an effect element having an input and an output, the effect element being configured to operate on a first data stream to provide a second data stream that is different from the first data stream.

54. The computer-readable media of claim 49, wherein the first data structure comprises a tree structure.

55. The computer-readable media of claim 49, wherein the second data structure comprises a grid structure.

56. The computer-readable media of claim 55, wherein the instructions cause the computer to change one or more data values within the grid structure to ensure that no two switch inputs are routed to a primary switch output at any one time.

57. The computer-readable media of claim 55, wherein the instructions that cause the computer to process the first data structure to provide the second data structure cause the computer to:

define a grid row for each of a number of data stream sources;

enter data values in each grid row, the data values being associated with a time period for which the data stream source desires to be routed to a primary output of the matrix switch; and

change at least one data value in at least a portion of a grid row if a determination is made that another grid row has an entry that indicates that its associated data stream source desires to be routed to the primary output of the matrix switch at the same time.

58. The computer-readable media of claim 57, wherein the instructions cause the computer to change said at least one data value responsive to a transition that is defined to occur between two data stream sources.

59. The computer-readable media of claim 57, wherein the instructions cause the computer to change said at least one data value responsive to an effect that is applied on a data stream source.

18. A multi-media editing method comprising:

defining a first data structure that represents a user-defined multi-media editing project;

providing a software-implemented matrix switch that is programmable to route multiple switch inputs to multiple switch outputs, at least two of the inputs being capable of competing for a single output during a common time period, the single output being configured to provide a data stream defined by the multi-media editing project;

processing the first data structure to provide a second data structure that contains data that can be used to program the matrix switch so that multiple switch inputs are routed to multiple switch outputs; and

operating on the second data structure so that no two inputs are routed to said single output during a common time period.

38. One or more computer-readable media having computer-readable instructions thereon which, when executed by a computer, cause the computer to:

define a multimedia editing project responsive to input received from a user;

represent the editing project as a hierarchical tree structure;

process the hierarchical tree structure to provide a grid structure containing data that defines an association between inputs, outputs and a time line defined by the editing project;

provide a matrix switch having multiple inputs and multiple outputs that correspond to the respective inputs and outputs of the grid structure; and

use the grid structure to program routing of the matrix switch'"'"'s inputs to the matrix switch'"'"'s outputs for the given time line.

49. One or more computer-readable media having computer-readable instructions thereon which, when executed by a computer, cause the computer to:

represent a multi-media editing project as a first data structure;

process the first data structure to provide a second data structure containing data that defines an association between inputs, outputs and a time line defined by the editing project;

provide a matrix switch having multiple inputs and multiple outputs that correspond to the respective inputs and outputs of the second data structure; and

use the second data structure to program routing of the matrix switch'"'"'s inputs to the matrix switch'"'"'s outputs for the given time line.

60. A data stream processing method comprising:

receiving at least one digital data stream using a software-implemented matrix switch having a primary output and at least one other output;

routing said at least one digital data stream to said at least one other output;

outputting said at least one digital data stream at said at least one other output;

processing said at least one digital data stream to provide a different digital data stream;

inputting the different digital data stream into the matrix switch; and

routing the input different digital data stream to the matrix switch'"'"'s primary output.

61. The data stream processing method of claim 60, wherein said receiving comprises receiving multiple streams and routing multiple streams to individual other switch outputs.

62. The data stream processing method of claim 61, wherein at least some of the multiple streams comprise video streams.

63. The data stream processing method of claim 61, wherein at least some of the multiple streams comprise audio streams.

64. The data stream processing method of claim 60, wherein said processing comprises transitioning from one stream to another stream.

65. The data stream processing method of claim 60, wherein said processing comprises using two video data streams to provide a video data stream containing portions of both video data streams.

66. The data stream processing method of claim 60, wherein said processing comprises applying an effect on said at least one digital data stream.

67. The data stream processing method of claim 60, wherein said processing comprises mixing multiple digital data streams.

68. The data stream processing method of claim 60, wherein said processing comprises mixing multiple audio or video digital data streams.

69. One or more computer-readable media having computer-readable instructions thereon which, when executed by a computer, implement the method of claim 60.

70. A data stream processing method comprising:

receiving at least one digital data stream using a software-implemented matrix switch object, the matrix switch object being configured to receive one or more digital data streams and output at least one digital data stream, individual digital data streams comprising portions of a user-defined multi-media editing project comprising one or more of video data streams and audio data streams;

outputting, from the matrix switch object, at least one digital data stream;

receiving, with another object, the output digital data stream;

processing the received digital data stream to provide a different digital data stream;

receiving the different digital data stream with the matrix switch object; and

outputting the received different digital data stream as a primary output stream that represents the user-defined multi-media editing project.

71. An input-managing system comprising:

a software-implemented matrix switch having multiple input pins and multiple output pins, the input pins being individually associated with inputs that can compete, during a common time period, for a particular output pin that is associated with the matrix switch; and

a data structure associated with the matrix switch and configured for use in programming the matrix switch to provide a routing scheme for routing input pins to output pins such that at any given time, only one input pin is routed to the particular output pin.

72. The input-managing system of claim 71, wherein the input pins are configured to receive digital data streams.

73. The input-managing system of claim 71, wherein the input pins are configured to receive digital video data streams.

74. The input-managing system of claim 71, wherein the input pins are configured to receive digital audio data streams.

75. The input-managing system of claim 71, wherein the input pins are configured to receive digital video and audio data streams.

76. The input-managing system of claim 71, wherein said data structure comprises a grid structure.

77. The input-managing system of claim 71 further comprising a second data structure associated with and different from the first-mentioned data structure, the second data structure being configured for deriving the first-mentioned data structure.

78. An input-managing system comprising:

a software-implemented matrix switch object configured to process multiple inputs and provide one or more outputs, one of the outputs comprising a primary output for which contention can take place between the multiple inputs; and

a data structure associated with the matrix switch object and configured for use in programming the matrix switch object to resolve contention issues between the multiple inputs.

79. A multimedia system comprising:

an application program configured to enable a user to define a multi-media project in which multiple digital source streams can be combined;

a software-implemented matrix switch having multiple input pins and multiple output pins, the input pins being individually associated with inputs that can compete, during a common time period, for a particular output pin that is associated with the matrix switch, the switch being configured to receive, at its input pins, digital source streams;

a first data structure associated with the matrix switch and configured for use in programming the matrix switch to provide a routing scheme for routing input pins to output pins such that at any given time, only one input pin is routed to the particular output pin; and

a second data structure associated with and different from the first data structure, the second data structure representing a user-defined multi-media project and being configured so that the first data structure can be derived therefrom.

80. The multi-media system of claim 80, wherein the first data structure comprises a grid structure.

81. The multi-media system of claim 80, wherein the second data structure comprises a tree structure.