Media processing methods, systems and application program interfaces

• US 7,900,140 B2
• Filed: 12/08/2003
• Issued: 03/01/2011
• Est. Priority Date: 12/08/2003
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Speed, Robin C. B., Weiss, Rebecca C., Nelson, Patrick N., Mohammed, Sohail Baig, Sherwani, Adil, Debique, Kirt A., Grigorovitch, Alexandre V., Barde, Sumedh N., Dunbar, Geoffrey T.
• Primary Examiner(s): Queler; Adam M
• Application Number:
  US10/730,735
• Publication Number:
  US 20050204289A1
• Time in Patent Office: 2,640 Days
• Field of Search: 715/201, 715/202, 715/203, 715/204, 715/249, 719/310, 709/228, 709/231
• US Class Current: 715/249
• CPC Class Codes: H04L 65/612 for unicast
• CPC Class Codes: H04L 65/765 intermediate
• CPC Class Codes: H04L 65/80 Responding to QoS

Abstract

Media processing methods, systems and application program interfaces (APIs) are described. In but one embodiment, a media engine component, also referred to as a media engine, provides a simple and unified way of rendering media from an origin to a destination of choice without requiring intimate knowledge about the underlying components, their connectivity and management. Clients of the media engine need not worry about how to render the particular media, but rather can simply focus on what media to render and where to render the media. In at least one embodiment, a media session is provided and is used by the media engine and provides a mechanism by which additional components are made transparent to the application and, in at least some embodiment, the media engine. In some embodiments, the media engine and media session provide a simple API for building, configuring, and manipulating a pipeline of components (e.g. media sources, transforms, and sinks) for media flow control between an origin and one or more destinations.

Claims

1. A system comprising:

first and second computing devices; and

a media engine distributed among and implemented in the first and second computing devices and configured to communicatively interact with an application of the second computing device to present a presentation on the second computing device, the first and second computing devices being remote from each other,

the media engine implemented in the first computing device being configured to;

use one or more media sources individual ones of which serving as a source of media content; and

first partially resolve a topology that is to be utilized to present the presentation, and then cause a full topology to be resolved and activated,

distribute the full topology to the second computing device, and

the media engine implemented in the second computing device being configured to use;

one or more transforms linked to the full topology, communicatively linked with one or more media sources, and configured to operate on data received from the one or more media sources; and

one or more media sinks configured to sink a media stream.

2. The system of claim 1, wherein the media engine exposes an application program interface that is used by the application to interact directly with the media engine, and indirectly with components used by the media engine.

3. The system of claim 1, wherein the media engine is configured to provide support for both linear and non-linear media sources.

4. The system of claim 1, wherein the media engine is configured to provide transport control for the media content.

5. The system of claim 1, wherein the media engine is configured to provide for asynchronous building and management of a media pipeline given a source of media content.

6. The system of claim 1, wherein the media engine is configured to provide source resolution for the media content.

7. The system of claim 1, wherein the media engine is configured to enable adjustment of a media processing pipeline configuration.

8. The system of claim 1, wherein the media engine is configured to support multiple different modes of stream selection.

9. The system of claim 8, wherein one mode comprises a mode in which the media engine selects which media streams are used.

10. The system of claim 8, wherein one mode comprises a mode in which the application selects which media streams are used.

11. The system of claim 8, wherein one mode comprises a mode in which the media engine selects which media streams are used, and another mode comprises a mode in which the application selects which media streams are used.

12. The system of claim 1, wherein the media engine is configured to set up a media session which uses said one or more media sources, said one or more transforms, and said one or more media sinks, said media session being configured to fully resolve a partial topology that has been resolved by said media engine.

13. The system of claim 12, wherein the media session is configured to fully resolve said partial topology by at least ascertaining transforms that are to be placed between the media sources and the media sinks.

14. The system of claim 12, wherein the media engine is configured to receive calls from the application and forward the calls to the media session, said calls comprising calls to start, stop and pause the presentation.

15. The system of claim 12, wherein the media session is configured to create a media processor that uses one or more media sources and one or more transforms.

16. The system of claim 12, wherein the media session is configured to create a media processor that uses one or more media sources and one or more transforms, wherein the media session is configured to set a topology on the media processor.

17. The system of claim 12, wherein the media session is configured to make determinations as to which time sources are to be used to drive the presentation.

18. The system of claim 12, wherein the media session is configured to prevent drift between a rate of media sources and a rate of a time source being used in live scenarios.

19. The system of claim 12, wherein the media session is configured to receive calls from the media processor to at least start, stop and pause the presentation.

20. The system of claim 12, wherein the media session is configured to receive calls from the media processor to at least start, stop and pause the presentation, wherein the media session is configured to send events to the media engine associated with calls that the media session receives from the media engine.

21. The system of claim 12, wherein the media session is configured to reduce glitches associated with a presentation by prerolling media data samples to one or more media sinks.

22. The system of claim 12, wherein the media session is configured to validate one or more component that handle data of the presentation.

23. The system of claim 1, wherein the media engine partially resolves said topology by at least determining one or more media sources and one or more media sinks for the presentation.

24. A system comprising:

first and second computing devices; and

a media engine distributed among and implemented in the first and second computing devices and configured to communicatively interact with an application of the second computing device to present a presentation on the second computing device, the first and second computing devices being remote from each other,

the media engine being configured to a provide plurality of open methods that can be called by the application to specify data sources in different manners,

the media engine implemented in the first computing device being configured to;

use one or more media sources individual ones of which serving as a source of media content; and

first partially resolve a topology that is to be utilized to present the presentation, and then cause a full topology to be resolved and activated,

distribute the full topology to the second computing device, and

the media engine implemented in the second computing device being configured to use;

one or more transforms linked to the full topology, communicatively linked with one or more media sources, and configured to operate on data received from the one or more media sources; and

one or more media sinks configured to sink a media stream.

25. The system of claim 24, wherein the media engine is configured to send events associated with a media presentation to the application.

26. The system of claim 24, wherein one of the open methods specifies a URL as a data source.

27. The system of claim 24, wherein one of the open methods specifies a media source created by the application.

28. The system of claim 24, wherein one of the open methods specifies an object that has an interface from which a media source object can be obtained.

29. The system of claim 24, wherein one of the open methods specifies an object from which a byte stream can be obtained.

30. The system of claim 24, wherein one of the open methods specifies a topology to be used.

31. The system of claim 24, wherein the open methods are selected from a group of open methods that:

specify a URL as a data source,

specify a media source created by the application,

specify an object that has an interface from which a media source object can be obtained,

specify an object from which a byte stream can be obtained, and

specify a topology to be used.

32. The system of claim 24, wherein the media engine is configured to provide methods to start a presentation, stop a presentation, and pause a presentation.

33. The system of claim 32, wherein the media engine is configured to generate and send an event to an application associated with each of said start, stop and pause methods.

34. The system of claim 24, wherein the media engine further comprises a plurality of information methods that can be used by the application to obtain information that pertains to the presentation.

35. The system of claim 34, wherein one of the information methods enables the application to be exposed to multiple capabilities of the media engine.

36. The system of claim 34, wherein one of the information methods enables the application to ascertain when the system'"'"'s capabilities change.

37. The system of claim 34, wherein one of the information methods enables the application to obtain metadata associated with the presentation.

38. The system of claim 34, wherein one of the information methods enables the application to obtain metadata associated with the presentation, the metadata being obtained in the form of a property store that can be queried for the metadata.

39. The system of claim 34, wherein one of the information methods enables the application to ascertain a current destination.

40. The system of claim 34, wherein one of the information methods enables the application to ascertain statistics associated with the media engine.

41. The system of claim 34, wherein one of the information methods enables the application to ascertain a current state of the media engine.

42. The system of claim 34, wherein one of the information methods enables the application to retrieve a clock according to which the media engine is presenting.

43. The system of claim 34, wherein the information methods are selected from a group of information methods comprising methods that enable the application to:

(1) be exposed to multiple capabilities of the media engine;

(2) obtain metadata associated with the presentation;

(3) ascertain a current destination;

(4) ascertain statistics associated with the media engine;

(5) ascertain a current state of the media engine; and

(6) retrieve a clock according to which the media engine is presenting.

44. The system of claim 24, wherein the media engine is configured to generate a plurality of events associated with the presentation, the media engine being configured to send the events to the application.

45. The system of claim 44, wherein one event is associated with a new presentation that is to be presented.

46. The system of claim 44, wherein one event is associated with a completion of an open method.

47. The system of claim 44, wherein one event is associated with completion of an operation begun by calling a start method on the media engine.

48. The system of claim 44, wherein one event is associated with completion of an operation begun by calling a stop method on the media engine.

49. The system of claim 44, wherein one event is associated with completion of an operation begun by calling a pause method on the media engine.

50. The system of claim 44, wherein one event is associated with rendering of a last data sample from an active media source.

51. The system of claim 44, wherein one event is associated with completion of an operation begun by calling a close method on the media engine.

52. The system of claim 44, wherein one event is associated with a switch between presentations.

53. The system of claim 44, wherein one event is associated with a presentation destination change.

54. The system of claim 44, wherein one event is associated with a state change on the media engine.

55. The system of claim 44, wherein one event is associated with a change in a set of allowed operations on the media engine.

56. The system of claim 44, wherein one event is associated with a media rate change.

57. A system comprising:

first and second computing devices; and

a media engine distributed among and implemented in the first and second computing devices and configured to communicatively interact with an application of the second computing device to present a presentation, the media engine being configured to use a media session, the media engine and the media session configured to present the presentation on the second computing device, the first and second computing devices being remote from each other, the media engine implemented in the first computing device being configured to;

use one or more media sources individual ones of which serving as a source of media content; and

first partially resolve a topology that is to be utilized to present the presentation, and then cause a full topology to be resolved and activated,

distribute the full topology to the second computing device, and

the media engine implemented in the second computing device being configured to use;

one or more transforms linked to the full topology, communicatively linked with one or more media sources, and configured to operate on data received from the one or more media sources; and

one or more media sinks configured to sink a media stream.

58. The system of claim 57, wherein the media engine exposes application program interfaces that are used by the application to interact directly with the media engine, and indirectly with components used by the media engine.

59. The system of claim 57 further comprising a destination associated with the media engine and configured to provide one or more media sinks.

60. The system of claim 57, wherein at least some components used by the media session are not visible to the application or media engine.

61. The system of claim 57, wherein the media session is configured to:

receive information from the media engine, said information being associated with (a) media content that is to be the subject of a presentation, and (b) a destination that is configured to provide one or more media sinks, and

cause the media content to be presented.

62. The system of claim 57, wherein the media session is configured to manage data flow from said one or more media sources to said one or more media sinks.

63. The system of claim 57, wherein the media session exposes one or more methods that enable the media engine to configure the media session for a presentation.

64. The system of claim 57, wherein the media session exposes one or more methods that enable the media engine to configure the media session for a presentation, wherein one method comprises a method through which a topology on the media session is initialized.

65. The system of claim 57, wherein the media session exposes one or more methods that enable the media engine to configure the media session for a presentation, wherein one method comprises a method through which one or more components can subscribe to receive notifications from a clock that is used to control the presentation.

66. The system of claim 57, wherein the media session provides methods for starting, stopping and pausing a presentation.

67. The system of claim 57, wherein the media session provides a preroll method that is used by the media engine to notify the media session to prepare for the start of a presentation.

68. The system of claim 57, wherein the media session further comprises a plurality of information methods that can be used by the media engine to obtain information from the media session.

69. The system of claim 57, wherein the media session further comprises a plurality of information methods that can be used by the media engine to obtain information from the media session, wherein one information method enables the media engine to ascertain a globally unique identifier that is associated with a particular implementation of a media session.

70. The system of claim 57, wherein the media session further comprises a plurality of information methods that can be used by the media engine to obtain information from the media session, wherein one information method enables the media engine to ascertain capabilities associated with the media session.

71. The system of claim 57, wherein the media session is further configured to generate a plurality of events associated with the presentation, the media session being configured to send the events to the media engine.

72. The system of claim 57, wherein the media session is further configured to generate a plurality of events associated with the presentation, the media session being configured to send the events to the media engine, wherein the media engine is configured to forward at least some of the events generated by the media session to the application.

73. The system of claim 57, wherein the media session is further configured to generate a plurality of events associated with the presentation, the media session being configured to send the events to the media engine, wherein one event comprises a session started event that is generated when a session is started.

74. The system of claim 57, wherein the media session is further configured to generate a plurality of events associated with the presentation, the media session being configured to send the events to the media engine, wherein one event comprises a session stopped event that is generated when a session is stopped.

75. The system of claim 57, wherein the media session is further configured to generate a plurality of events associated with the presentation, the media session being configured to send the events to the media engine, wherein one event comprises a session ended event that is generated with a session is ended.

76. The system of claim 57, wherein the media session is further configured to generate a plurality of events associated with the presentation, the media session being configured to send the events to the media engine, wherein one event comprises a session paused event that is generated when a session is paused.

77. The system of claim 57, wherein the media session is further configured to generate a plurality of events associated with the presentation, the media session being configured to send the events to the media engine, wherein one event comprises a rate change event that is generated when a media rate is changed.

78. The system of claim 57, wherein the media session is further configured to generate a plurality of events associated with the presentation, wherein the events are selected from a group of events comprising:

(1) a session started event that is generated when a session is started;

(2) a session stopped event that is generated when a session is stopped;

(3) a session ended event that is generated with a session is ended;

(4) a session paused event that is generated when a session is paused;

(5) a rate change event that is generated when a media rate is changed.

79. The system of claim 57 further comprising a media processor used by the media session and using at least one of said media sources and at least one transform.

80. A system comprising:

first and second computing devices; and

a media engine distributed among and implemented in the first and second computing devices and configured to communicatively interact with an application of the second computing device to present a presentation, the media engine being configured to use a media session, the media engine and the media session configured to present the presentation on the second computing device, the first and second computing devices being remote from each other,

the media session being configured to use at least one media processor, one or more bit pumps communicatively linked with the media processor, and one or more media sinks communicatively linked with respective bit pumps,

the media processor being configured to use one or more media sources and one or more transforms communicatively linked with one or more media sources and configured to operate on data received from the one or more media sources,

the media engine implemented in the first computing device being configured to;

use the one or more media sources, individual ones of which serving as a source of media content; and

first partially resolve a topology that is to be utilized to present the presentation, and then cause a full topology to be resolved and activated,

distribute the full topology to the second computing device, and

the media engine implemented in the second computing device being configured to use;

the one or more transforms, which are linked to the full topology, communicatively linked with one or more media sources, and configured to operate on data received from the one or more media sources; and

the one or more media sinks, which are configured to sink a media stream.

81. The system of claim 80, wherein the one or more bit pumps are configured to pull data from the media processor.

82. The system of claim 80, wherein the one or more bit pumps are configured to pull data from the media processor and to push pulled data to one or more media sinks.