Media timeline

• US 7,934,159 B1
• Filed: 02/19/2004
• Issued: 04/26/2011
• Est. Priority Date: 02/19/2004
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Leroux, Stephen T., Novak, Michael J., Rahman, Shafiq Ur, Mohammed, Sohail Baig, Sherwani, Adil, Debique, Kirt A., Zhi, Xiqiang, Grigorovitch, Alexandre V., Barde, Sumedh N.
• Primary Examiner(s): Lo; Weilun
• Assistant Examiner(s): Iturralde; Enrique W
• Application Number:
  US10/782,734
• Time in Patent Office: 2,623 Days
• Field of Search: 715/716, 715/201, 715/202, 715/203, 715/704, 715/717, 715/719, 715/721, 715/722, 715/723, 715/727, 715/730, 715/731, 715/732, 715/853, 715/854, 715/855
• US Class Current: 715/716
• CPC Class Codes: H04N 21/26258 for generating a list of items to be played back in a given order, e.g. playlist, or scheduling item distribution according to such list retrieval of multimedia data based on playlists G06F16/40
• CPC Class Codes: H04N 21/4825 using a list of items to be played back in a given order, e.g. playlists

Abstract

A media timeline is described. In an implementation, a method includes receiving a request from an application at an application programming interface (API) to interact with a plurality of media. Based on the request, a media timeline is generated. The media timeline is for exposure via the API to the application and includes a plurality of nodes. The media timeline defines a presentation of a first media referenced by a first node with respect to a second media referenced by a second node.

Claims

1. A method comprising:

receiving a request, from an application at an application programming interface (API), to interact with a plurality of media comprising streaming media; and

generating an original media timeline based on the request, wherein the original media timeline;

is exposed to the application via the API;

includes a plurality of nodes;

defines a presentation, to be output via one or more computers, of a first media referenced by a first node with respect to a second media referenced by a second node, wherein;

the first and second nodes are configured as parallel nodes such that the media referenced by the first node that is a child of a parent node is rendered concurrently with the media referenced by the second node that is a child of the same parent node;

the original media timeline is configured for dynamic creation, automatically by the one or more computers without human user interaction, such that at least one node is dynamically created while at least one of the media referenced by the plurality of nodes of the original media timeline is being rendered; and

at least one node includes metadata, the metadata describing;

rendering of the at least one node; and

a collection of additional nodes to be dynamically modified when the original media timeline is rendered; and

specifies, automatically by the one or more computers without human user interaction, delayed creation of one or more of the plurality of nodes when the media timeline is rendered, wherein the delayed creation includes creating the one or more of the plurality of nodes when called by the application.

2. A method as described in claim 1, wherein one or more nodes are configured as a sequence node such that one node that is a child of the sequence node is rendered after another node that is also a child of the sequence node.

3. A method as described in claim 1, wherein one or more nodes is configured as a root node that specifies a starting point for rendering the original media timeline.

4. A method as described in claim 1, wherein the first and second said nodes reference the respective first and second media utilizing respective first and second pointers.

5. A method as described in claim 1, wherein the metadata is selected from a group of metadata, the group of metadata consisting of:

a uniform resource locator (URL) property for the media referenced by the at least one node;

a source object property that specifies a source object which can resolve to a media source that provides the media referenced by the at least one node;

a source object identifier (ID) property that specifies a unique identifier of the source object;

a media start property that specifies a time, during a duration of the media referenced by the at least one node, that rendering of the media is to be started;

a media stop property that specifies a time, during a duration of the media referenced by the at least one node, that rendering of the media is to be stopped;

a time format property that specifies a time format for at least one of the media start property, and the media stop property;

a stream selection property which specifies one of a plurality of streams for rendering of the media referenced by the at least one node;

a format based property that specifies a format for the media referenced by the at least one node;

a loop count property that specifies a number of times the at least one node is to be rendered;

a disabled property that specifies whether the at least one node is to be rendered when the original media timeline is rendered;

a generic property that serves as a repository of information related to the at least one node, wherein the generic property is configured for specification by at least one of the application and a timeline source for rendering the original media timeline;

a noskip property that specifies that the rendering of the at least one node is not to be skipped when the original media timeline is rendered; and

a noskip child property that specifies that the at least one node has another node, which is a child of the at least one node, which specifies that the rendering of the other node is not to be skipped when the original media timeline is rendered.

6. A method as described in claim 1, wherein at least one node is configured to reference an effect to be applied to an output of media referenced by the node.

7. A method as described in claim 1, wherein the original media timeline is configured for dynamic loading such that metadata included in at least one node specifies a collection of nodes to be loaded when the original media timeline is rendered.

8. A method as described in claim 1, wherein at least one node of the original media timeline is specified as read-only by creating a read-only wrapper of the original media timeline.

9. A method as described in claim 1, wherein at least one node is configured for communication of events to another node such that a change may be made to the original media timeline while the original media timeline is rendered.

10. A method as described in claim 1, wherein the first and second media have different formats.

11. One or more computer storage devices storing computer executable instructions that, when executed by a computer, direct the computer to perform the method of claim 1.

12. A method as recited in claim 8, wherein a node specified as read-only disables functionality comprising one or more of:

skipping and/or deleting advertisements;

modifying the media timeline by other components while allowing dynamic changes to the original media timeline;

or

adding new children to the original media timeline while allowing other components to set custom metadata on the original media timeline nodes.

13. A method as recited in claim 8, wherein the read-only wrapper contains cloned nodes that mirror a structure of the original media timeline.

14. A method as recited in claim 13, wherein the cloned nodes are configured to subscribe to events generated on the nodes of the original media timeline such that the structure of cloned nodes is kept updated as the original media timeline changes.

15. A method comprising:

generating a media timeline by an application, wherein the media timeline;

includes a plurality of nodes referencing media comprising streaming media;

defines a presentation of a first media referenced by a first node with respect to a second media referenced by a second node, the presentation being configured to be output by one or more computers;

specifies, automatically by the application executing on the one or more computers without human user interaction, that one or more of the plurality of nodes is created in a delayed manner in response to being called by one or more applications while the media timeline is being rendered; and

is configured, automatically by the application executing on the one or more computers without human user interaction, for dynamic creation such that at least a first node grouping is created while media referenced by a second node grouping in the media timeline is being rendered; and

passing the media timeline to a timeline source for rendering.

16. A method as described in claim 15, wherein the first and second media have different formats.

17. A method as described in claim 15, wherein at least one node is configured to reference an effect to be applied to an output of media referenced by the node.

18. A method as described in claim 15, wherein the media timeline is configured for dynamic loading such that metadata included in at least one node specifies a collection of nodes to be loaded when the media timeline is rendered.

19. One or more computer storage devices storing computer executable instructions that, when executed by a computer, direct the computer to perform the method of claim 15.

20. A method as described in claim 15, wherein the media timeline is configured for dynamic loading such that metadata included in at least one node specifies a collection of nodes to be loaded when the media timeline is rendered, the collection of nodes comprising a parallel node that includes metadata specifying a plurality of leaf nodes for simultaneous rendering.

21. A method for outputting a media presentation via one or more computers comprising:

specifying an effect to be applied to one or more of a plurality of media comprising streaming media when the media is rendered; and

generating a media timeline configured for exposure via an application programming interface (API), wherein;

the media timeline includes a plurality of nodes;

two or more of the plurality of nodes reference respective media;

one or more of the plurality of nodes that reference the one or more of the plurality of media include metadata that controls the effect to be applied to the one or more of the plurality of media when the media is rendered;

the media timeline specifies, automatically by the one or more computers without human user interaction, delaying creation of one or more of the plurality of nodes when the media timeline is rendered, wherein delaying the creation includes at least one of the one or more computers automatically creating the one or more of the plurality of nodes when called by one or more applications;

the media timeline being configured for dynamic creation, automatically by the one or more computers without human user interaction, such that at least one node of the plurality of nodes is created while the media timeline is rendered; and

at least one node of the plurality of nodes includes metadata, the metadata describing;

rendering of the at least one node of the plurality of nodes; and

a collection of additional nodes of the plurality of nodes to be dynamically modified when the at least one node of the plurality of nodes is rendered.

22. A method as described in claim 21, wherein the effect is a simple effect provided by a software component that is configured to:

receive a single stream of media;

apply the effect to the single stream; and

output the applied single stream.

23. A method as described in claim 21, wherein the effect is a composite effect provided by a software component that is configured to:

receive at least two streams of media;

apply the effect to the at least two streams; and

output a single stream of media composed of the applied at least two streams.

24. A method as described in claim 21, wherein the effect is a composite effect provided by a software component that is configured to analyze at least two streams of media or output at least two streams of media.

25. A method as described in claim 21, wherein the effect is a transition effect to be applied as a transition from a first media referenced by a first node to a second media referenced by a second node.

26. A method as described in claim 21, wherein the effect includes metadata that describes the effect that is selected from the group of metadata, the group of metadata comprising:

an effect object globally unique identifier (GUID) property that specifies a GUID to be used to create a transform object that is configured to provide the effect;

an effect object property that references an effect object that is configured to provide the effect;

a priority property that specifies an ordering of a plurality of effects, one to another;

a media start property that specifies a time, during a duration of the media referenced by the node, that rendering of the media is to be started;

a media stop property that specifies a time, during a duration of the media referenced by the node, that rendering of the media is to be stopped;

a time format property that specifies a time format for at least one of the media start property and the media stop property;

a number of inputs property that specifies a number of inputs to the effect;

a number of outputs property that specifies a number of outputs from the effect;

an output major type property that specifies a major type for media, to which, the effect is to be applied; and

an input connections property that specifies the one or more nodes that are to be processed by the effect.

27. A method as described in claim 21, wherein the media timeline is configured for dynamic loading such that metadata included in at least one node specifies a collection of nodes to be loaded when the media timeline is rendered.

28. A method as described in claim 21, wherein at least one node is specified as read-only.

29. A method as described in claim 21, wherein at least one node is configured for communication of events to another node such that a change may be made to the media timeline while the media timeline is rendered.

30. One or more computer storage devices storing computer executable instructions that, when executed by a computer, cause the computer to perform the method of claim 21.

31. In a media timeline exposed via an application programming interface and having a plurality of nodes, a method comprising:

rendering a first media item of a plurality of media items, at least one of the plurality of media items comprising a streaming media item, the first media item being referenced by a first node of a first node type of a plurality of node types, the plurality of node types comprising a sequence node type that includes metadata describing a rendering order of a plurality of leaf nodes to the sequence node;

receiving a call for a second node that references a second media item;

dynamically creating automatically, without user intervention, the second node of a second node type of the plurality of node types, while rendering the first media item;

wherein the media timeline is configured for automatic dynamic updating, automatically by the one or more computers without human user interaction, such that metadata included in at least one node specifies a collection of nodes to be modified when the at least one node is loaded; and

delaying, automatically by the one or more computers without human user interaction, creation of one or more of the collection of nodes when the media timeline is rendered, wherein the delayed creation includes a computer automatically creating the one or more of the collection of nodes when called by one or more applications.

32. A method as described in claim 31, further comprising rendering the second media item referenced by the second node when the rendering of the first media item is completed.

33. A method as described in claim 31, further comprising:

rendering the second media item referenced by the second node;

receiving a call for a third node that references a third media item; and

creating the third node.

34. A method as described in claim 31, wherein at least one node is configured to reference an effect to be applied to an output of media referenced by the node.

35. A method as described in claim 31, wherein at least one node is specified as read-only.

36. A method as described in claim 31, wherein at least one node is configured for communication of events to another node such that a change may be made to the media timeline while the media timeline is rendered.

37. One or more computer storage devices storing computer executable instructions that, when executed by a computer, direct the computer to perform the method of claim 31.

38. A method as described in claim 31, wherein the plurality of node types further comprises:

a root node that specifies a starting point for rendering the media timeline, the root node including metadata that describes how rendering is to be initiated;

a leaf node that directly maps to media to be rendered and output, the leaf node including metadata that describes how to retrieve the media; and

a parallel node type that includes metadata specifying a plurality of leaf nodes that are rendered simultaneously.

39. In a media timeline exposed via an application programming interface, the media timeline having a plurality of nodes, at least two of which reference respective media, one or more nodes each having metadata that references a node grouping, a method comprising:

utilizing a computer to load a first node for rendering, wherein the first node is selected from a plurality of node types, the plurality of node types comprising a parallel node type that includes metadata specifying a plurality of leaf nodes that are rendered simultaneously;

examining metadata associated with the first node to determine a first node grouping to be loaded in conjunction with the first node;

loading each node referenced by the first node grouping;

rendering the first node grouping;

examining at least one second node in the first node grouping to determine a second node grouping, wherein the examining the at least one second node in the first node grouping is performed during the rendering of the first node grouping;

loading each node referenced by the second node grouping; and

rendering the second node grouping when the rendering of the first node grouping is completed, wherein;

the media timeline is configured for dynamic creation, automatically by the one or more computers without human user interaction, where at least a third node is specified automatically by the computer without human user interaction for delayed creation while the media timeline is being rendered, the dynamic creation of the third node being performed by a node source that includes data that defines properties and interrelationships of the created third node with respect to one or more nodes in the first node grouping or one or more nodes in the second node grouping; and

at least a fourth node is configured for communication of an initiated event to another a fifth node which has subscribed to receive events initiated by the fourth node, such that a change may be is made to one or more nodes in the media timeline that are affected by the initiated event while the media timeline is being rendered, wherein the plurality of one or more nodes of the media timeline that are affected by the initiated event change are automatically dynamically updated.

40. A method as described in claim 39, wherein at least one node is configured to reference an effect to be applied to an output of media referenced by the node, wherein the media comprises streaming media.

41. A method as described in claim 39, wherein at least one node is specified as read-only.

42. A method as described in claim 39, wherein a first node references media having a plurality of different formats that are a different format than media referenced by one or more nodes of the second node grouping.

43. One or more computer storage devices storing computer executable instructions that, when executed by a computer, direct the computer to perform the method of claim 39.

44. A method as described in claim 39, wherein the plurality of node types further comprises:

a root node that specifies a starting point for rendering the media timeline, the root node including metadata that describes how rendering is to be initiated; and

a sequence node type that includes metadata that describes a rendering order of a plurality of leaf nodes to the sequence node.

45. A method as described in claim 39, wherein the media timeline is configured for dynamic creation that occurs without user interaction.

46. A method comprising:

exposing a media timeline via an application programming interface (API), the media timeline having a plurality of nodes, two or more nodes each referencing respective media at least one of which comprises streaming media, and wherein the media timeline is configured for dynamic creation, automatically by the one or more computers without human user interaction, such that metadata included in at least one node specifies a collection of nodes to be loaded when the media timeline is rendered;

rendering a first node to output a referenced first said media;

during the rendering of the first node, the API dynamically changing one or more properties of a second node;

initiating, by an event generator located on the second node, an event for communication to a parent node of the second node, wherein the event describes the changing; and

delaying, automatically by the one or more computers without human user interaction, creation of one or more of the plurality of nodes when the media timeline is exposed, wherein the delayed creation includes creating the one or more of the plurality of nodes when called by one or more applications.

47. A method as described in claim 46, wherein the event is communicated to at least one of an application over the API and a timeline source for rendering the media timeline.

48. A method as described in claim 46, wherein the one or more properties are selected from the group consisting of:

node added event;

node removed event;

node changing event;

remove children event;

node source added event;

node source removed event;

node sort event; and

node moved event.

49. A method as described in claim 46, wherein:

at least one node of the media timeline is configured as a root node; and

each event generated by one of the plurality of nodes that is a child of the root node is communicated to the root node.

50. A method as described in claim 46, wherein the media timeline is configured for dynamic creation such that at least one node is created while the media timeline is rendered.

51. A method as described in claim 46, wherein at least one node is specified as read-only.

52. A method as described in claim 46, wherein at least one node is configured to reference an effect to be applied to an output of media referenced by the node.

53. One or more computer storage devices storing computer executable instructions that, when executed by a computer, direct the computer to perform the method of claim 46.

54. A method as described in claim 46, wherein one or more nodes of the media timeline subscribes to events initiated by other nodes of the media timeline.

55. An application programming interface stored on a computer storage device, which when interfaced with a computer, exposes a media timeline to one or more independent applications, the application programming interface comprising:

the media timeline, automatically by the computer without human user interaction, delaying creation of one or more of a plurality of nodes when the media timeline is rendered, wherein the delayed creation includes the computer creating the one or more of the plurality of nodes when called by one or more applications;

the media timeline comprising the plurality of nodes callable by the one or more applications, wherein;

each node includes metadata that describes the node, the metadata comprising a source object property that specifies a source object which can resolve to a media source that provides the media referenced by the node;

the one or more of the plurality of nodes reference a corresponding media item comprising a streaming media item;

the plurality of nodes are arranged in a tree structure; and

the arrangement of the plurality of nodes, one to another, describes an order for rendering the plurality of nodes, wherein the media timeline is configured for dynamic creation, automatically by the computer without human user interaction, such that at least one node is created while the media timeline is rendered and at least one node is dynamically updated in response to the at least one node being created.

56. An application programming interface as described in claim 55, wherein the metadata for each node is selected from a group of metadata, the group of metadata comprising:

a uniform resource locator (URL) property for the media referenced by the node;

a source object identifier (ID) property that specifies a unique identifier of the source object;

a media start property that specifies a time, during a duration of the media referenced by the node, that rendering of the media is to be started;

a media stop property that specifies a time, during a duration of the media referenced by the node, that rendering of the media is to be stopped;

a time format property that specifies a time format for at least one of the media start property, and the media stop property;

a stream selection property which specifies one of a plurality of streams for rendering of the media referenced by the node;

a format based property that specifies a format for the media referenced by the node;

a loop count property that specifies a number of times the node is to be rendered;

a disabled property that specifies whether the node is to be rendered when the media timeline is rendered;

a noskip property that specifies that the rendering of the node is not to be skipped when the media timeline is rendered; and

a noskip child property that specifies that the node has another node, which is a child of the node, which specifies that the rendering of the other node is not to be skipped when the media timeline is rendered.

57. An application programming interface as described in claim 55, wherein at least one node is configured to reference an effect to be applied to an output of media referenced by the node.

58. An application programming interface as described in claim 55, wherein at least one node includes metadata that describes rendering of the at least one node.

59. An application programming interface as described in claim 55, wherein the media timeline is configured for dynamic loading such that metadata included in at least one node specifies a collection of nodes to be loaded when the media timeline is rendered.

60. An application programming interface as described in claim 55, wherein at least one node is specified as read-only.

61. An application programming interface as described in claim 55, wherein at least one node is configured for communication of events to another node such that a change may be made to the media timeline while the media timeline is rendered.

62. An application programming interface stored on a computer storage device, that when accessed by a computer facilitates acts comprising:

exposing, automatically by the computer without human user interaction, a media timeline to one or more independent applications, the media timeline comprising a plurality of nodes callable by one application, wherein;

two or more of the nodes reference respective media, one of which comprises streaming media;

at least one of the nodes specifies, automatically by the computer without human user interaction, delayed creation of one of the plurality of nodes as a delayed node when the media timeline is rendered, wherein the delayed creation includes creating the delayed node when called by one or more applications;

the plurality of nodes are arranged in a hierarchy to include a parent node and a child node; and

the child node is configured for initiating an event for communication to the parent node, wherein the event;

is configured such that a change may be made to one or more properties of the child node while the media timeline is rendered; and

describes the change such that additional nodes associated with the child node are dynamically created, automatically by the computer without human user interaction, in accordance with the communicated event.

63. An application programming interface as described in claim 62, wherein another node, which is not a parent of the child node, subscribes to the child node to receive the event.

64. An application programming interface as described in claim 62, wherein another node subscribes to the child node to receive:

the event initiated by the child node; and

one or more events initiated by children of the child node.

65. An application programming interface as described in claim 62, wherein the event describes a change made to the media timeline, the event selected from the group consisting of:

node added event;

node removed event;

node changing event;

node changed event;

remove children event;

node source added event;

node source removed event;

node sort event; and

node moved event.

66. An application programming interface as described in claim 62, wherein:

one node of the media timeline is configured as a root node; and

each event generated by one of the plurality of nodes that is a child of the root node is communicated to the root node.

67. An application programming interface as described in claim 62, wherein the media timeline is configured for dynamic loading such that metadata included in at least one node specifies a collection of nodes to be loaded when the media timeline is rendered.

68. An application programming interface as described in claim 62, wherein at least one node is configured to reference an effect to be applied to an output of media referenced by the node.

69. An application programming interface as described in claim 62, wherein at least one node is specified as read-only.

70. An application programming interface stored on a memory of a computer that, when interacted with by an application facilitates actions comprising:

exposing a media timeline comprising two or more nodes to the application; and

enabling the application to call any of the two or more nodes, wherein each of the two or more nodes;

references corresponding media, at least one of the corresponding media comprising streaming media while another of the corresponding media does not include streaming media;

includes metadata describing one or more properties for rendering the corresponding media; and

includes metadata specifying the node as read-only; and

configuring, automatically by the computer without human user interaction, the media timeline for dynamic creation such that at least one of the two or more nodes is created while the media timeline is being rendered; and

specifying, automatically by the computer without human user interaction, delayed creation of one of the two or more nodes as a delayed node when the media timeline is rendered, wherein the delayed creation includes creating the delayed node when called by one or more applications.

71. A system comprising:

a plurality of media comprising streaming media;

a plurality of applications; and

an infrastructure layer that;

provides an application programming interface (API) for interaction by the plurality of applications with the plurality of media when any application is executed; and

exposes a media timeline, callable by the plurality of applications via the API upon an execution thereof, and that defines a presentation of the plurality of media, wherein the media timeline;

includes a plurality of nodes that each reference respective media;

is configured to specify, automatically by a computer without human user interaction, delayed creation such that at least one node is not created until called by one or more of the plurality of applications;

is configured for dynamic creation, automatically by the computer without human user interaction, such that at least one node is created while the media timeline is rendered; and

is configured for dynamic loading such that metadata included in the at least one node created specifies a collection of nodes to be loaded when the media timeline is rendered.

72. A system as described in claim 71, wherein the media timeline is configured to reference an effect for application to an output of one or more of the plurality of media.

73. A system as described in claim 71, wherein:

the media timeline defines a presentation of a first media referenced by a first node with respect to a second media referenced by a second node; and

at least one node includes metadata that describes rendering of the at least one node.

74. A system as described in claim 71, wherein at least one node is specified as read-only.

75. A system as described in claim 71, wherein at least one said node is configured for communication of events to another node such that a change may be made to the media timeline while the media timeline is rendered.

76. A computer comprising:

a processor; and

memory configured to maintain;

a plurality of media;

a plurality of applications, wherein each application is configured to request at least one of editing, encoding, and rendering of the plurality of media;

an infrastructure layer configured to;

provide an application programming interface (API) for interaction by the plurality of applications with the plurality of media; and

expose a media timeline, callable by the plurality of applications via the API, which includes a plurality of nodes that define a presentation of the plurality of media, wherein the media timeline specifies;

dynamic creation such that at least one of the plurality of nodes is created while the media timeline is being rendered automatically by the computer without human user interaction;

delayed creation of one or more nodes when the media timeline is rendered, wherein the delayed creation comprises creating the one or more nodes, automatically by the computer without human user interaction, when called by one or more applications; and

metadata that is utilized by the plurality of applications, wherein the metadata describes;

 

initiating rendering of the plurality of nodes is to be initiated;

 

properties and interrelationships of the plurality of nodes;

 

node types of the plurality of nodes; and

 

dynamic changes to the media timeline such that a group of nodes affected by a modification to an associated node are automatically updated in accordance with the modification as specified in the properties and interrelationships of the plurality of nodes;

at least one node that is configured for communication of events to another node such that a change may be made to the media timeline while the media timeline is being rendered; and

at least one node that is a parallel node that provides simultaneous rendering of at least two child nodes the child nodes including respective metadata and having respective pointers to respective media.

77. A computer as described in claim 76, wherein the media timeline is configured for dynamic loading such that metadata included in at least one node specifies a collection of nodes to be loaded when the media timeline is rendered.

78. A computer as described in claim 76, wherein at least one node is configured to reference an effect to be applied to an output of media referenced by the node.