Interfaces for digital media processing

• US 9,043,504 B2
• Filed: 12/16/2013
• Issued: 05/26/2015
• Est. Priority Date: 06/30/2007
• Status: Expired due to Fees

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: Rangarajan, Rajasekaran, Regen, Martin, Russell, Richard Gains
• Primary Examiner(s): Shyu, Jing-Yih
• Application Number:
  US14/107,529
• Publication Number:
  US 20140281054A1
• Time in Patent Office: 526 Days
• US Class Current: 710/22
• CPC Class Codes: G06F 3/0611 in relation to response time
• CPC Class Codes: G06F 3/0656 Data buffering arrangements
• CPC Class Codes: G06F 9/541 via adapters, e.g. between incompatible applications
• CPC Class Codes: G06F 9/544 Buffers; Shared memory; Pipes
• CPC Class Codes: G11B 20/10527 Audio or video recording; Data buffering arrangements G11B20/12 - G11B20/18 take precedence
• CPC Class Codes: G11B 2020/10537 Audio or video recording
• CPC Class Codes: G11B 2020/10666 Ring buffers, e.g. buffers wherein an iteratively progressing read or write pointer moves back to the beginning of the buffer when reaching the last storage cell
• CPC Class Codes: G11B 2020/10759 content data
• CPC Class Codes: G11B 2020/10944 Real-time recording or reproducing, e.g. for ensuring seamless playback of AV data
• CPC Class Codes: G11B 2220/2562 DVDs [digital versatile discs]; Digital video discs; MMCDs; HDCDs
• CPC Class Codes: H04L 65/762 at the source reformatting of additional data in video distribution servers H04N21/2355
• CPC Class Codes: H04N 21/42646 for reading from or writing on a non-volatile solid state storage medium, e.g. DVD, CD-ROM
• CPC Class Codes: H04N 21/4325 by playing back content from the storage medium reproduction of recorded television signals H04N5/76; reproduction of recorded television signals H04N9/79
• CPC Class Codes: H04N 21/4431 characterized by the use of Application Program Interface [API] libraries
• CPC Class Codes: H04N 5/85 on discs or drums

Abstract

APIs discussed herein promote efficient and timely interoperability between hardware and software components within the media processing pipelines of media content players. A PhysMemDataStructure API facilitates a hardware component'"'"'s direct access to information within a memory used by a software component, to enable the hardware component to use direct memory access techniques to obtain the contents of the memory, instead of using processor cycles to execute copy commands. The PhysMemDataStructure API exposes one or more fields of data structures associated with units of media content stored in a memory used by a software component, and the exposed fields store information about the physical properties of the memory locations of the units of media content. SyncHelper APIs are used for obtaining information from, and passing information to, hardware components, which information is used to adjust the hardware components'"'"' timing for preparing media samples of synchronously-presentable media content streams.

Claims

1. A computer-readable medium, not consisting of a propagated data signal, encoded with computer-executable instructions which, when executed by a processor, perform a method for preparing media content as a media presentation by a software-based media player including a decoder/renderer, the media content receivable from a media source as a plurality of media content units, the method comprising:

identifying a ring buffer having a beginning memory block and an ending memory block that are configured to be separately allocated, the ring buffer allocated for storing media content units received from the media source which include individually-presentable portion of clips that form the media presentation received at a component, the component providing scheduling information, responsive to a timing signal, to the decoder/renderer to enable time-based synchronization of the individually-presentable portions of clips,

wherein the ring buffer is implemented using a begin pointer for referencing the beginning of used memory in the ring buffer and an end pointer for referencing the end of used memory in the ring buffer;

identifying an offset of a first media content unit within the ring buffer, the offset specified relative to the begin pointer or the end pointer or both;

identifying a plurality of media content units received from the media source;

identifying storage locations and offsets within the ring buffer in which each of the plurality of media content units has been stored;

forming data structures associated with each of the plurality of media content units, the data structures each having a field for storing information about the storage location of a particular media content unit;

arranging for exposure of the data structures to a memory, the information about the storage locations of the particular media content units obtained from the data structures usable by the memory to directly access the particular media content unit from the ring buffer; and

after accessing a particular media content unit, releasing the particular media content unit from the storage location by moving the begin pointer or the end pointer or both.

2. The computer-readable medium according to claim 1, wherein the ring buffer is selected from the group consisting of:

a contiguous physical memory, a locked scatter-gathered physical memory, an unlocked scatter-gathered physical memory, a cached virtual memory, and an uncached virtual memory, and wherein the information about the storage locations exposed by the data structures is selected from the group consisting of;

a type of the ring buffer;

a size of a memory block of the ring buffer;

a location of a pointer to the ring buffer; and

an offset location of the first storage location with respect to a pointer to the ring buffer.

3. The computer-readable medium according to claim 1, wherein the beginning memory block is duplicated, using virtual memory, after the ending memory block.

4. The computer-readable medium according to claim 3, wherein the method further comprises the step of:

when a first storage location would include the beginning memory block and the ending memory block, storing a first portion of the first media content unit in the ending memory block and a second portion of the first media content unit in the beginning memory block duplicated using virtual memory.

5. The computer-readable medium according to claim 4, the method further comprising:

after the first media content unit has been transmitted, releasing the first media content unit from the first storage location by moving the begin pointer or the end pointer or both.

6. The computer-readable medium according to claim 1, further comprising:

maintaining a list of offset positions including storage locations of all media content units stored in the ring buffer; and

using the list of offset positions to ensure that the begin pointer and the end pointer do not bypass each other when media content units are released from the storage locations.

7. The computer-readable medium according to claim 1, wherein the method is performed within a media processing pipeline comprising a filter graph having a plurality of filters, each filter having one or more input pins and one or more output pins.

8. The computer-readable medium according to claim 7, wherein the component comprises one of the plurality of filters, and wherein the component is a hardware component.

9. The computer-readable medium according to claim 8, wherein the hardware component is selected from the group consisting of:

a demultiplexer, a decoder, and a renderer.

10. The computer-readable medium according to claim 7, wherein the method is performed by a software component comprising one of the plurality of filters, the software component having an input pin configured to receive media content units and an output pin configured for communication with the component.

11. A computer-readable medium, not consisting of a propagated data signal, encoded with computer-executable instructions which, when executed by a processor, perform a method for preparing media content as a media presentation by a software-based media player including a decoder/renderer, the media content receivable from a media source as a plurality of media content units, the method comprising:

identifying a first storage location in which a first media content unit has been stored in an allocated portion of a first memory for storing media content units comprising individually-presentable portions of clips received from the media source, the individually-presentable portions of clips being multiplexed into a single program stream comprising the media presentation received at a first component providing scheduling information to the decoder/renderer to enable time-based synchronization of the individually-presentable portions of clips;

forming a first data structure associated with the first media content unit, the first data structure having a field for storing information about the first storage location;

arranging for exposing the first data structure to the first component;

the information about the first storage location obtained from the first data structure usable to access the first media content unit from the first memory;

wherein the method is performed by a software component comprising one of the plurality of filters, the software component having an input pin configured to receive media content units and an output pin configured for communication with the first component; and

wherein forming the first data structure includes;

using the software component to issue a call to receive information about the first storage location and to populate the field of the first data structure with the information about the first storage location; and

receiving a response exposing the field of the first data structure with the information about the first storage location.

12. The computer-readable medium according to claim 11, wherein the step of arranging for exposing the first data structure to a first component comprises transmitting, on the output pin configured for communication with the first component, the first data structure with the field of the first data structure exposed.

13. The computer-readable medium according to claim 11, wherein the allocated portion of the first memory allocated for storing media content units received from the media source is arranged as a ring buffer, the ring buffer having a beginning memory block and an ending memory block, and

wherein the ring buffer is implemented using a begin pointer for referencing the beginning of used memory in the ring buffer and an end pointer for referencing the end of used memory in the ring buffer.

14. The computer-readable medium according to claim 13, wherein the ring buffer is selected from the group consisting of:

a contiguous physical memory, a locked scatter-gathered physical memory, an unlocked scatter-gathered physical memory, a cached virtual memory, and an uncached virtual memory, and wherein the information about the first storage location exposed by the first data structure is selected from the group consisting of;

a type of the ring buffer;

a size of a memory block of the ring buffer;

a location of a pointer to the ring buffer; and

an offset location of the first storage location with respect to a pointer to the ring buffer.

15. The computer-readable medium according to claim 13, wherein the beginning memory block is duplicated, using virtual memory, after the ending memory block.

16. A computer-readable medium, not consisting of a propagated data signal, encoded with computer-executable instructions which, when executed by a processor, perform a method for playing a digital media presentation having a play duration and a media content component, the media content component comprising:

a first clip arranged into a first plurality of media samples for rendering in a multiplexed media content stream, the first clip having a first play duration and playable by a decoder/renderer and

a second clip arranged into a second plurality of media samples for rendering in the multiplexed media content stream, the second clip having a second play duration and playable by the decoder/renderer, the method comprising;

at a first time associated with preparing a first media sample from the first clip for presentation based on a first timing signal,

ascertaining a first elapsed amount of the play duration of the digital media presentation, and

ascertaining an elapsed amount of the first and second play durations;

at a second time associated with preparing a second media sample from the second clip for presentation at a rate based on a second timing signal, the second media sample synchronously playable with the first media sample,

ascertaining a second elapsed amount of the play duration of the digital media presentation, and

ascertaining a difference between the first elapsed amount of the play duration of the digital media presentation and the second elapsed amount of the play duration of the digital media presentation, the ascertaining utilizing scheduling information provided responsive to a received timing signal, to the decoder/renderer during playback of the media presentation to enable synchronization of the first and second clips in the multiplexed media content stream; and

based on the difference, adjusting either the first time or the second time or both;

wherein the steps of ascertaining the first elapsed amount of the play duration of the digital media presentation and the elapsed amount of the first play duration further comprise;

based on a first call that includes information about the first media sample, receiving a first response that includes information about the first elapsed amount of the play duration of the digital media presentation and the elapsed amount of the first play duration, and

wherein the steps of ascertaining the second elapsed amount of the play duration of the digital media presentation and the elapsed amount of the second play duration further comprise;

based on a second call that includes information about the second media sample, receiving a second response, the second response including information about the second elapsed amount of the play duration of the digital media presentation and the elapsed amount of the second play duration.

17. The computer-readable medium according to claim 16 wherein the scheduling information is provided by a hardware component.

18. The computer-readable medium according to claim 17 wherein the hardware component is selected from the group consisting of:

demultiplexers;

decoders; and

renderers.

19. The computer-readable medium according to claim 16 wherein preparation of the first media sample from the first clip for presentation uses a first hardware component at a rate based on a first timing signal and preparation of the second media sample from the second clip for presentation uses a second hardware component at a rate based on a second timing signal.

20. The computer-readable medium according to claim 19, wherein the step of adjusting either the first time or the second time or both further comprises:

issuing a call to an Application Programing Interface (API), the call including information selected from the group consisting of;

the first response, the second response, and the difference between the first elapsed amount of the play duration of the digital media presentation and the second elapsed amount of the play duration of the digital media presentation;

based on the call to the API, receiving a response from the API; and

based on the response from the API, adjusting one or more of the group consisting of;

the elapsed amount of the first play duration;

the elapsed amount of the second play duration;

the first timing signal; and

the second timing signal.