Interfaces for digital media processing

• US 8,612,643 B2
• Filed: 06/30/2007
• Issued: 12/17/2013
• Est. Priority Date: 06/30/2007
• Status: Expired due to Fees

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Rangarajan, Rajasekaran, Regen, Martin, Russell, Richard W.
• Primary Examiner(s): Tsai, Henry
• Assistant Examiner(s): Shyu, Jing-Yih
• Application Number:
  US11/824,720
• Publication Number:
  US 20090007159A1
• Time in Patent Office: 2,362 Days
• Field of Search: None
• US Class Current: 710/22
• CPC Class Codes: G06F 3/0611 in relation to response time
• CPC Class Codes: G06F 3/0656 Data buffering arrangements
• CPC Class Codes: G06F 9/541 via adapters, e.g. between incompatible applications
• CPC Class Codes: G06F 9/544 Buffers; Shared memory; Pipes
• CPC Class Codes: G11B 20/10527 Audio or video recording; Data buffering arrangements G11B20/12 - G11B20/18 take precedence
• CPC Class Codes: G11B 2020/10537 Audio or video recording
• CPC Class Codes: G11B 2020/10666 Ring buffers, e.g. buffers wherein an iteratively progressing read or write pointer moves back to the beginning of the buffer when reaching the last storage cell
• CPC Class Codes: G11B 2020/10759 content data
• CPC Class Codes: G11B 2020/10944 Real-time recording or reproducing, e.g. for ensuring seamless playback of AV data
• CPC Class Codes: G11B 2220/2562 DVDs [digital versatile discs]; Digital video discs; MMCDs; HDCDs
• CPC Class Codes: H04L 65/762 at the source reformatting of additional data in video distribution servers H04N21/2355
• CPC Class Codes: H04N 21/42646 for reading from or writing on a non-volatile solid state storage medium, e.g. DVD, CD-ROM
• CPC Class Codes: H04N 21/4325 by playing back content from the storage medium reproduction of recorded television signals H04N5/76; reproduction of recorded television signals H04N9/79
• CPC Class Codes: H04N 21/4431 characterized by the use of Application Program Interface [API] libraries
• CPC Class Codes: H04N 5/85 on discs or drums

Abstract

APIs discussed herein promote efficient and timely interoperability between hardware and software components within the media processing pipelines of media content players. A PhysMemDataStructure API facilitates a hardware component'"'"'s direct access to information within a memory used by a software component, to enable the hardware component to use direct memory access techniques to obtain the contents of the memory, instead of using processor cycles to execute copy commands. The PhysMemDataStructure API exposes one or more fields of data structures associated with units of media content stored in a memory used by a software component, and the exposed fields store information about the physical properties of the memory locations of the units of media content. SyncHelper APIs are used for obtaining information from, and passing information to, hardware components, which information is used to adjust the hardware components'"'"' timing for preparing media samples of synchronously-presentable media content streams.

Claims

1. A computer-readable medium, not consisting of a propagated data signal, encoded with computer-executable instructions which, when executed by a processor, perform a method for preparing media content as a media presentation by a software-based media player including a decoder/renderer, the media content receivable from a media source as a plurality of media content units, the method comprising:

identifying a portion of a first memory having blocks that are configured to be separately allocated, the portion allocated for storing media content units comprising individually-presentable portions of clips received from the media source, the individually-presentable portions of clips being multiplexed into a single program stream comprising the media presentation received at a hardware component, the hardware component providing scheduling information, responsively to a timing signal, to the decoder/renderer to enable time-based synchronization of the individually-presentable portions of clips,

wherein the portion of the first memory allocated for storing media content units received from the media source is arranged as a ring buffer, the ring buffer having a beginning memory block and an ending memory block, and

wherein the ring buffer is implemented using a begin pointer for referencing the beginning of used memory in the ring buffer and an end pointer for referencing the end of used memory in the ring buffer, and wherein the step of identifying storage locations for each of a plurality of media content units in the allocated portion of the first memory comprises identifying an offset of the first media content unit within the ring buffer, the offset specified relative to the begin pointer or the end pointer or both;

identifying a plurality of media content units received from the media source;

identifying storage locations and offsets within the ring buffer in which each of the plurality of media content units has been stored in the allocated portion of the first memory;

forming data structures associated with each of the plurality of media content units, the data structures each having a field for storing information about the storage location of a particular media content unit;

arranging for exposure of the data structures to a hardware component having a second memory, the information about the storage locations of the particular media content units obtained from the data structures usable by the hardware component to directly access the particular media content units from the first memory without using a central processing unit; and

after a particular media content unit has been accessed, releasing the particular media content unit from the storage location by moving the begin pointer or the end pointer or both.

2. The computer-readable medium according to claim 1, wherein the portion of the first memory allocated for storing media content units received from the media source is selected from the group consisting of:

a contiguous physical memory, a locked scatter-gathered physical memory, an unlocked scatter-gathered physical memory, a cached virtual memory, and an uncached virtual memory, and wherein the information about the first storage location exposed by the first data structure is selected from the group consisting of;

a type of the first memory;

a size of a memory block of the first memory;

a location of a pointer to the first memory; and

an offset location of the first storage location with respect to a pointer to the first memory.

3. The computer-readable medium according to claim 1, wherein the beginning memory block is duplicated, using virtual memory, after the ending memory block.

4. The computer-readable medium according to claim 3, wherein the method further comprises the step of:

when the first storage location would include the beginning memory block and the ending memory block, storing a first portion of the first media content unit in the ending memory block and a second portion of the first media content unit in the beginning memory block duplicated using virtual memory.

5. The computer-readable medium according to claim 1, the method further comprising:

after the first media content unit has been transmitted, releasing the first media content unit from the first storage location by moving the begin pointer or the end pointer or both.

6. The computer-readable medium according to claim 1, further comprising:

maintaining a list of offset positions including storage locations of all media content units stored in the ring buffer; and

using the list of offset positions to ensure that the begin pointer and the end pointer do not bypass each other when media content units are released from the storage locations.

7. The computer-readable medium according to claim 1, wherein the method is performed within a media processing pipeline comprising a filter graph having a plurality of filters, each filter having one or more input pins and one or more output pins.

8. The computer-readable medium according to claim 7, wherein the hardware component comprises one of the plurality of filters, and wherein the hardware component is selected from the group consisting of:

a demultiplexer, a decoder, and a renderer.

9. The computer-readable medium according to claim 7, wherein the method is performed by a software component comprising one of the plurality of filters, the software component having an input pin configured to receive media content units and an output pin configured for communication with the hardware component.

10. A computer-readable medium, not consisting of a propagated data signal, encoded with computer-executable instructions which, when executed by a processor, perform a method for preparing media content as a media presentation by a software-based media player including a decoder/renderer, the media content receivable from a media source as a plurality of media content units, the method comprising:

identifying a portion of a first memory having blocks that are configured to be separately allocated, the portion allocated for storing media content units comprising individually-presentable portions of clips received from the media source, the individually-presentable portions of clips being multiplexed into a single program stream comprising the media presentation received at a hardware component, the hardware component providing scheduling information, responsively to a timing signal, to the decoder/renderer to enable time-based synchronization of the individually-presentable portions of clips;

identifying a first media content unit received from the media source;

identifying a first storage location in which the first media content unit has been stored in the allocated portion of the first memory;

forming a first data structure associated with the first media content unit, the first data structure having a field for storing information about the first storage location;

arranging for exposing the first data structure to the hardware component having a second memory;

the information about the first storage location obtained from the first data structure usable by the hardware component to directly access the first media content unit from the first memory without using a central processing unit;

wherein the method is performed within a media processing pipeline comprising a filter graph having a plurality of filters, each filter having one or more input pins and one or more output pins;

wherein the method is performed by a software component comprising one of the plurality of filters, the software component having an input pin configured to receive media content units and an output pin configured for communication with the hardware component; and

wherein the method step of forming a first data structure associated with the first media content unit, the first data structure having a field for storing information about the first storage location, comprises;

issuing, by the software component, a call to an application programming interface (“

API”

) configured to receive information about the first storage location and to populate the field of the first data structure with the information about the first storage location; and

receiving, by the software component, a response from the application programming interface exposing the field of the first data structure with the information about the first storage location.

11. The computer-readable medium according to claim 10, wherein the step of arranging for exposing the first data structure to a hardware component comprises transmitting, on the output pin configured for communication with the hardware component, the first data structure with the field of the first data structure exposed.

12. A computer-readable medium, not consisting of a propagated data signal, encoded with computer-executable instructions which, when executed by a processor, perform a method for playing a digital media presentation having a play duration and a media content component, the media content component comprising

a first clip arranged into a first plurality of media samples for rendering in a multiplexed media content stream, the first clip having a first play duration and playable by a decoder/renderer and

a second clip arranged into a second plurality of media samples for rendering in the multiplexed media content stream, the second clip having a second play duration and playable by the decoder/renderer, the method comprising:

identifying a first media sample from the first clip;

identifying a second media sample from the second clip, the second media sample synchronously playable with the first media sample;

at a first time associated with preparing the first media sample for presentation using a first hardware component at a rate based on a first timing signal,

ascertaining a first elapsed amount of the play duration of the digital media presentation, and

ascertaining an elapsed amount of the first play duration;

at a second time associated with preparing the second media sample for presentation using a second hardware component at a rate based on a second timing signal,

ascertaining a second elapsed amount of the play duration of the digital media presentation, and

ascertaining an elapsed amount of the second play duration;

ascertaining a difference between the first elapsed amount of the play duration of the digital media presentation and the second elapsed amount of the play duration of the digital media presentation, the ascertaining utilizing scheduling information provided by either the first or second hardware component, responsively to a received timing signal, to the decoder/renderer during playback of the media presentation to enable time-based synchronization of the first and second clips in the multiplexed media content stream; and

based on the difference, adjusting either the first time or the second time or both;

wherein the steps of ascertaining the first elapsed amount of the play duration of the digital media presentation and the elapsed amount of the first play duration further comprise;

issuing a first call to a first application programming interface (“

API”

), the first call including information about the first media sample; and

based on the first call, receiving a first response from the first API, the first response including information about the first elapsed amount of the play duration of the digital media presentation and the elapsed amount of the first play duration, and

wherein the steps of ascertaining the second elapsed amount of the play duration of the digital media presentation and the elapsed amount of the second play duration further comprise;

issuing a second call to the first API, the second call including information about the second media sample; and

based on the second call, receiving a second response from the first API, the second response including information about the second elapsed amount of the play duration of the digital media presentation and the elapsed amount of the second play duration.

13. The computer-readable medium according to claim 12, wherein the first and second hardware components are selected from the group consisting of:

demultiplexers;

decoders; and

renderers.

14. The computer-readable medium according to claim 12, wherein the step of adjusting either the first time or the second time or both further comprises:

issuing a call to a second API, the call including information selected from the group consisting of;

the first response, the second response, and the difference between the first elapsed amount of the play duration of the digital media presentation and the second elapsed amount of the play duration of the digital media presentation;

based on the call to the second API, receiving a response from the second API; and

based on the response from the second API, adjusting one or more of the group consisting of;

the elapsed amount of the first play duration;

the elapsed amount of the second play duration;

the first timing signal; and

the second timing signal.