Interfaces for digital media processing

• US 20090007159A1
• Filed: 06/30/2007
• Published: 01/01/2009
• Est. Priority Date: 06/30/2007
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Russell, Richard W., Regen, Martin, Rangarajan, Rajasekaran
• Granted Patent: US 8,612,643 B2
• Time in Patent Office: Days
• US Class Current: 719/328
• CPC Class Codes: G06F 3/0611 in relation to response time
• CPC Class Codes: G06F 3/0656 Data buffering arrangements
• CPC Class Codes: G06F 9/541 via adapters, e.g. between incompatible applications
• CPC Class Codes: G06F 9/544 Buffers; Shared memory; Pipes
• CPC Class Codes: G11B 20/10527 Audio or video recording; Data buffering arrangements G11B20/12 - G11B20/18 take precedence
• CPC Class Codes: G11B 2020/10537 Audio or video recording
• CPC Class Codes: G11B 2020/10666 Ring buffers, e.g. buffers wherein an iteratively progressing read or write pointer moves back to the beginning of the buffer when reaching the last storage cell
• CPC Class Codes: G11B 2020/10759 content data
• CPC Class Codes: G11B 2020/10944 Real-time recording or reproducing, e.g. for ensuring seamless playback of AV data
• CPC Class Codes: G11B 2220/2562 DVDs [digital versatile discs]; Digital video discs; MMCDs; HDCDs
• CPC Class Codes: H04L 65/762 at the source reformatting of additional data in video distribution servers H04N21/2355
• CPC Class Codes: H04N 21/42646 for reading from or writing on a non-volatile solid state storage medium, e.g. DVD, CD-ROM
• CPC Class Codes: H04N 21/4325 by playing back content from the storage medium reproduction of recorded television signals H04N5/76; reproduction of recorded television signals H04N9/79
• CPC Class Codes: H04N 21/4431 characterized by the use of Application Program Interface [API] libraries
• CPC Class Codes: H04N 5/85 on discs or drums

Abstract

APIs discussed herein promote efficient and timely interoperability between hardware and software components within the media processing pipelines of media content players. A PhysMemDataStructure API facilitates a hardware component'"'"'s direct access to information within a memory used by a software component, to enable the hardware component to use direct memory access techniques to obtain the contents of the memory, instead of using processor cycles to execute copy commands. The PhysMemDataStructure API exposes one or more fields of data structures associated with units of media content stored in a memory used by a software component, and the exposed fields store information about the physical properties of the memory locations of the units of media content. SyncHelper APIs are used for obtaining information from, and passing information to, hardware components, which information is used to adjust the hardware components'"'"' timing for preparing media samples of synchronously-presentable media content streams.

Claims

1. A computer-readable medium encoded with computer-executable instructions which, when executed by a processor, perform a method for preparing media content for presentation, the media content receivable from a media source as a plurality of media content units, the method comprising:

identifying a portion of a first memory allocated for storing media content units received from the media source;

identifying a first media content unit received from the media source;

identifying a first storage location for the first media content unit in the allocated portion of the first memory;

forming a first data structure associated with the first media content unit, the first data structure having a field for storing information about the first storage location; and

arranging for exposing the first data structure to a hardware component having a second memory,

the information about the first storage location obtained from the first data structure usable by the hardware component to directly access the first media content unit from the first memory without using a central processing unit.

2. The computer-readable medium according to claim 1, wherein the portion of the first memory allocated for storing media content units received from the media source is selected from the group consisting of:

a contiguous physical memory, a locked scatter-gathered physical memory, an unlocked scatter-gathered physical memory, a cached virtual memory, and an uncached virtual memory, and wherein the information about the first storage location exposed by the first data structure is selected from the group consisting of;

a type of the first memory;

a size of a memory block of the first memory;

a location of a pointer to the first memory; and

an offset location of the first storage location with respect to a pointer to the first memory.

3. The computer-readable medium according to claim 1, wherein the portion of the first memory allocated for storing media content units received from the media source is arranged as a ring buffer, the ring buffer having a beginning memory block and an ending memory block.

4. The computer-readable medium according to claim 3, wherein the beginning memory block is duplicated, using virtual memory, after the ending memory block.

5. The computer-readable medium according to claim 4, wherein the method further comprises the step of:

when the first storage location would include the beginning memory block and the ending memory block, storing a first portion of the first media content unit in the ending memory block and a second portion of the first media content unit in the beginning memory block duplicated using virtual memory.

6. The computer-readable medium according to claim 3, wherein the ring buffer is implemented using a begin pointer for referencing the beginning of used memory in the ring buffer and an end pointer for referencing the end of used memory in the ring buffer, and wherein the step of identifying a first storage location for the first media content unit in the allocated portion of the first memory comprises identifying an offset of the first media content unit within the ring buffer, the offset specified relative to the begin pointer or the end pointer or both.

7. The computer-readable medium according to claim 6, the method further comprising:

after the first media content unit has been transmitted, releasing the first media content unit from the first storage location by moving the begin pointer or the end pointer or both.

8. The computer-readable medium according to claim 6, further comprising:

identifying a plurality of media content units received from the media source;

identifying storage locations and offsets within the ring buffer for each of the plurality of media content units;

forming data structures associated with each of the plurality of media content units, the data structures each having a field for storing information about the storage location of a particular media content unit;

arranging for exposure of the data structures to a hardware component having a second memory, the information about the storage locations of the particular media content units obtained from the data structures usable by the hardware component to directly access the particular media content units from the first memory without using a central processing unit; and

after a particular media content unit has been accessed, releasing the particular media content unit from the storage location by moving the begin pointer or the end pointer or both.

9. The computer-readable medium according to claim 8, further comprising:

maintaining a list of offset positions including storage locations of all media content units stored in the ring buffer; and

using the list of offset positions to ensure that the begin pointer and the end pointer do not bypass each other when media content units are released from the storage locations.

10. The computer-readable medium according to claim 1, wherein the method is performed within a media processing pipeline comprising a filter graph having a plurality of filters, each filter having one or more input pins and one or more output pins.

11. The computer-readable medium according to claim 10, wherein the hardware component comprises one of the plurality of filters, and wherein the hardware component is selected from the group consisting of:

a demultiplexer, a decoder, and a renderer.

12. The computer-readable medium according to claim 10, wherein the method is performed by a software component comprising one of the plurality of filters, the software component having an input pin configured to receive media content units and an output pin configured for communication with the hardware component.

13. The computer-readable medium according to claim 12, wherein the method step of forming a first data structure associated with the first media content unit, the first data structure having a field for storing information about the first storage location, comprises:

issuing, by the software component, a call to an application programming interface (“

API”

) configured to receive information about the first storage location and to populate the field of the first data structure with the information about the first storage location; and

receiving, by the software component, a response from the application programming interface exposing the field of the first data structure with the information about the first storage location.

14. The computer-readable medium according to claim 13, wherein the step of arranging for exposing the first data structure to a hardware component comprises transmitting, on the output pin configured for communication with the hardware component, the first data structure with the field of the first data structure exposed.

15. An architecture for a media processing pipeline for use in a media presentation system, the architecture comprising:

a software component configured to receive media content units and to arrange for storage of the received media content units in a first memory; and

a hardware component having a second memory, the hardware component responsive to the software component and configured to transfer representations of media content units stored in the first memory to the second memory; and

an API operable at a boundary between the software component and the hardware component, the API configured to expose, at the request of the first software component, at least one data structure field having information about the storage locations of particular media content units within the first memory to first hardware component, the exposed data structure field enabling the first hardware component to directly transfer representations of media content units stored in the first memory to the second memory without using a central processing unit.

16. The architecture according to claim 15, wherein the software component and the hardware component comprise filters in a filter graph, the software component has an output pin that connects to an input pin of the hardware component, and the API is responsive to a call from the output pin of the software component to expose the data structure field to the input pin of the hardware component.

17. A computer-readable medium encoded with computer-executable instructions which, when executed by a processor, perform a method for playing a digital media presentation having a play duration and a media content component, the media content component comprising

a first clip arranged into a first plurality of media samples, the first clip having a first play duration and playable as a first media content stream, and

a second clip arranged into a second plurality of media samples, the second clip having a second play duration and playable as a second media content stream, the method comprising:

identifying a first media sample from the first clip;

identifying a second media sample from the second clip, the second media sample synchronously playable with the first media sample;

at a first time associated with preparing the first media sample for presentation using a first hardware component at a rate based on a first timing signal,

ascertaining a first elapsed amount of the play duration of the digital media presentation, and

ascertaining an elapsed amount of the first play duration;

at a second time associated with preparing the second media sample for presentation using a second hardware component at a rate based on a second timing signal,

ascertaining a second elapsed amount of the play duration of the digital media presentation, and

ascertaining an elapsed amount of the second play duration;

ascertaining a difference between the first elapsed amount of the play duration of the digital media presentation and the second elapsed amount of the play duration of the digital media presentation; and

based on the difference, adjusting either the first time or the second time or both.

18. The computer-readable medium according to claim 17, wherein the first and second hardware components are selected from the group consisting of:

demultiplexers;

decoders; and

renderers.

19. The computer-readable medium according to claim 17, wherein the steps of ascertaining the first elapsed amount of the play duration of the digital media presentation and the elapsed amount of the first play duration further comprise:

issuing a first call to a first API, the first call including information about the first media sample; and

based on the first call, receiving a first response from the first API, the first response including information about the first elapsed amount of the play duration of the digital media presentation and the elapsed amount of the first play duration, and

wherein the steps of ascertaining the second elapsed amount of the play duration of the digital media presentation and the elapsed amount of the second play duration further comprise;

issuing a second call to the first API, the second call including information about the second media sample; and

based on the second call, receiving a second response from the first API, the second response including information about the second elapsed amount of the play duration of the digital media presentation and the elapsed amount of the second play duration.

20. The computer-readable medium according to claim 19, wherein the step of adjusting either the first time or the second time or both further comprises:

issuing a call to a second API, the call including information selected from the group consisting of;

the first response, the second response, and the difference between the first elapsed amount of the play duration of the digital media presentation and the second elapsed amount of the play duration of the digital media presentation;

based on the call to the second API, receiving a response from the second API; and

based on the response from the second API, adjusting one or more of the group consisting of;

the elapsed amount of the first play duration;

the elapsed amount of the second play duration;

the first timing signal; and

the second timing signal.