Media stream slicing and processing load allocation for multi-user media systems

• US 8,380,864 B2
• Filed: 12/27/2006
• Issued: 02/19/2013
• Est. Priority Date: 12/27/2006
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Bowra, Todd, Speicher, Ashley C., Fang, Nicholas J.
• Primary Examiner(s): Whipple, Brian P
• Application Number:
  US11/645,885
• Publication Number:
  US 20080162713A1
• Time in Patent Office: 2,246 Days
• Field of Search: 709/231
• US Class Current: 709/231
• CPC Class Codes: H04L 65/75 Media network packet handling
• CPC Class Codes: H04L 65/752 adapting media to network capabilities
• CPC Class Codes: H04L 67/56 Provisioning of proxy services store-and-forward switching systems in data switching networks H04L12/54
• CPC Class Codes: H04L 67/565 Conversion or adaptation of application format or content adding application control or application functional data H04L67/561
• CPC Class Codes: H04L 67/62 Establishing a time schedule for servicing the requests
• CPC Class Codes: H04N 21/4344 Remultiplexing of multiplex streams, e.g. by modifying time stamps or remapping the packet identifiers
• CPC Class Codes: H04N 21/43615 Interfacing a Home Network, e.g. for connecting the client to a plurality of peripherals home Audio Video Interoperability [HAVI] data switching networks H04L12/2805
• CPC Class Codes: H04N 21/440236 by media transcoding, e.g. video is transformed into a slideshow of still pictures, audio is converted into text

Abstract

Transrate/transcode functionality of media processing units in devices distributed within a networked media system are harnessed to enable media streaming to devices across networks while overcoming insufficient bandwidth to transmit the source content or while providing reformatted media to devices that do not support the source format. A policy engine may reallocate media processing resources for greater efficiency. The policy engine may further schedule time-slicing of media segments in available hardware and software-based codecs. Time-slicing may be used to transformatively process one or more media streams across potentially multiple media processing units connected within a network. The media streams are divided into independently processable segments based upon knowledge of underlying media formats. Processing requirements and a priority processing order may be determined for and assigned to each independently processable segment. The segments may be scheduled for delivery to the media processing units based upon the assigned priority and processing requirements.

Claims

1. A method in a computer system for transformatively processing two or more media streams across multiple media processing units of media devices connected within a network, the method comprising:

monitoring competing requests for processing the two or more media streams directed to a media device within the network;

allocating a media processing unit of the media device for processing a first media stream of the two or more media streams;

determining that processing resources of the allocated media processing unit of the media device are sufficient for simultaneous processing of more than one media stream but insufficient for simultaneous processing of all of the two or more media streams;

allocating available processing resources of one or more media processing units of other media devices connected within the network to split processing of a second media stream of the two or more media streams between the allocated media processing unit of the media device and the one or more media processing units of the other media devices;

dividing each of the two or more media streams into independently processable segments based upon media formats of the two or more media streams;

assigning priority for processing order to the independently processable segments;

determining processing requirements for each independently processable segment; and

scheduling the independently processable segments for delivery to the allocated media processing unit of the media device and the one or more media processing units of the other media devices based upon the assigned priority and the processing requirements for each independently processable segment.

2. The method of claim 1, further comprising:

dispatching the independently processable segments to the allocated media processing unit of the media device and the one or more media processing units of the other media devices based on the scheduling.

3. The method of claim 2, further comprising:

receiving a stream of processed segments corresponding to the second media stream from the allocated media processing unit of the media device;

receiving a stream of processed segments corresponding to the second media stream from the one of the one or more media processing units of the other media devices;

assembling the processed segments corresponding to the second media stream into a processed version of the second media stream; and

routing the processed version of the second media stream for output to a presentation device.

4. The method of claim 3, further comprising:

receiving a stream of processed segments corresponding to the first media stream from the allocated media processing unit of the media device; and

assembling the processed segments corresponding to the first media stream into a processed version of the first media stream; and

storing the processed version of first media stream as a media file on a storage device.

5. The method of claim 1, further comprising:

assigning priority to the two or more media streams based upon a hierarchy of requested end uses of the two or more media streams, the requested end uses including processing the first media stream for recording and processing the second media stream for real-time viewing.

6. The method of claim 1, wherein the dividing operation further comprises including identification data within each independently processable segment indicating an original media format and bit-rate of the respective media stream from which the independently processable segment is generated.

7. A computer-readable storage device storing computer-executable instructions that, when executed, cause a computing device to perform a method comprising:

monitoring competing requests for processing two or more media streams directed to a media device within a network;

allocating a media processing unit of the media device for processing a first media stream of the two or more media streams;

determining that processing resources of the allocated media processing unit of the media device are sufficient for simultaneous processing of more than one media stream but insufficient for simultaneous processing of all of the two or more media streams;

allocating available processing resources of one or more media processing units of other media devices connected within the network to split processing of a second media stream of the two or more media streams between the allocated media processing unit of the media device and the one or more media processing units of the other media devices;

dividing each of the two or more media streams into independently processable segments based upon media formats of the two or more media streams;

assigning a priority processing order to the independently processable segments;

determining processing requirements for each independently processable segment; and

scheduling the independently processable segments for delivery to the allocated media processing unit of the media device and the one or more media processing units of the other media devices based upon the assigned priority processing order and the processing requirements for each independently processable segment.

8. A method in a media processing unit of a media device within a network for processing independently processable segments of a media stream, the method comprising:

receiving competing requests for processing two or more media streams at the media device, the media processing unit of the media device having processing resources sufficient for simultaneous processing of more than one media stream but insufficient for simultaneous processing of all of the two or more media streams;

receiving, at the media processing unit of the media device from a media server within the network, a plurality of independently processable segments corresponding to all of a first media stream of the two or more media streams and a portion of a second media stream of the two or more media streams, wherein processing of the second media stream is split between the media processing unit of the media device and one or more media processing units of other media devices within the network;

performing a transformative processing operation on the plurality of independently processable segments; and

transmitting an output stream of processed segments to the media server for assembling a processed version of the first media stream from processed segments corresponding to the first media stream received from the media processing unit of the media device and assembling a processed version of the second media stream from processed segments corresponding to the second media stream received from the media processing unit of the media device and processed segments corresponding to the second media stream received from the one or more media processing units of the other media devices.

9. The method of claim 8, wherein the competing requests received by the media device comprise a request for processing the first media stream for recording and a request for processing the second media stream for real-time viewing.

10. The method of claim 8, wherein the transformative processing of individual independently processable segments occurs in an order that the individual independently processable segments are received at the media processing unit of the media device without regard to any relationship of the individual independently processable segments to the first or second media streams.

11. The method of claim 8, wherein the transformative processing of individual independently processable segments occurs in an order as determined by a processing priority assigned to each individual independently processable segment based on a respective priority of the first and second media streams.

12. The method of claim 8, wherein the transformative processing operation comprises one or more of an encode function, a decode function, a transcode function, or a transrate function.

13. The method of claim 8, wherein the transformative processing operation is performed using a hardware codec.

14. The method of claim 8, wherein the transformative processing operation is performed using a software-implemented codec.

15. A computer-readable storage device storing computer-executable instructions that, when executed, cause a computing device in a network to perform a method comprising:

receiving competing requests for processing two or more media streams;

receiving, at a media processing unit of the computing device from a media server within the network, a plurality of independently processable segments corresponding to all of a first media stream of the two or more media streams and a portion of a second media stream of the two or more media streams when processing resources of the media processing unit are sufficient for simultaneous processing of more than one media stream but insufficient for simultaneous processing of all of the two or more media streams, wherein processing of the second media stream is split between the media processing unit of the computing device and one or more media processing units of other media devices within the network;

performing a transformative processing operation on the plurality of independently processable segments; and

transmitting an output stream of processed segments to the media server, wherein the media server is configured to;

assemble a processed version of the first media stream from processed segments corresponding to the first media stream received from the media processing unit of the computing device, and

assemble a processed version of the second media stream from processed segments corresponding to the second media stream received from the media processing unit of the computing device and processed segments corresponding to the second media stream received from the one or more media processing units of the other media devices.

16. A system for transformatively processing two or more media streams across multiple media processing units of media devices connected within a network, the system comprising:

memory storing computer-executable program modules comprising;

a slicing engine module that divides the two or more media streams into independently processable segments based upon media formats of the two or more media streams;

a policy engine module that;

monitors competing requests for processing the two or more media streams directed to a media device within the network,

allocates a media processing unit of the media device for processing a first media stream of the two or more media streams,

determines that processing resources of the allocated media processing unit of the media device are sufficient for simultaneous processing of more than one media stream but insufficient for simultaneous processing of all of the two or more media streams,

allocates available processing resources of one or more media processing units of other media devices connected within the network to split processing of a second media stream of the two or more media streams between the allocated media processing unit of the media device and the one or more media processing units of the other media devices, and

assigns a priority processing order to the independently processable segments; and

a scheduling engine module that determines processing requirements for each independently processable segment and schedules the independently processable segments for delivery to the allocated media processing unit of the media device and the one or more media processing units of the other media devices based upon the assigned priority processing order and the processing requirements for each independently processable segment; and

a processing unit for executing the computer-executable program modules.

17. The system of claim 16, wherein the slicing engine further includes data within each independently processable segment indicating an original media format and bit-rate of the respective media stream from which each independently processable segment is generated.

18. The system of claim 16, wherein the scheduling engine module further dispatches the independently processable segments to the allocated media processing unit of the media device and the one or more media processing units of the other media devices.

19. The system of claim 18, wherein the computer-executable program modules further comprise:

a routing engine module that routes a processed version of the second media stream to a presentation device, the processed version of the second media stream being assembled from processed segments corresponding to the second media stream returned from the allocated media processing unit of the media device and the one or more media processing units of the other media devices.

20. The system of claim 19, wherein the routing engine module routes a processed version of the first media stream to a storage device, the processed version of the first media stream being assembled from processed segments corresponding to the first media stream returned from the allocated media processing unit of the media device.