Adaptive variable fidelity media distribution system and method

• US 7,953,882 B2
• Filed: 07/28/2008
• Issued: 05/31/2011
• Est. Priority Date: 07/26/2007
• Status: Active Grant

• Current Assignee: Intel Corporation
• Original Assignee: RealNetworks, Inc.
• Inventors: Colwell, Aaron James, Shukla, Amol
• Primary Examiner(s): Eng; David Y
• Application Number:
  US12/181,310
• Publication Number:
  US 20090031038A1
• Time in Patent Office: 1,037 Days
• Field of Search: 709/231
• US Class Current: 709/231
• CPC Class Codes: H04L 65/4015 where at least one of the additional parallel sessions is real time or time sensitive, e.g. white board sharing, collaboration or spawning of a subconference
• CPC Class Codes: H04L 65/60 Network streaming of media packets
• CPC Class Codes: H04L 65/70 Media network packetisation
• CPC Class Codes: H04L 65/764 at the destination reformatting of additional data in video clients H04N21/4355
• CPC Class Codes: H04L 65/80 Responding to QoS
• CPC Class Codes: H04L 67/104 Peer-to-peer [P2P] networks
• CPC Class Codes: H04L 67/1091 Interfacing with client-server systems or between P2P systems

Abstract

A layered media stream comprising media information layers of different priorities may be adaptively distributed by transmitting at least one of the media information layers to a client on a time-windowed basis. Within each time window, the transmission of the media information layers is attempted in priority order and lower priority layers are transmitted as network conditions allow.

Claims

1. A computer-implemented method of adaptively distributing via a network a layered media stream, the method comprising:

hosting, by a hosting device, an encoded media content stream comprising a plurality of media information layers of different priorities, including at least one highest-priority layer, wherein;

said plurality of layers comprise non-redundant media information;

each of at least one lower-priority layers can be decoded into a media content stream in combination with at least one higher-priority layer; and

as more layers are combined, the decoded media content stream improves with respect to at least one of spatial resolution, signal-to-noise-ratio, encoding method, or bit-depth; and

adaptively transmitting, by said hosting device to a client via a network on a time-windowed basis, at least one of said plurality of media information layers, wherein within each time window, said transmission of the plurality of media information layers is attempted in priority order and lower priority layers are transmitted in accordance with a network condition determination.

2. The method of claim 1, wherein said adaptively transmitting is without reference to a network bandwidth estimation.

3. The method of claim 1, wherein said adaptively transmitting further comprises making a network congestion determination in accordance with at least one of a packet loss rate and a packet round-trip time.

4. The method of claim 3, wherein said network uses the Transmission Control Protocol (TCP) and said congestion determination utilizes at least one of slow-start algorithm, congestion avoidance algorithm, fast retransmit algorithm, and fast recovery algorithm.

5. The method of claim 3, wherein said network uses the Real-time Transport Protocol (RTP) and said congestion determination utilizes at least the TCP-Friendly Rate Control (TFRC) congestion control mechanism.

6. The method of claim 1, wherein determining said network condition comprises receiving at least one of a transport protocol acknowledgement message or an application layer acknowledgement message.

7. The method of claim 1, wherein determining said network condition comprises receiving network information from an intermediary.

8. The method of claim 1, further comprising adjusting the duration of said time windows.

9. A computer readable medium having stored thereon instructions that, when executed by a processor, perform a method of adaptively distributing via a network a layered media stream, the method comprising:

hosting an encoded media content stream comprising a plurality of media information layers of different priorities, including at least one highest-priority layer, wherein;

said plurality of layers comprise non-redundant media information;

each of at least one lower-priority layers is capable of being decoded into a media content stream in combination with at least one higher-priority layer; and

as more layers are combined, the decoded media content stream improves with respect to at least one of spatial resolution, signal-to-noise-ratio, encoding method, or bit-depth;

adaptively transmitting to a client via a network on a time-windowed basis at least one of said plurality of media information layers, wherein within each time window, said transmission of the plurality of media information layers is attempted in priority order and lower priority layers are transmitted in accordance with a network condition determination.

10. A computer apparatus having a processor and memory containing computer executable instructions that, when executed by the processor, perform a method of adaptively distributing via a network a layered media stream, the method comprising:

hosting an encoded media content stream comprising a plurality of media information layers of different priorities, including at least one highest-priority layer, wherein;

said plurality of layers comprise non-redundant media information;

each of at least one lower-priority layers is capable of being decoded into a media content stream in combination with at least one higher-priority layer; and

as more layers are combined, the decoded media content stream improves with respect to at least one of spatial resolution, signal-to-noise-ratio, encoding method, or bit-depth;

adaptively transmitting to a client via a network on a time-windowed basis at least one of said plurality of media information layers, wherein within each time window, said transmission of the plurality of media information layers is attempted in priority order and lower priority layers are transmitted in accordance with a network condition determination.

11. A computer-implemented method of adaptively distributing to a client via a network a layered media stream the method comprising:

hosting, by a hosting device, an encoded media content stream comprising a plurality of media information layers of different priorities, including at least one highest-priority layer, wherein;

said plurality of layers comprise non-redundant media information;

each of at least one lower-priority layers can be decoded into a media content stream in combination with at least one higher-priority layer; and

the decoded media content stream improves as more layers are combined;

transmitting, by said hosting device, a first time window of a first set of said plurality of media information layers, said first set comprising all media information layers having a priority above a first determined threshold;

determining, by said hosting device, whether said first time window was likely timely received; and

when said first time window is not likely timely received, transmitting, by said hosting device, a second time window of a second set of said plurality of media information layers, said second set comprising fewer media information layers than said first set.

12. The method of claim 11, further comprising, when said first time window is timely received, transmitting a second time window of a third set of said plurality of media information layers, said third set comprising more media information layers than said first set.

13. The method of claim 12, further comprising waiting until the end of said first time window before transmitting said second time window.

14. The method of claim 11, wherein said determining whether said first chunk of media information was timely received comprises a selected one of receiving a timely acknowledgement message from the client or failing to receive a timely acknowledgement message from the client.

15. The method of claim 11, further comprising adjusting the duration of said time windows.

16. A computer readable medium having stored thereon instructions that, when executed by a processor, perform a method of adaptively distributing to a client via a network a layered media stream, the method comprising:

hosting an encoded media content stream comprising a plurality of media information layers of different priorities, including at least one highest-priority layer, wherein;

said plurality of layers comprise non-redundant media information;

each of at least one lower-priority layers is capable of being decoded into a media content stream in combination with at least one higher-priority layer; and

the decoded media content stream improves as more layers are combined;

transmitting a first time window of a first set of said plurality of media information layers, said first set comprising all media information layers having a priority above a first determined threshold;

determining whether said first time window was likely timely received; and

when said first time window is not likely timely received, transmitting a second time window of a second set of said plurality of media information layers, said second set comprising fewer media information layers than said first set.

17. A computer apparatus having a processor and memory containing computer executable instructions that, when executed by the processor, perform the executed by a processor, perform a method of adaptively distributing to a client via a network a layered media stream, the method comprising:

hosting an encoded media content stream comprising a plurality of media information layers of different priorities, including at least one highest-priority layer, wherein;

said plurality of layers comprise non-redundant media information;

each of at least one lower-priority layers is capable of being decoded into a media content stream in combination with at least one higher-priority layer; and

the decoded media content stream improves as more layers are combined;

transmitting a first time window of a first set of said plurality of media information layers, said first set comprising all media information layers having a priority above a first determined threshold;

determining whether said first time window was likely timely received; and

when said first time window is not likely timely received, transmitting a second time window of a second set of said plurality of media information layers, said second set comprising fewer media information layers than said first set.

18. A computer-implemented method of distributing to a client via a network a layered media stream that adapts to changing client terminal conditions, the method comprising:

hosting, by a hosting device, an encoded media content stream comprising a plurality of media information layers of different priorities, including at least one highest-priority layer, wherein;

said plurality of layers comprise non-redundant media information;

each of at least one lower-priority layers can be decoded into a media content stream in combination with at least one higher-priority layer; and

the decoded media content stream improves as more layers are combined;

transmitting, by said hosting device, a first time window of a first set of said plurality of media information layers in accordance with a client terminal condition; and

transmitting, by said hosting device, a second time window of a second set of said plurality of media information layers in accordance with a change in said client terminal condition.

19. The method of claim 18, wherein said client terminal condition comprises at least one of a rendering resolution, a rendering size, a media rendering capability, a processor load, a power consumption, and a memory capacity.

20. The method of claim 19, wherein said change comprises an increase in said rendering resolution and said second set comprises layers that can be decoded to a higher resolution than said first set.

21. The method of claim 19, wherein said change comprises a decrease in said rendering resolution and said second set comprises layers that can be decoded to a lower resolution than said first set.

22. A computer readable medium having stored thereon instructions that, when executed by a processor, perform a method of distributing to a client via a network a layered media stream that adapts to changing client terminal conditions, the method comprising:

hosting an encoded media content stream comprising a plurality of media information layers of different priorities, including at least one highest-priority layer, wherein;

said plurality of layers comprise non-redundant media information;

each of at least one lower-priority layers can be decoded into a media content stream in combination with at least one higher-priority layer; and

the decoded media content stream improves as more layers are combined;

transmitting a first time window of a first set of said plurality of media information layers in accordance with a client terminal condition; and

transmitting a second time window of a second set of said plurality of media information layers in accordance with a change in said client terminal condition.

23. A computer apparatus having a processor and memory containing computer executable instructions that, when executed by the processor, perform the a method of distributing to a client via a network a layered media stream that adapts to changing client terminal conditions, the method comprising:

hosting an encoded media content stream comprising a plurality of media information layers of different priorities, including at least one highest-priority layer, wherein;

said plurality of layers comprise non-redundant media information;

each of at least one lower-priority layers can be decoded into a media content stream in combination with at least one higher-priority layer; and

the decoded media content stream improves as more layers are combined;

transmitting a first time window of a first set of said plurality of media information layers in accordance with a client terminal condition; and

transmitting a second time window of a second set of said plurality of media information layers in accordance with a change in said client terminal condition.