Adaptation of data centers' bandwidth contribution to distributed streaming operations

• US 20100094975A1
• Filed: 10/15/2009
• Published: 04/15/2010
• Est. Priority Date: 10/15/2008
• Status: Active Grant

• Current Assignee: Xenogenic Development Limited Liability Company (Intellectual Ventures LLC)
• Original Assignee: Patentvc Limited
• Inventors: Thieberger, Gil, Zuckerman, Gal
• Granted Patent: US 7,822,869 B2
• Time in Patent Office: Days
• US Class Current: 709/219
• CPC Class Codes: G06F 16/1834 implemented based on peer-to-peer networks, e.g. gnutella p2p communication protocols H04L67/104
• CPC Class Codes: G06F 16/70 of video data
• CPC Class Codes: H03M 13/37 Decoding methods or techniques, not specific to the particular type of coding provided for in groups H03M13/03 - H03M13/35
• CPC Class Codes: H03M 13/3761 using code combining, i.e. using combining of codeword portions which may have been transmitted separately, e.g. Digital Fountain codes, Raptor codes or Luby Transform [LT] codes
• CPC Class Codes: H04L 67/1001 for accessing one among a plurality of replicated servers
• CPC Class Codes: H04L 67/1008 based on parameters of servers, e.g. available memory or workload monitoring of computer activity G06F11/30
• CPC Class Codes: H04L 67/101 based on network conditions
• CPC Class Codes: H04L 67/1012 based on compliance of requirements or conditions with available server resources
• CPC Class Codes: H04L 67/1021 based on client or server locations
• CPC Class Codes: H04L 67/1023 based on a hash applied to IP addresses or costs
• CPC Class Codes: H04L 67/1091 Interfacing with client-server systems or between P2P systems
• CPC Class Codes: H04L 67/1097 for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]

Abstract

Adaptation of data centers'"'"' bandwidth contribution to distributed streaming operations, including data centers comprising fractional-storage CDN servers storing erasure-coded fragments encoded with a redundancy factor R greater than one, assembling devices obtaining the fragments from subsets of the servers, and measuring fragment delivery parameters, and at least one decision component that occasionally changes at least some of the servers of the subsets to generally improve the measured parameters. Wherein the smaller the number of subsets in which the servers of a data center participate, the lower the center'"'"'s fragment delivery throughput, the higher the center'"'"'s cost of delivering a fragment, and the higher the likelihood of reducing the amount of bandwidth acquired from that data center by the operator of the system.

Claims

1. ) A large scale system, comprising:

data centers comprising fractional-storage CDN servers configured to store erasure-coded fragments;

assembling devices configured to obtain the fragments from subsets of the servers, and to measure fragment delivery parameters; and

at least one decision component configured to occasionally change at least some of the servers of the subsets to generally improve the measured parameters;

wherein the smaller the number of subsets in which the servers of a data center participate, the lower the center'"'"'s fragment delivery throughput, and the higher the likelihood of reducing the amount of bandwidth acquired from that data center by the operator of the system.

2. ) The large scale system of claim 1, wherein the measured fragment delivery parameters comprise at least one of the following:

latency in responding to data requests, variance in latency in responding to fragment requests, fragment loss, and service outage.

3. ) The large scale system of claim 2, wherein the assembling devices are configured to obtain the fragments using a fragment pull protocol that is used to estimate at least one of the parameters.

4. ) The large scale system of claim 1, wherein each assembling device comprises its decision component that is configured to occasionally change the servers in the assembling device'"'"'s subset.

5. ) The large scale system of claim 1, further comprising a control server comprising the decision component that is configured to occasionally change at least some of the servers of the subsets;

wherein the control server is configured to receive at least some of the measured fragment delivery parameters from the assembling device.

6. ) The large scale system of claim 1, wherein the erasure-coding is rateless-coding enabling a high redundancy factor;

the system comprises;

more than 100 data centers spread across a wide geographical area, more than 1,000 servers storing fragments with a high storage gain, and more than 1,000,000 assembling devices located at user premises and configured to obtain the fragments from the servers via the Internet.

7. ) The large scale system of claim 1, wherein the fragments are associated with approximately sequential segments of streaming contents, and the assembling devices are configured to obtain the fragments at rates approximately equal to or higher than the rates of presenting the streaming contents.

8. ) The large scale system of claim 7, wherein the larger the number of subsets in which the servers of a data center participate, the higher the center'"'"'s fragment delivery throughput, the lower the center'"'"'s cost of delivering a fragment, and the higher the likelihood of acquiring additional bandwidth from that data center.

9. ) The large scale system of claim 7, wherein the lower the data center'"'"'s fragment delivery throughput, the higher the center'"'"'s cost of delivering a fragment;

and over a long period of time, most of the data centers having the highest costs of delivering a fragment compared to the other data centers, even if still capable of providing a substantial fragment delivery throughput, are excluded from the system.

10. ) The large scale system of claim 7, wherein the erasure-coding is rateless-coding enabling a high redundancy factor, and the measured fragment delivery parameters comprise reported load level encountered by the servers when delivering the fragments.

11. ) A large scale system, comprising:

data centers comprising fractional-storage CDN servers configured to store erasure-coded fragments; and

assembling devices configured to obtain the fragments from subsets of the servers, and to measure fragment delivery parameters;

wherein the servers of the subsets are occasionally reselected to generally improve the measured parameters, and the fragment delivery throughput from a data center is determined mainly by the number of subsets in which the servers of that data center participate;

wherein the fragment delivery capacity of data centers whose fragment delivery throughput approaches their capacity, is usually enhanced until the throughput does not regularly approach the capacity.

12. ) The large scale system of claim 11, wherein the measured fragment delivery parameters comprise at least one of the following:

latency in responding to data requests, variance in latency in responding to fragment requests, fragment loss, and service outage.

13. ) The large scale system of claim 11, wherein at least one of the data centers is connected to the Internet via a high bandwidth fixed line having a certain fragment delivery capacity, and when the fragment delivery throughput from that center approaches the capacity of the fixed line, the capacity is upgraded.

14. ) The large scale system of claim 11, wherein at least one of the centers provides Internet bandwidth services to multiple applications via a shared communication line, the fragment delivery throughput in that center is limited to a capacity smaller than the bandwidth capacity of the shared line, and when the fragment delivery throughput from the center approaches the limited capacity, the limited capacity is enhanced;

whereby the capacity is optionally enhanced by upgrading the service agreement with the data center.

15. ) The large scale system of claim 11, wherein the erasure-coding is rateless-coding enabling a high redundancy factor;

the system comprises;

more than 100 data centers spread across a wide geographical area, more than 1,000 servers storing fragments with a high storage gain, and more than 1,000,000 assembling devices located at user premises and configured to obtain the fragments from the servers via the Internet.

16. ) The large scale system of claim 11, wherein the fragments are associated with approximately sequential segments of streaming contents, and the assembling devices are configured to obtain the fragments at rates approximately equal or higher than the rates of presenting the streaming contents.

17. ) The large scale system of claim 16, wherein the quality of the fragment delivery from at least one of the data centers is monitored, and when the fragment delivery throughput approaches a level that lowers the quality, the bandwidth capacity of the center is enhanced in order to improve the quality.

18. ) The large scale system of claim 16, wherein when the fragment delivery throughput of a data center approaches its fragment delivery capacity, at least one other center located nearby is added to the system.

19. ) The large scale system of claim 16, wherein when the fragment delivery throughput of a data center significantly decreases below its fragment delivery capacity, the capacity is reduced, potentially to the point that the center is excluded from the system.

20. ) A large scale system, comprising:

data centers comprising fractional-storage CDN servers configured to store erasure-coded fragments; and

assembling devices configured to obtain the fragments from subsets of the servers, and to measure fragment delivery parameters;

wherein the servers of the subsets are occasionally reselected to generally improve the measured parameters;

wherein, over a long period of time, the system features a reduction in the fragment delivery capacity of centers having a ratio of actual fragment delivery throughput to fragment delivery capacity that is significantly lower than the average ratio of the other centers, potentially up to point where the centers having a lower ratio are practically excluded from the system.