Termination of fragment delivery services from data centers participating in distributed streaming operations

• US 20100094971A1
• Filed: 10/15/2009
• Published: 04/15/2010
• Est. Priority Date: 10/15/2008
• Status: Abandoned Application

• Current Assignee: Dzsolar Ltd
• Original Assignee: Patentvc Limited
• Inventors: Thieberger, Gil, Zuckerman, Gal
• Application Number:
  US12/579,904
• Publication Number:
  US 20100094971A1
• Time in Patent Office: Days
• US Class Current: 709/219
• CPC Class Codes: G06F 16/1834 implemented based on peer-to-peer networks, e.g. gnutella p2p communication protocols H04L67/104
• CPC Class Codes: G06F 16/70 of video data
• CPC Class Codes: H03M 13/37 Decoding methods or techniques, not specific to the particular type of coding provided for in groups H03M13/03 - H03M13/35
• CPC Class Codes: H03M 13/3761 using code combining, i.e. using combining of codeword portions which may have been transmitted separately, e.g. Digital Fountain codes, Raptor codes or Luby Transform [LT] codes
• CPC Class Codes: H04L 67/1001 for accessing one among a plurality of replicated servers
• CPC Class Codes: H04L 67/1008 based on parameters of servers, e.g. available memory or workload monitoring of computer activity G06F11/30
• CPC Class Codes: H04L 67/101 based on network conditions
• CPC Class Codes: H04L 67/1012 based on compliance of requirements or conditions with available server resources
• CPC Class Codes: H04L 67/1021 based on client or server locations
• CPC Class Codes: H04L 67/1023 based on a hash applied to IP addresses or costs
• CPC Class Codes: H04L 67/1091 Interfacing with client-server systems or between P2P systems
• CPC Class Codes: H04L 67/1097 for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]

Abstract

Termination of fragment delivery services from data centers participating in distributed streaming operations, including the steps of monitoring a large-scale streaming system including a plurality of data centers comprising together a large quantity of fractional-storage CDN servers storing erasure-coded fragments encoded with a redundancy factor R greater than one and associated with approximately sequential segments of streaming contents. Identifying a data center that is still capable of delivering a substantial fragment throughput but is underperforming compared to the other centers, wherein the servers of the center store less than (1−1/R) of the fragments associated with each segment to be reconstructed. And terminating the fragment delivery service of the identified data center while maintaining continuous streaming of the contents by diverting traffic from the terminated center to the remaining centers.

Claims

1. ) A large scale streaming system, comprising:

a plurality of data centers, spread across a wide geographical area, comprising together >

100 fractional-storage CDN servers configured to store erasure-coded fragments encoded with a redundancy factor R and associated with approximately sequential segments of streaming contents; and

>

10,000 assembling devices configured to obtain the fragments for streaming the contents;

wherein no data center stores more than (1−

1/R) of the fragments associated with a single segment to be reconstructed, and wherein upon termination of a fragment delivery service from approximately any one of the data centers, the affected assembling devices continue streaming the contents by obtaining fragments from servers still in service.

2. ) The large scale streaming system of claim 1, wherein, just before the termination of the service, the alternative servers'"'"' aggregated unutilized bandwidth for fragment delivery is larger than the fragment delivery throughput provided by the service to be terminated.

3. ) The large scale streaming system of claim 2, wherein the erasure-coding is rateless-coding enabling a high redundancy factor;

the system comprises;

more than 100 data centers spread across a wide geographical area, more than 1,000 servers storing fragments with a high storage gain, and more than 1,000,000 assembling devices located at user premises and configured to obtain the fragments from the servers via the Internet.

4. ) The large scale streaming system of claim 2, wherein the erasure-coding is rateless-coding, the system comprises more than 1,000 data centers, and the average data center includes less than two fractional-storage CDN servers.

5. ) The large scale streaming system of claim 2, wherein just before the termination of the service, the terminated service is still capable of delivering a substantial fragment throughput but is underperforming compared to the other data centers.

6. ) The large scale streaming system of claim 5, wherein the performance is measured by one or more of the following:

fragment delivery throughput, latency in responding to fragment requests, variance in the latency in responding to fragment requests, service outage, and fragment loss.

7. ) The large scale streaming system of claim 2, wherein different data centers feature significantly different characteristics comprising one or more of the following:

available storage space, available fragment delivery throughput, latency in responding to fragment requests, and the specific network to which the data center is directly connected.

8. ) The large scale streaming system of claim 2, wherein different data centers feature significantly different characteristics comprising one or more of the following:

data center ownership, operation cost, data center outage, data center reliability, and utilization profile of shared lines.

9. ) The large scale streaming system of claim 2, wherein the assembling devices are configured to use a fragment pull protocol to obtain the fragments from the servers.

10. ) The large scale streaming system of claim 2, wherein the assembling devices are configured to use sub-transmissions to obtain the fragments from the servers, the erasure-coded fragments support source-selection diversity, and further comprising a managing server configured to select the servers to deliver the sub-transmissions.

11. ) A method for replacing data centers, comprising:

monitoring a large scale streaming system comprising a plurality of data centers comprising together hundreds of fractional-storage CDN servers storing erasure-coded fragments encoded with a redundancy factor R and associated with approximately sequential segments of streaming contents;

identifying a data center that is still capable of delivering a substantial fragment throughput but is underperforming compared to the other centers;

wherein the servers of the center store less than (1−

1/R) of the fragments associated with each segment to be reconstructed; and

terminating the fragment delivery service of the identified data center while maintaining continuous streaming of the contents by diverting traffic from the terminated center to the remaining centers.

12. ) The method of claim 11, further comprising the step of adding a new data center storing unique fragments after the step of terminating the fragment delivery service, wherein, just before terminating the service, the aggregated unutilized bandwidth for fragment delivery of the remaining data centers is larger than the fragment delivery throughput supplied by the service to be terminated.

13. ) The method of claim 11, further comprising the step of adding a new data center storing unique fragments before the step of terminating the fragment delivery service, wherein the aggregated unutilized fragment delivery bandwidth of the new data center and the remaining data centers is larger than the fragment delivery throughput supplied by the service to be terminated.

14. ) A large scale streaming system, comprising:

a plurality of data centers, spread across a wide geographical area, comprising together >

200 fractional-storage CDN servers configured to store unique rateless-coded fragments associated with approximately sequential segments of streaming contents; and

>

10,000 assembling devices configured to obtain the fragments for streaming operations;

wherein the system occasionally adds a data center to the streaming operations, and occasionally excludes a data center from the streaming operations while maintaining continuous streaming;

wherein the added data center stores unique rateless-coded fragments associated with the segments.

15. ) The large scale streaming system of claim 14, wherein data centers that underperform compared to the other data centers are candidates for exclusion, and wherein just before the exclusion, the excluded data center is still capable of delivering a substantial fragment throughput.

16. ) The large scale streaming system of claim 15, wherein the performance is measured by one or more of the following:

fragment delivery throughput, latency in responding to fragment requests, variance in the latency in responding to fragment requests, service outage, and fragment loss.

17. ) The large scale streaming system of claim 14, wherein most of the excluded data centers provide fragments to approximately the smallest number of assembling devices over a predefined period before their exclusion.

18. ) The large scale streaming system of claim 14, wherein using a rateless code practically enabling an infinite number of unique fragments, thus allowing many additions and exclusions of data centers.

19. ) The large scale streaming system of claim 18, wherein the fragments stored on servers of the excluded data center are usually not stored again on servers of the remaining or to be added data centers.

20. ) The large scale streaming system of claim 14, wherein over a long period there are more additions of data centers than exclusions, and therefore the streaming and storage capacity of the streaming system increases.