FORWARD-BASED RESOURCE DELIVERY NETWORK MANAGEMENT TECHNIQUES

• US 20150249579A1
• Filed: 03/10/2015
• Published: 09/03/2015
• Est. Priority Date: 10/02/2009
• Status: Active Grant

• Current Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Original Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Inventors: Ellsworth, Joseph L., Porter, Brandon W., Suver, Christopher Allen, Newcombe, Christopher R.
• Granted Patent: US 9,246,776 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 16/13 File access structures, e.g. distributed indices arrangements of input from, or output to, record carriers G06F3/06
• CPC Class Codes: H04L 41/0823 characterised by the purposes of a change of settings, e.g. optimising configuration for enhancing reliability for optimising operational conditions of wireless networks H04W24/02
• CPC Class Codes: H04L 41/5025 by proactively reacting to service quality change, e.g. by reconfiguration after service quality degradation or upgrade
• CPC Class Codes: H04L 47/70 Admission control; Resource allocation
• CPC Class Codes: H04L 47/782 Hierarchical allocation of resources, e.g. involving a hierarchy of local and centralised entities
• CPC Class Codes: H04L 47/83 based on usage prediction
• CPC Class Codes: H04L 67/125 involving control of end-device applications over a network
• CPC Class Codes: H04N 21/2225 Local VOD servers
• CPC Class Codes: H04N 21/23103 using load balancing strategies, e.g. by placing or distributing content on different disks, different memories or different servers storage management G06F3/0604; allocation of resources considering the load in multiprogramming arrangements G06F9/505; techniques for rebalancing the load in a distributed system G06F9/5083; access to distributed or replicated servers, e.g. load balancing, in data networks H04L67/1001

Abstract

A resource delivery network and method for distributing content in the network is disclosed herein. The network comprises a plurality of servers arranged in tiers and partitioned. Each server includes a resource store with a set of resources for distribution to a successive tier. Updates to each successive tier are provided by a pull-forward client on servers in the tier. This forward propagation mechanism maximizes resource availability at edge servers in the network. Resources transmitted to the edge tier servers may be transformed, combined, and rendered without taxing lower tier servers. Transformation and pre-rendering of data can be performed by low priority CPU tasks at each layer of the system.

Claims

1. A computer-implemented method for managing a resource delivery network including a plurality of tiers, comprising:

under control of one or more computer systems configured with executable instructions,

reading, by a first server associated with a first data store, first write data of a second server in a tier of the resource delivery network, the first write data identifying one or more writes to a second data store associated with the second server;

obtaining, by the first server, one or more first resources identified in the first write data;

storing the one or more first resources to the first data store;

comparing the one or more first resources to one or more second resources of a third server in a preceding tier of the resource delivery network;

determining that at least one of the second resources is not included among the one or more first resources;

obtaining, by the first server, the at least one of the second resources;

storing the at least one of the second resources to the first data store; and

adding the first server to the tier.

2. The computer-implemented method of claim 1, wherein:

the resource delivery network is organized according to a plurality of partitions, a partition including a subset of a plurality of servers in a tier of the plurality of tiers; and

at least one partition is dedicated to distribution of a subset of the resources of an authoritative data store, the subset of the resources corresponding to at least one of a specified size, a specified data type, or a specified set of users.

3. The computer-implemented method of claim 2, wherein the one or more first resources correspond to the at least one of the specified size, the specified data type, or the specified set of users.

4. The computer-implemented method of claim 1, wherein a respective number of servers of each tier of the plurality of tiers is determined based at least in part upon maintaining maximum write performance of a plurality of servers of a preceding tier at a specified threshold when the plurality of servers of the tier concurrently read at a specified throttle rate from the plurality of servers of the preceding tier.

5. The computer-implemented method of claim 1, further comprising:

obtaining, by the first server, a configuration file including a mapping of the first server to a cluster and a network address for obtaining configuration data for the cluster; and

obtaining, by the first server, the configuration data using the network address.

6. The computer-implemented method of claim 5, further comprising:

filtering out at least one of the one or more resources based at least in part upon the configuration data.

7. The computer-implemented method of claim 1, wherein comparing the one or more first resources to the one or more second resources includes:

comparing the first write data to second write data of the third server, the second write data identifying one or more writes to a third data store associated with the third server.

8. The computer-implemented method of claim 7, wherein the second write data includes at least one delete operation of at least one of the second resources, and the method further comprises:

deleting, by the first server, the at least one of the second resources from the first data store.

9. A computer-implemented method for managing a resource delivery network including a plurality of partitions, comprising:

determining a connection failure between a first partition and a second partition of the resource delivery network;

electing one server of a plurality of servers of the first partition as a master for the first partition;

reconfiguring one or more other servers of the plurality of servers to route write requests to the master;

tracking the write requests to a data store associated with the master; and

propagating the write requests to the one or more other servers.

10. The computer-implemented method of claim 9, further comprising:

determining that a connection between the first partition and the second partition has been restored;

determining that one or more resources written to the data store since the connection failure are newer than corresponding resources in the second partition; and

propagating the one or more resources to the second partition.

11. The computer-implemented method of claim 10, further comprising:

determining that at least one of the resources is older than the corresponding resources in the second partition; and

retrieving the corresponding resources for the first partition.

12. The computer-implemented method of claim 10, further comprising:

reconfiguring the plurality of servers to an original configuration of the first partition.

13. A resource delivery network including a plurality of tiers, comprising:

at least one processor;

memory including instructions that, upon being executed by the at least one processor, cause the resource delivery network to;

read, by a first server associated with a first data store, first write data of a second server in a tier of the resource delivery network, the first write data identifying one or more writes to a second data store associated with the second server;

obtain, by the first server, one or more first resources identified in the first write data;

store the one or more first resources to the first data store;

compare the one or more first resources to one or more second resources of a third server in a preceding tier of the resource delivery network;

determine that at least one of the second resources is not included among the one or more first resources;

obtain, by the first server, the at least one of the second resources;

store the at least one of the second resources to the first data store; and

add the first server to the tier.

14. The resource delivery network of claim 13, wherein:

the resource delivery network is organized according to a plurality of partitions, a partition including a subset of a plurality of servers in a tier of the plurality of tiers; and

at least one partition is dedicated to distribution of a subset of the resources of an authoritative data store, the subset of the resources corresponding to at least one of a specified size, a specified data type, or a specified set of users.

15. The resource delivery network of claim 14, wherein the one or more first resources correspond to the at least one of the specified size, the specified data type, or the specified set of users.

16. The resource delivery network of claim 13, wherein a respective number of servers of each tier of the plurality of tiers is determined based at least in part upon maintaining maximum write performance of a plurality of servers of a preceding tier at a specified threshold when the plurality of servers of the tier concurrently read at a specified throttle rate from the plurality of servers of the preceding tier.

17. The resource delivery network of claim 13, wherein the instructions upon being executed further cause the resource delivery network to:

obtain, by the first server, a configuration file including a mapping of the first server to a cluster and a network address for obtaining configuration data for the cluster; and

obtain, by the first server, the configuration data using the network address.

18. The resource delivery network of claim 17, wherein the instructions upon being executed further cause the resource delivery network to:

filter out at least one of the one or more resources based at least in part upon the configuration data.

19. The resource delivery network of claim 13, wherein the instructions that cause the resource delivery network to compare the one or more first resources to the one or more resources include causing the resource delivery network to:

compare the first write data to second write data of the third server, the second write data identifying one or more writes to a third data store associated with the third server.

20. The resource delivery network of claim 19, wherein the second write data includes at least one delete operation of at least one of the second resources, and the instructions upon being executed further cause the resource delivery network to:

delete, by the first server, the at least one of the second resources from the first data store.