CENTRALIZED NETWORKING CONFIGURATION IN DISTRIBUTED SYSTEMS

• US 20150127783A1
• Filed: 11/04/2013
• Published: 05/07/2015
• Est. Priority Date: 11/04/2013
• Status: Active Grant

• Current Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Original Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Inventors: LISSACK, AVICHAI MENDLE
• Granted Patent: US 10,002,011 B2
• Time in Patent Office: Days
• US Class Current: 709/220
• CPC Class Codes: G06F 2009/45595 Network integration; Enabling network access in virtual machine instances
• CPC Class Codes: G06F 9/45533 Hypervisors; Virtual machine monitors
• CPC Class Codes: H04L 41/0893 Assignment of logical groups to network elements
• CPC Class Codes: H04L 41/0894 Policy-based network configuration management
• CPC Class Codes: H04L 41/0895 Configuration of virtualised networks or elements, e.g. virtualised network function or OpenFlow elements
• CPC Class Codes: H04L 41/5022 by giving priorities, e.g. assigning classes of service
• CPC Class Codes: H04L 41/5054 Automatic deployment of services triggered by the service manager, e.g. service implementation by automatic configuration of network components

Abstract

Methods and apparatus for centralized networking configuration in distributed systems are disclosed. Networking related metrics from a plurality of sources within a distributed system are obtained at a networking configuration server. A set of rules to be used to apply a network configuration option to a particular category of traffic associated with a node of the distributed system is determined based on the collected metrics and on networking management policies. A representation of the set of rules is transmitted to the node of the distributed system to schedule network transmissions in accordance with the networking configuration option.

Claims

1. A system, comprising:

a plurality of computing devices configured to;

obtain, at one or more centralized servers of a networking configuration service, metrics from a plurality of sources, including traffic metrics collected at a set of nodes configurable to implement a network-accessible service of a provider network;

determine, at the networking configuration service, based at least in part on the metrics and at least in part on a networking management policy, metadata to be used to apply a networking configuration option to a particular category of traffic associated with a service instance of the network-accessible service at a particular node of the set of nodes;

transmit, to a control module at a particular node of the set of nodes, a representation of the metadata; and

implement, at the particular node, a procedure indicated by the metadata determined at the networking configuration service to schedule one or more network transmissions.

2. The system as recited in claim 1, wherein the networking configuration option comprises one or more of:

(a) a bandwidth limit, (b) a latency constraint, (c) a quality-of-service goal, (d) a packet fragmentation configuration setting, or (e) a configuration setting dependent at least in part on packet size.

3. The system as recited in claim 1, wherein the plurality of sources include one or more of:

(a) a denial-of-service detector, (b) a per-network-address traffic metric aggregation service, or (c) a per-client traffic metric aggregation service.

4. The system as recited in claim 1, wherein the representation of the metadata is transmitted based at least in part on one of:

(a) a pre-determined schedule, (b) an occurrence of a triggering event, or (c) a request from the particular node.

5. The system as recited in claim 1, wherein the metadata determined at the networking configuration server comprises one or more rules to be used to classify a unit of network traffic into the particular category based at least in part on one of:

(a) a network endpoint address associated with the unit of network traffic, (b) a client on whose behalf the unit of network traffic is generated, or (c) an application on behalf of which the unit of network traffic is generated.

6. The system as recited in claim 1, wherein the plurality of computing devices are further configured to:

receive an indication of a configuration request submitted by one of;

(a) a client of the network-accessible service or (b) an administrator of the network-accessible service; and

modify one or more elements of the metadata in accordance with the configuration request.

7. A method, comprising:

performing, by a plurality of computing devices;

obtaining, at a networking configuration server, metrics from a plurality of sources associated with a network-accessible service of a provider network;

determining, at the networking configuration server, based at least in part on the metrics, a procedure to be used to apply a networking configuration option to a particular category of traffic associated with a service instance of the network-accessible service;

transmitting, from the networking configuration server to a particular node of the provider network associated with the service instance, a representation of the procedure; and

implementing, at the particular node, the procedure determined at the networking configuration server to schedule one or more network transmissions in accordance with the networking configuration option.

8. The method as recited in claim 7, wherein the networking configuration option comprises one or more of:

(a) a bandwidth limit, (b) a latency constraint, (c) a quality-of-service goal, (d) a packet fragmentation configuration setting, or (e) a configuration setting dependent at least in part on packet size.

9. The method as recited in claim 7, wherein the plurality of sources include one or more of:

(a) the particular node, (b) a denial-of-service detector, (c) a per-network-address traffic aggregation service, or (d) a per-client traffic aggregation service.

10. The method as recited in claim 7, wherein said transmitting the representation is initiated based at least in part on one of:

(a) a pre-determined schedule, (b) an occurrence of a triggering event, or (c) a request from the particular node.

11. The method as recited in claim 7, wherein the procedure determined at the networking configuration server comprises one or more rules to be used to classify a unit of network traffic into the particular category based at least in part on one of:

(a) a network endpoint address associated with the unit of network traffic, (b) a client on whose behalf the unit of network traffic is generated, or (c) an application on behalf of which the unit of network traffic is generated.

12. The method as recited in claim 7, wherein the provider network comprises a plurality of availability containers with respective failure profiles, further comprising performing, by the plurality of computing devices:

configuring at least one networking configuration server at each availability container of the plurality of availability containers.

13. The method as recited in claim 7, further comprising performing, by the plurality of computing devices:

determining a number of networking configuration servers to be instantiated based at least in part on a performance requirement associated with a time interval between a detection of an event that leads to a change to a networking configuration option, and an implementation of a modified procedure to apply the change to the networking configuration option.

14. The method as recited in claim 7, wherein said determining the procedure is performed in response to a request from one of:

(a) a client of the network-accessible service or (b) an administrator of the network-accessible service.

15. The method as recited in claim 7, wherein the particular node to which the representation of the procedure is transmitted comprises one of:

(a) an instance host configurable to implement a service unit the network-accessible service, (b) a switch, (c) a router, (d) a gateway (e) a load-balancer, or (f) a storage device.

16. The method as recited in claim 7, further comprising performing, by the plurality of computing devices:

providing, via one or more programmatic interfaces, a consolidated view of a networking status of a plurality of nodes configurable to support the network-accessible service.

17. The method as recited in claim 7, further comprising performing, by the plurality of computing devices:

providing, from the networking configuration server to an instance placement component associated with the network-accessible service, an indication of networking status information of a plurality of instance hosts of the network-accessible service, wherein the instance placement component is configured to utilize the indication to select a particular instance host at which a service instance of the network-accessible service is to be launched.

18. A non-transitory computer-accessible storage medium storing program instructions that when executed on one or more processors:

obtain networking-related metrics from a plurality of sources;

determine, in accordance with one or more networking management policies and based at least in part on the networking-related metrics, a set of rules to be used to apply a networking configuration option to a particular category of traffic associated with a computing device; and

transmit, to the computing device from a networking configuration server, a representation of the set of rules to be used to schedule one or more network transmissions in accordance with the networking configuration option.

19. The non-transitory computer-accessible storage medium as recited in claim 18, wherein the networking configuration option comprises one or more of:

(a) a bandwidth limit, (b) a latency constraint, (c) a quality-of-service goal, (d) a packet fragmentation configuration setting, or (e) a configuration setting dependent at least in part on packet size.

20. The non-transitory computer-accessible storage medium as recited in claim 18, wherein the plurality of sources include one or more of:

(a) the computing device, (b) a denial-of-service detector, (c) a per-network-address traffic aggregation service, or (d) a per-client traffic aggregation service.

21. The non-transitory computer-accessible storage medium as recited in claim 18, wherein the instructions when executed on the one or more processors:

determine, in accordance with the one or more networking management policies and based at least in part on the networking-related metrics, a different set of rules to be used to apply one or more networking configuration options to respective categories of traffic at a different computing device; and

transmit, to the different computing device, a representation of the different set of rules.

22. The non-transitory computer-accessible storage medium as recited in claim 18, wherein the set of rules comprises one or more rules to be used to classify a unit of network traffic into the particular category based at least in part on one of:

(a) a network endpoint address associated with the unit of network traffic, (b) a client on whose behalf the unit of network traffic is generated, or (c) an application on behalf of which the unit of network traffic is generated.

23. The non-transitory computer-accessible storage medium as recited in claim 18, wherein the computing device is configured to implement a service instance of a network-accessible service comprising one of:

(a) a virtualized computing service, (b) a storage service, or (c) a database service.