Load shedding techniques for distributed services with persistent client connections to ensure quality of service

• US 10,346,367 B1
• Filed: 04/30/2015
• Issued: 07/09/2019
• Est. Priority Date: 04/30/2015
• Status: Active Grant

• Current Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Original Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Inventors: Luszcz, Jacob David, Strauss, Jacob A.
• Primary Examiner(s): Rahman, Sm A
• Assistant Examiner(s): Pant, Ranjan
• Application Number:
  US14/701,462
• Time in Patent Office: 1,531 Days
• CPC Class Codes: G06F 16/182 Distributed file systems
• CPC Class Codes: G06F 16/1827 Management specifically adapted to NAS management of storage area networks [SAN] G06F3/067
• CPC Class Codes: G06F 16/183 Provision of network file services by network file servers, e.g. by using NFS, CIFS network file access protocols H04L67/1097
• CPC Class Codes: H04L 67/01 Protocols
• CPC Class Codes: H04L 67/1008 based on parameters of servers, e.g. available memory or workload monitoring of computer activity G06F11/30
• CPC Class Codes: H04L 67/1097 for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]
• CPC Class Codes: H04L 67/568 Storing data temporarily at an intermediate stage, e.g. caching

Abstract

An access node of a distributed service collects workload data pertaining to at least one peer group of access nodes established for handling client requests. During a particular load shedding analysis, the access node uses the collected metrics to detect that a triggering condition for load shedding with respect to a set of persistent client connections has been met. Each persistent client connection is set up to be usable for a plurality of client requests. The access node initiates a phased termination of at least one selected persistent client connection. The phased termination comprises allowing completion of in-flight requests on the connection and rejecting new requests on the connection.

Claims

1. A system, comprising:

a plurality of access nodes (ANs) of a file storage service implemented at a provider network, including a first AN in a first AN peer group (APG) collectively responsible for processing received client requests directed to at least a first file system instance;

wherein the first AN implements a plurality of load shedding analysis iterations (LSAIs), wherein a particular LSAI of the plurality of LSAIs comprises;

determining that a workload level associated with one or more persistent client connections (PCCs) meets a triggering condition for initiating load shedding, wherein each PCC of the one or more PCCs is established to process a plurality of client requests;

selecting at least a first PCC of the one or more PCCs as a candidate for termination;

examining a workload metric cache associated with the first APG, wherein the cache is populated based at least in part on one or more updates received from a workload information distributor, and wherein the cache comprises one or more metrics including at least a thread pool utilization metric of a different AN of the first APG;

determining, based at least in part on said examining, that one or more ANs of the APG meet an available-capacity criterion; and

initiating, based at least in part on said determining that one or more ANs of the APG meet the available-capacity criterion, a phased termination of the first PCC, wherein the phased termination comprises allowing completion of in-flight requests on the first PCC, and rejecting new requests on the first PCC before the phased termination of the first PCC is completed.

2. The system as recited in claim 1, wherein a file system protocol implemented at the file storage service comprises one of:

(a) a version of an NFS (Network File System) protocol or (b) a version of an SMB (Server Message Block) protocol.

3. The system as recited in claim 1, wherein said determining that the workload level meets the triggering condition comprises one or more of:

(a) determining that a local thread pool utilization metric of the first AN exceeds a first threshold, (b) determining that a connection-count metric indicative of a number of active PCCs exceeds a second threshold, (c) determining that an aggregate throughput metric of the one or more PCCs exceeds a third threshold, (d) determining that a utilization metric of a resource utilized by the first AN exceeds a fourth threshold, wherein the resource includes one of;

a processor, memory, a network device, a storage device, or a data structure, or (e) determining that a response time metric of the first AN exceeds a fifth threshold.

4. The system as recited in claim 1, wherein said determining that one or more ANs of the APG meet an available-capacity criterion comprises determining that at least a second AN of the first APG has one or more of:

(a) a lower local thread pool utilization metric than the first AN, (b) a lower connection-count metric than the first AN, (c) a lower aggregate throughput metric than the first AN, (d) a lower resource utilization metric of a resource than the first AN, wherein the resource includes one of;

a processor, memory, a network device, a storage device, or a data structure, or (e) a lower response time metric than the first AN.

5. The system as recited in claim 1, wherein said phased termination of the first PCC comprises performing, by the first AN:

issuing a request to close the first PCC subsequent to one or more of;

(a) an expiration of a timeout interval, or (b) a determination that an in-flight request has been fulfilled using the first PCC, wherein processing of the in-flight request had been initiated prior to beginning the phased termination.

6. A method, comprising:

performing, by a first access node of a file storage service, wherein the first access node is a member of a first group comprising one or more access nodes responsible for processing received client requests directed to at least a first file system instance, a load shedding analysis, wherein the load shedding analysis comprises;

detecting that a workload level associated with one or more client connections meets a triggering condition for initiating load shedding, wherein the client connections are established to process a plurality of client requests;

determining, based at least in part on examining a workload metric cache comprising one or more metrics associated with other access nodes of the first group, wherein the one or more metrics include a thread pool utilization metric, that the first group meets an available-capacity criterion; and

initiating, based at least in part on said determining that the first group meets an available-capacity criterion, a phased termination of at least a particular client connection of the one or more client connections, wherein the phased termination comprises allowing completion of in-flight requests on the particular client connection, and rejecting new requests on the particular client connection before the phased termination of at least the particular client connection is completed.

7. The method as recited in claim 6, wherein said determining that the first group meets an available capacity criterion comprises verifying that at least one metric stored in the workload metric cache was received within a particular time.

8. The method as recited in claim 6, wherein said detecting that the workload level meets the triggering condition comprises one or more of:

(a) determining that a local thread pool utilization metric of the first access node exceeds a first threshold, (b) determining that a connection-count metric indicative of a number of active client connections exceeds a second threshold, (c) determining that an aggregate throughput metric of the one or more client connections exceeds a third threshold, (d) determining that a utilization metric of a resource utilized by the first access node exceeds a fourth threshold, wherein the resource includes one of;

a processor, memory, a network device, a storage device, or a data structure, or (e) determining that a response time metric of the first access node exceeds a fifth threshold.

9. The method as recited in claim 6, wherein said determining that the first group meets the available-capacity criterion comprises determining that at least a second access node of the first group has one or more of:

(a) a lower local thread pool utilization metric than the first access node, (b) a lower connection-count metric than the first access node, (c) a lower aggregate throughput than the first access node, (d) a lower resource utilization metric of a resource than the first access node, wherein the resource includes one of;

a processor, memory, a network device, a storage device, or a data structure, or (e) a lower response time metric than the first access node.

10. The method as recited in claim 6, wherein said detecting that the workload level meets the triggering condition comprises utilizing a time-dependent decay function.

11. The method as recited in claim 6, wherein the first access node is a member of a second group, wherein the second group comprises one or more access nodes responsible for processing received client requests directed to at least a second file system instance, wherein at least one access node of the second group is not a member of the first group.

12. The method as recited in claim 11, wherein the first file system instance is established on behalf of a first customer of the file storage service, and wherein the second file system instance is established on behalf of a second customer of the file storage service.

13. The method as recited in claim 6, further comprising, performing by the first access node:

transmitting a heartbeat message to one or more workload information distributors designated for at least the first group, wherein the heartbeat message includes workload metrics of the first access node;

receiving a workload update message from a particular workload information distributor of the one or more workload information distributors, including a first workload update message indicating a utilization metric; and

modifying the workload cache based at least in part on the first workload update message.

14. The method as recited in claim 6, further comprising:

receiving, at a client component after said initiating the phased termination, an indication that the particular client connection has been closed;

transmitting, by the client component, to a load balancer associated with the first group, a request to establish a replacement connection for the particular client connection; and

selecting a particular one of the one or more access nodes of the first group to host the replacement connection.

15. The method as recited in claim 14, wherein the particular one of the one or more access nodes is the first access node.

16. A non-transitory computer-accessible storage medium storing program instructions that when executed on one or more processors implement a first access node of a distributed service, wherein the first access node is a member of a first group comprising one or more access nodes responsible for processing received client requests, wherein the first access node is configured to perform a load shedding analysis, wherein the load shedding analysis comprises:

collecting one or more workload metrics pertaining to one or more members of one or more groups, including the first group;

detecting, based at least in part on an analysis of the one or more workload metrics, that a triggering condition for initiating load shedding with respect to one or more client connections has been met, wherein the one or more client connections are established to process a plurality of client requests; and

initiating, based at least in part on said detecting that a triggering condition for initiating load shedding with respect to one or more client connections has been met, a phased termination of at least a particular client connection of the one or more client connections, wherein the phased termination comprises allowing completion of in-flight requests on the particular client connection, and rejecting new requests on the particular client connection before the phased termination of at least the particular client connection is completed.

17. The non-transitory computer-accessible storage medium as recited in claim 16, wherein the one or more client connections include a plurality of client connections, wherein the first access node is configured to:

select the particular client connection for phased termination based at least in part on one or more of;

(a) a comparison of performance metrics of the plurality of client connections, or (b) random selection.

18. The non-transitory computer-accessible storage medium as recited in claim 16, wherein said detecting that the triggering condition has been met comprises one or more of:

(a) determining that a local thread pool utilization metric of the first access node exceeds a first threshold, (b) determining that a connection-count metric indicative of a number of active client connections exceeds a second threshold, (c) determining that an aggregate throughput metric of the one or more client connections exceeds a third threshold, (d) determining that a utilization metric of a resource utilized by the first access node exceeds a fourth threshold, wherein the resource includes one of;

a processor, memory, a network device, a storage device, or a data structure, or (e) determining that a response time metric of the first access node exceeds a fifth threshold.

19. The non-transitory computer-accessible storage medium as recited in claim 16, wherein the load shedding analysis comprises:

determining, prior to initiating the phased termination, that the first group meets an available-capacity criterion.

20. The non-transitory computer-accessible storage medium as recited in claim 16, wherein said detecting that the triggering condition has been met comprises utilizing a time-dependent decay function.

21. The non-transitory computer-accessible storage medium as recited in claim 16, wherein the distributed service comprises a storage service implemented at a provider network.

22. The non-transitory computer-accessible storage medium as recited in claim 16, wherein said detecting that the triggering condition has been met comprises detecting an imbalance between respective workload levels of the first access node and a second access node of the first group.