Technique for throttling data access requests

• US 7,529,836 B1
• Filed: 01/08/2004
• Issued: 05/05/2009
• Est. Priority Date: 01/08/2004
• Status: Active Grant

• Current Assignee: NetApp, Inc.
• Original Assignee: Network Appliance Incorporated (NetApp, Inc.)
• Inventors: Bolen, David Brittain
• Primary Examiner(s): Winder; Patrice
• Assistant Examiner(s): Goodchild; William J
• Application Number:
  US10/753,750
• Time in Patent Office: 1,944 Days
• Field of Search: 709/226, 709/234
• US Class Current: 709/226
• CPC Class Codes: G06F 3/0611 in relation to response time
• CPC Class Codes: G06F 3/0656 Data buffering arrangements
• CPC Class Codes: G06F 3/067 Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
• CPC Class Codes: H04L 67/1097 for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]

Abstract

A system and method for throttling data access rates through data containers prevents starvation of storage system resources by active clients. A SCSI throttle module of a storage operating system of storage system determines if the number of outstanding access requests associated with the client or initiator group exceeds a preset throttle value.

Claims

1. A method for reducing issuance of data access requests to a storage system from clients of the storage system, comprising:

configuring a first group of clients with a first predetermined number of queues that may be utilized by the clients in the first group, the predetermined number hereinafter a throttle value, and a second group of clients with a second throttle value;

receiving, by the storage system, a data access request from a client of the first group;

determining that execution of the received data access request will cause the throttle value of the first group of clients to be exceeded;

determining if the client of the first group may use an available queue from the second group of clients, where the second group of clients has not exceeded its throttle value;

sending, if the client of the first group may not use the available queue from the second group, a queue full message to the client of the first group; and

reducing, in response to the queue full message, issuance of the data access requests from the first group to the storage system.

2. The method of claim 1 further comprising, in response to determining that the data access request exceeds the throttle value,

determining if a borrowing flag is set to determine if an available queue may be borrowed.

3. The method of claim 2 further comprising, in response to the borrowing flag not being set or no queue being available, sending N queue full messages to a next N received data access requests.

4. The method of claim 3 wherein N is calculated as a predetermined percentage of a total number of available queues.

5. The method of claim 1 further comprising:

performing, if the client of the first group may use the available queue from the second throttle value of the second group of clients, the pending data access request using the available queue.

6. The method of claim 5 further comprising:

incrementing a master throttle counter; and

decrementing the master throttle counter, in response to performing the pending data access request.

7. The method of claim 6 further comprising:

using the master throttle counter in determining the number of Pending data access requests.

8. The method of claim 1 wherein the throttle value is associated with a particular initiator group.

9. The method of claim 1 wherein the throttle value is associated with a particular client of the storage system.

10. The method of claim 1 wherein the storage system comprises a storage appliance.

11. The method of claim 1 wherein a throttle counter is contained within an initiator group data structure.

12. A storage system for reducing issuance of data access requests to the storage system from the clients of the storage system, comprising:

means for configuring a first group of clients with a first predetermined number of queues that may be utilized by the clients in the first group, the predetermined number hereinafter a throttle value, and a second group of clients with a second throttle value;

means for receiving, by the storage system, a data access request from a client of the first group;

means for determining that execution of the received data access request will cause throttle value of the first group of clients to be exceeded;

means for determining if the client of the first group may use an available queue from the second group of clients, where the second group of clients has not exceeded its throttle value;

means for sending, if the client of the first group may not use the available queue from the second group, a queue full message to the client of the first group; and

means for reducing, in response to the queue full message, issuance of the data access requests from the first group to the storage system.

13. The storage system of claim 12 further comprising:

means for determining if a borrowing flag is set to determine that an available queue may be borrowed.

14. The storage system of claim 13 further comprising means for sending N queue full messages to a next N received data access requests in response to determining that the borrowing flag is not set or no queue being available.

15. The storage system of claim 14 wherein N is calculated as a predetermined percentage of a total number of available queues.

16. The storage system of claim 12 wherein the throttle value is associated with a particular initiator group.

17. The storage system of claim 12 wherein the throttle value is associated with a particular client of the storage system.

18. The storage system of claim 12 wherein the storage system comprises a storage appliance.

19. A storage system for reducing issuance of data access requests from clients, comprising:

the storage system having a target module adapted to interpret data access requests from one or more clients;

a throttle module adapted to determine that a number of data access requests will exceed a predetermined throttle value of pending data access requests for the one or more clients;

the throttle module further adapted to determine if the one or more client may use an available queue from another client that has not exceeded its predetermined throttle value; and

in response to the determination that no queue from the another client is available, the throttle module further adapted to reduce issuance of the data access requests from the one or more clients.

20. The storage system of claim 19 wherein the throttle module is further adapted to determine if a borrowing flag is set and to determine if a queue is available.

21. The storage system of claim 20 wherein the throttle module is further adapted to send N queue full messages in response to determining that the borrowing flag is not set or if no queue is available.

22. The storage system of claim 21 wherein the N queue full messages are sent in response to a next N received data access requests.

23. The storage system of claim 19 wherein the throttle module is further adapted to modify throttle values as a result of a failover operation.

24. The storage system of claim 23 wherein the throttle module is further adapted to half each throttle value when virtual ports are associated with physical ports during the failover operation.

25. The storage system of claim 19 wherein the throttle module further comprises a master throttle associated with clients that are not associated with another throttle.

26. The storage system of claim 19 further comprises a master throttle associated with one or more queues that are not associated with another throttle associated with the storage system.

27. The storage system of claim 19 wherein the throttle module is further adapted to count a received data access request to be proxied to a partner storage system against the throttle value.

28. A computer readable media, comprising:

said computer readable media containing instructions for execution on a processor for the practice of a method for reducing issuance of data access requests to a storage system, the method having,

configuring a first group of clients with a first predetermined number of queues that may be utilized by the clients in the first group, the predetermined number hereinafter a throttle value, and a second group of clients with a second throttle value;

receiving, by the storage system, a data access request from a client of the first group;

determining that execution of the received data access request will cause the throttle value of the first group of clients to be exceeded;

determining if the client of the first group may use an available queue from the second group of clients, where the second group of clients another client that has not exceeded its throttle value;

sending, if the client of the first group may not use the available queue from the second group, a queue full message to the client of the first group; and

reducing, in response to the queue full message, issuance of the data access requests from the first group to the storage system.

29. A method for reducing issuance of data access requests to a storage system from a plurality of clients of the storage system, comprising:

receiving one or more data access requests from the plurality of clients;

associating a plurality of queues with a first initiator group;

determining if the received data access requests causes a number of data access requests to exceed a predetermined throttle value associated with the first initiator group;

in response to determining that the number of data access requests exceeds the throttle value associated with the first initiator group,

(i) determining if any queues are available from another initiator group;

(ii) borrowing the available queues from the another initiator group; and

(iii) performing the data access requests using the available queues.

30. The method of claim 29 further comprising, in response to determining that the number of data access requests exceeds the throttle value, determining if a borrowing flag is set.

31. The method of claim 30 further comprising, in response to the borrowing flag not being set or no queue being available, sending N queue full messages to a next N received data access requests.

32. The method of claim 29 further comprising:

incrementing a throttle counter;

performing the data access requests; and

decrementing the throttle counter.

33. A method for reducing issuance of data access requests to a storage system from a plurality of clients of the storage system, comprising:

receiving one or more data access requests from a first plurality of clients;

associating a first plurality of queues with a first initiator group;

associating a second plurality of queues with a second initiator group;

determining if the received data access requests from the first plurality of clients causes a number of data access requests to exceed a predetermined throttle value associated with the first initiator group;

in response to determining that the number of data access requests exceeds the throttle value associated with the first initiator group,

(i) determining if any queues are available from the second initiator group;

(ii) borrowing the available queues from the second plurality of queues for use with the first plurality of queues associated with the first initiator group; and

(iii) performing the data access requests using the second plurality of available queues.

34. The method of claim 33 further comprising, in response to determining that the number of data access requests exceeds the throttle value, determining if a borrowing flag is set.

35. The method of claim 34 further comprising, in response to the borrowing flag not being set or no queue being available, sending N queue full messages to a next N received data access requests.

36. The method of claim 33 further comprising:

incrementing a throttle counter;

performing the data access requests; and

decrementing the throttle counter.

37. A method for reducing issuance of data access requests to a storage system from a plurality of clients of the storage system, comprising:

receiving one or more data access requests from the plurality of clients;

associating a plurality of queues with a first initiator group;

determining if the received data access requests causes a number of data access requests to exceed a predetermined throttle value associated with the first initiator group;

in response to determining that the number of data access requests exceeds the throttle value associated with the first initiator group,

(i) determining if a borrowing flag is set;

(ii) determining if any queues are available from another initiator group;

(iii) borrowing the available queues from the another initiator group; and

(iv) performing the data access requests using the available queues.

38. The method of claim 37 further comprising, in response to the borrowing flag not being set or no queue being available, sending N queue full messages to a next N received data access requests.

39. A method for reducing issuance of data access requests to a storage system from clients of the storage system, comprising:

configuring a first group of clients with a first predetermined number of queues that may be utilized by the clients in the first group, the predetermined number hereinafter a throttle value;

configuring a second group of clients with a second throttle value;

determining that execution of a next data access request sent from the first client group will cause the first throttle value of the first group of clients to be exceeded; and

reducing issuance of the data access requests, if the first group of clients may not use a queue from the second group of clients to execute the next data access request.

40. The method of claim 39 further comprising:

sending, if the first group of clients may not use the queue from the second group of clients, a queue full message indicating the first group of clients may not use the queue from the second group of clients.