Method and system for achieving fair command processing in storage systems that implement command-associated priority queuing

• US 20080104283A1
• Filed: 10/31/2006
• Published: 05/01/2008
• Est. Priority Date: 10/31/2006
• Status: Active Grant

• Current Assignee: Hewlett Packard Enterprise Development LP (Hewlett-Packard Enterprise Company)
• Original Assignee: Hewlett-Packard Development Company, L.P. (HP Inc.)
• Inventors: Shin, George, Rao, Santosh Ananth, Grover, Rajiv K.
• Granted Patent: US 7,797,468 B2
• Time in Patent Office: Days
• US Class Current: 710/6
• CPC Class Codes: G06F 3/0611 in relation to response time
• CPC Class Codes: G06F 3/0659 Command handling arrangements, e.g. command buffers, queues, command scheduling
• CPC Class Codes: G06F 3/0683 Plurality of storage devices
• CPC Class Codes: H04L 47/6215 Individual queue per QOS, rate or priority
• CPC Class Codes: H04L 47/6285 Provisions for avoiding starvation of low priority queues
• CPC Class Codes: H04L 49/90 Buffering arrangements
• CPC Class Codes: H04L 67/1097 for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]

Abstract

In certain, currently available data-storage systems, incoming commands from remote host computers are subject to several levels of command-queue-depth-fairness-related throttles to ensure that all host computers accessing the data-storage systems receive a reasonable fraction of data-storage-system command-processing bandwidth to avoid starvation of one or more host computers. Recently, certain host-computer-to-data-storage-system communication protocols have been enhanced to provide for association of priorities with commands. However, these new command-associated priorities may lead to starvation of priority levels and to a risk of deadlock due to priority-level starvation and priority inversion. In various embodiments of the present invention, at least one additional level of command-queue-depth-fairness-related throttling is introduced in order to avoid starvation of one or more priority levels, thereby eliminating or minimizing the risk of priority-level starvation and priority-related deadlock.

Claims

1. An array controller comprising:

ports associated with one or more first communications media that link the array controller to one or more host computers;

ports associated with one or more second communications media that link the array controller to one or more data-storage targets; and

array-controller logic that

receives commands with associated priorities from host computers through the first communications media,

queues received commands according to the priorities associated with the commands to an internal work queue, from which they are subsequently processed for delivery to the data-storage targets,

throttles command reception from host computers in order to control an overall command-reception rate,

throttles command reception from individual host computers in order to provide fair access to all host computers accessing the array controller, and

throttles command reception from host computers according to associated priorities in order to provide fair access to all priority levels of all host computers accessing the array controller.

2. The array controller of claim 1 wherein the array-controller throttles command reception by:

for each received command,

determining whether the received command can be processed;

when the received command can be processed, queuing the received command to the internal work queue; and

when the received command cannot be processed, returning a busy response to the host computer that sent the command to the array controller.

3. The array controller of claim 2 wherein the array-controller, in order to throttle command reception from host computers in order to control an overall command-reception rate, determines whether each received command can be processed by:

determining whether more than a threshold number of entries are currently queued to the work queue.

4. The array controller of claim 2 wherein the array-controller, in order to throttle command reception from individual host computers in order to provide fair access to all host computers accessing the array controller, determines whether each received command can be processed by:

determining whether more than a threshold number of entries representing commands received from a particular host computer that sent the received command are currently queued to the work queue.

5. The array controller of claim 4 wherein the array controller, in determining whether each received command can be processed, further considers a rate of busy-response return to the particular host computer, average rate of command reception from active host computers, and additional command-reception-rate and command-processing-bandwidth considerations.

6. The array controller of claim 2 wherein the array-controller, in order to throttle command reception from host computers according to associated priorities in order to provide fair access to all priority levels of all host computers accessing the array controller, determines whether each received command can be processed by:

determining whether more than a threshold number of entries representing commands received from a particular priority level of the particular host computer that sent the received command are currently queued to the work queue.

7. The array controller of claim 6 wherein the array controller, in determining whether each received command can be processed, further considers a rate of busy-response return to particular priority levels of the particular host computer, average rate of command reception from various priority levels of active host computers, and additional priority-level-related command-reception-rate and command-processing-bandwidth considerations.

8. The array controller of claim 1 wherein the array controller throttles command reception independently for each port associated with one or more first communications media.

9. A method for managing command-reception rates in an array controller that includes ports associated with one or more first communications media that link the array controller to one or more host computers, ports associated with one or more second communications media that link the array controller to one or more data-storage targets, and array-controller logic, the method comprising:

receiving commands with associated priorities from host computers through the first communications media,

queuing received commands according to the priorities associated with the commands to an internal work queue, from which they are subsequently processed for delivery to the data-storage targets,

throttling command reception from host computers in order to control an overall command-reception rate,

throttling command reception from individual host computers in order to provide fair access to all host computers accessing the array controller, and

throttling command reception from host computers according to associated priorities in order to provide fair access to all priority levels of all host computers accessing the array controller.

10. The method of claim 9 further including throttling command reception by:

for each received command,

determining whether the received command can be processed;

when the received command can be processed, queuing the received command to the internal work queue; and

when the received command cannot be processed, returning a busy response to the host computer that sent the command to the array controller.

11. The method of claim 10 wherein, in order to throttle command reception from host computers in order to control an overall command-reception rate, determining whether each received command can be processed by:

determining whether more than a threshold number of entries are currently queued to the work queue.

12. The method of claim 10 wherein, in order to throttle command reception from individual host computers in order to provide fair access to all host computers accessing the array controller, determining whether each received command can be processed by:

determining whether more than a threshold number of entries representing commands received from a particular host computer that sent the received command are currently queued to the work queue.

13. The method of claim 12 wherein, in determining whether each received command can be processed, further considering a rate of busy-response return to the particular host computer, average rate of command reception from active host computers, and additional command-reception-rate and command-processing-bandwidth considerations.

14. The method of claim 10 wherein, in order to throttle command reception from host computers according to associated priorities in order to provide fair access to all priority levels of all host computers accessing the array controller, determining whether each received command can be processed by:

determining whether more than a threshold number of entries representing commands received from a particular priority level of the particular host computer that sent the received command are currently queued to the work queue.

15. The method of claim 14 wherein, in determining whether each received command can be processed, further considering a rate of busy-response return to particular priority levels of the particular host computer, average rate of command reception from various priority levels of active host computers, and additional priority-level-related command-reception-rate and command-processing-bandwidth considerations.

16. The method of claim 9 further including throttling command reception independently for each port associated with one or more first communications media.

17. Processor instructions stored in a computer readable memory that implement the method of claim 9.

18. Firmware instructions stored in a computer readable memory that implement the method of claim 9.