Maximizing storage system throughput by measuring system performance metrics

• US 20070083727A1
• Filed: 10/06/2005
• Published: 04/12/2007
• Est. Priority Date: 10/06/2005
• Status: Active Grant

• Current Assignee: NetApp, Inc.
• Original Assignee: Network Appliance Incorporated (NetApp, Inc.)
• Inventors: Stager, Roger, Johnston, Craig
• Granted Patent: US 7,401,198 B2
• Time in Patent Office: Days
• US Class Current: 711/170
• CPC Class Codes: G06F 3/0613 in relation to throughput
• CPC Class Codes: G06F 3/0635 by changing the path, e.g. traffic rerouting, path reconfiguration
• CPC Class Codes: G06F 3/0653 Monitoring storage devices or systems
• CPC Class Codes: G06F 3/067 Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
• CPC Class Codes: G06F 3/0686 Libraries, e.g. tape libraries, jukebox

Abstract

A method for stream allocation in a data storage system begins by determining whether a stream needs storage space. Storage space for the stream is requested, and a device on which to allocate the storage space is selected. The storage space on the selected device is allocated to the stream.

Claims

1. A method for stream allocation in a data storage system, comprising the steps of:

determining whether a stream needs storage space;

requesting storage space for the stream;

selecting a device on which to allocate the storage space; and

allocating the storage space on the selected device to the stream.

2. The method according to claim 1, wherein the method is used to allocate additional storage space to an existing stream.

3. The method according to claim 1, wherein the determining step includes a trigger being activated to indicate that storage space is needed.

4. The method according to claim 3, wherein the trigger is activated by a stream exhausting its allocated storage space.

5. The method according to claim 1, wherein the selecting step includes:

determining if there is an unused device in the system; and

selecting the unused device if an unused device exists in the system.

6. The method according to claim 5 wherein if there are no unused devices in the system, then the selecting step further includes:

determining if there is an input limited device in the system; and

selecting an input limited device having a lowest amount of traffic if an input limited device exists in the system.

7. The method according to claim 6, wherein the determining step includes:

determining a length of time a command on the device requires for completion;

measuring a data transfer rate of the device;

multiplying the length of time by the data transfer rate to obtain a value; and

determining that the device is input limited if the value is less than or equal to one.

8. The method according to claim 6, wherein if there are no input limited devices in the system, then the selecting step further includes:

selecting a device having a lowest command completion time.

9. A method for selecting a device to use in a data storage system, comprising the steps of:

(a) determining if there is an unused device in the system;

(b) selecting the unused device if an unused device exists in the system;

(c) determining if there is an input limited device in the system if there are no unused devices in the system;

(d) selecting an input limited device having a lowest amount of traffic if an input limited device exists in the system; and

(e) selecting a device having a lowest command completion time if there are no input limited devices in the system.

10. The method according to claim 9, wherein step (c) includes:

determining a length of time a command on the device requires for completion;

measuring a data transfer rate of the device;

multiplying the length of time by the data transfer rate to obtain a value; and

determining that the device is input limited if the value is less than or equal to one.

11. A system for determining the length of time for command execution in a data storage system, comprising:

a command queue for providing commands to be processed;

a command processor connected to said command queue, said command processor processing each command and sending a command reply to said command queue; and

a timer connected to said command queue, said timer marking a first time, when a command is sent from said command queue to said command processor, and a second time, when the command reply is received at said command queue, whereby the length of time is determined by subtracting the first time from the second time.

12. The system according to claim 11, wherein said command processor includes:

a host bus adapter (HBA) driver; and

a small computer system interface mid-level device connected to said HBA driver, said mid-level device receiving the command from said command queue and passing the command to said HBA driver, said HBA driver processing the command and sending the command reply.

13. A method for switching streams between devices in a data storage system, the method comprising the steps of:

(a) determining a size of a new stream;

(b) classifying the new stream, based on its size;

(c) determining if any unused devices exist in the system;

(d) assigning the new stream to an unused device if an unused device exists in the system and if the new stream is larger than average; and

(e) assigning the new stream to an input limited device in the system if there are no unused devices in the system and if the new stream is smaller than average.

14. The method according to claim 13, wherein step (d) includes:

locating the largest stream; and

assigning the largest stream to an unused device.

15. The method according to claim 13, wherein step (e) includes:

locating the smallest stream; and

assigning the smallest stream to an input limited device.

16. A system for performing load balancing in a data storage system having a plurality of storage devices, the system comprising:

a command queue configured to process commands;

a device driver configured to send commands to the plurality of storage devices;

a statistics device configured to gather statistics from said device driver regarding command completion times; and

a device allocator configured to allocate space on one of the plurality of storage devices based on the gathered statistics, whereby the data storage system is load balanced.

17. The system according to claim 16, wherein said command queue includes:

a command handler configured to process a command; and

a statistics-gathering node configured to gather statistics from said device driver regarding command completion times, said statistics device requesting statistics from said statistics-gathering node.

18. The system according to claim 17, wherein said command queue further includes:

a virtual target configured to determine whether said command handler can process the command, said virtual target further configured to request an appropriate command handler if said command handler cannot process the command.

19. The system according to claim 18, further comprising:

a virtual tape device configured to process any command type and configured to instantiate a command handler in said command queue upon request of said virtual target.

20. The system according to claim 17, further comprising:

a virtual tape volume configured to request space on one of the plurality of storage devices, said command handler communicating with said virtual tape volume to request space needed by the command.

21. The system according to claim 20, further comprising:

a data map configured to allocate a portion of the space in the plurality of storage devices.

22. The system according to claim 21, wherein if said data map has no space to allocate, said data map is further configured to request additional space in the plurality of storage devices from said device allocator.