Load balancing a data storage system

• US 7,669,029 B1
• Filed: 11/15/2006
• Issued: 02/23/2010
• Est. Priority Date: 11/15/2006
• Status: Active Grant

• Current Assignee: Network Appliance Incorporated (NetApp, Inc.)
• Original Assignee: Network Appliance Incorporated (NetApp, Inc.)
• Inventors: Ramany, Swaminathan, Mishra, Kesari
• Primary Examiner(s): SONG, HUA JASMINE
• Application Number:
  US11/600,526
• Time in Patent Office: 1,196 Days
• Field of Search: 711111-112, 711170-172
• US Class Current: 711/170
• CPC Class Codes: G06F 11/1076 Parity data used in redundant arrays of independent storages, e.g. in RAID systems
• CPC Class Codes: G06F 11/3433 for load management allocation of a server based on load conditions G06F9/505; load rebalancing G06F9/5083; redistributing the load in a network by a load balancer H04L67/1029
• CPC Class Codes: G06F 2211/1061 Parity-single bit-RAID4, i.e. RAID 4 implementations
• CPC Class Codes: G06F 9/5061 Partitioning or combining of resources

Abstract

An apparatus and method for automatically load balancing one or more workload groups to a set of available physical resources of a data storage system, and generating a layout planning recommendation of the set of available physical resources that supports the one or more workload groups based on the load balancing.

Claims

1. A method for layout planning of a data storage system, comprising:

receiving information indicative of a set of available physical resources of a data storage system and a plurality of workload groups;

based on the information, automatically load balancing the plurality of workload groups to the set of available physical resources of the data storage system; and

generating a layout planning recommendation of the set of available physical resources that supports the plurality of workload groups based on the load balancing, wherein the load balancing further comprises;

performing a first packing algorithm using one or more parameters;

performing a second packing algorithm using the one or more parameters; and

determining which packing algorithm includes a least variance on a selected parameter of the one or more parameters to generate the layout planning recommendation.

2. The method of claim 1, wherein load balancing comprises mapping the plurality of workload groups to the set of available physical resources through heuristics.

3. The method of claim 2, wherein a first parameter of the one or more parameters is the available processing resources on each storage server after mapping, and a second parameter of the one or more parameters is the available capacity resources of one or more storage devices corresponding to each storage server after mapping, and wherein each workload group comprises a processing requirement and a capacity requirement.

4. The method of claim 2, wherein the first packing algorithm comprises a Worst-Fit bin packing algorithm, and the second packing algorithm comprises a First-Fit bin packing algorithm.

5. The method of claim 2, wherein generating the layout planning recommendation further comprises determining a number of storage servers and a number of storage devices to be coupled to each of the storage servers to support the plurality of workload groups.

6. The method of claim 2, wherein generating the layout planning recommendation further comprises determining one or more of the following:

a number of storage servers, a number of storage volumes, a number of databases, a number of logs, and a number of storage devices to be coupled to each of the storage servers to support the plurality of workload groups.

7. The method of claim 6, wherein generating the layout planning recommendation further comprises determining types of each of the storage servers and storage devices.

8. The method of claim 1, further comprising outputting the layout planning recommendation to a user.

9. The method of claim 8, wherein receiving information comprises receiving the information from a calling application program, and wherein outputting the layout planning recommendation comprises outputting the layout planning recommendation to the calling application program.

10. The method of claim 9, wherein receiving the information indicative of the set of available physical resources and the plurality of workload groups comprises:

receiving the information indicative of set of available physical resources from either from a sizing tool of a sizing infrastructure module or the calling application program; and

receiving the plurality of workload groups from either the calling application program or the sizing tool of the sizing infrastructure module.

11. The method of claim 10, wherein the sizing infrastructure module and the calling application program each execute on a respective computer, and wherein communication between the respective computers takes place over a network.

12. The method of claim 11, wherein receiving from the calling application program comprises receiving the requirements through an XML remote procedure call over an HTTP connection.

13. A method for layout planning of a data storage system, comprising:

receiving information indicative of a set of available physical resources of a data storage system and a plurality of workload groups;

based on the information, automatically load balancing the plurality of workload groups to the set of available physical resources of the data storage system, wherein the available physical resources comprise one or more storage devices, and one or more storage server, wherein the load balancing further comprises;

performing a first packing algorithm using one or more parameters; and

performing a second packing algorithm using the one or more parameters, wherein performing the first packing algorithm comprises;

selecting a workload group of the plurality of workload groups; and

mapping the workload group to a storage server of the one or more storage servers that has a maximum amount of available processing resource and that can support the capacity requirement of the selected workload group, and wherein performing the second packing algorithm comprises;

mapping one or more workload groups to a first storage server of the one or more storage servers until the first storage server either does not have enough available processing resources to support the processing requirement of a current workload group or cannot support the capacity requirement of the current workload group; and

mapping one or more workload groups to a second storage server when the first storage server does not have enough available processing resources to support the processing requirement of the current workload group or cannot support the capacity requirement of the current workload group; and

generating a layout planning recommendation of the set of available physical resources that supports the plurality of workload groups, wherein generating the layout planning recommendation comprises determining which packing algorithm includes a least variance on a selected parameter of the one or more parameters.

14. The method of claim 13, wherein determining which packing algorithm includes the least variance comprises:

determining whether both the first and second packing algorithms were successful in assigning the plurality of workload groups to the one or more storage servers; and

in response to determining that both the first and second packing algorithms were successful, selecting a layout planning recommendation from either the first or second packing algorithm based on the packing algorithm that includes the least variance on the selected parameter of the one or more parameters.

15. The method of claim 14, further comprising:

in response to determining that only one of the first and second packing algorithms were successful, selecting a layout planning recommendation from the one packing algorithm that was successful; and

in response to determining that none of the first and second packing algorithms were successful, returning a message to suggest splitting the plurality of workload groups into additional workload groups.

16. The method of claim 15, wherein determining which packing algorithm includes the least variance further comprises:

in response to determining that none of the first and second packing algorithms were successful, determining whether one of the first and second packing algorithms was unsuccessful due to a storage server not being able to support the capacity requirement of the selected workload group;

in response to determining that one of the packing algorithms was unsuccessful, performing a third packing algorithm and a fourth packing algorithm, wherein the third packing algorithm comprises;

selecting a workload group of the plurality of workload groups; and

mapping the workload group to a storage server that can support the capacity requirement and the processing requirement of the selected workload group; and

wherein the fourth packing algorithm comprises;

mapping one or more workload groups to a first storage server until the first storage server cannot support the capacity requirement of another workload group; and

mapping one or more workload groups to a second storage server when the first storage server cannot support the capacity requirement of another workload group;

in response to determining that both the third and fourth packing algorithms were successful, selecting a layout planning recommendation from either the third or fourth packing algorithm based on the packing algorithm that includes the least variance on the selected parameter of the one or more parameters;

in response to determining that only one of third and fourth packing algorithms were successful, selecting a layout planning recommendation from the one packing algorithm that was successful; and

in response to determining that none of the third and fourth packing algorithms were successful, returning a message to suggest splitting the plurality of workload groups into additional workload groups.

17. The method of claim 15, wherein the selected layout is based on the least variance of the first parameter, which is based on a least variance of available processing resources on each storage server after the mapping.

18. A sizing tool for a data storage system, comprising:

a layout planning recommendation module configured to receive information indicative of a set of available physical resources and a plurality of workload groups from a configuration estimation module, and to automatically load balance the plurality of workload groups to the set of available physical resources based on the information, wherein the layout planning recommendation module is configured to automatically load balance by;

performing a first packing algorithm using one or more parameters;

performing a second packing algorithm using the one or more parameters; and

determining which packing algorithm includes a least variance on a selected parameter of the one or more parameters to generate the layout planning recommendation.

19. The sizing tool of claim 18, wherein the layout planning recommendation module is configured to output a layout planning recommendation to a user based on the load balancing.

20. The sizing tool of claim 19, wherein the sizing tool is coupled a user interface, wherein the user interface is configured to receive the layout planning recommendation from the layout planning recommendation module and to display the layout planning recommendation on a display.

21. The sizing tool of claim 20, wherein the user interface is configured to receive the information indicative of the plurality of workload groups from a user and to send the information to the layout planning recommendation module.

22. The sizing tool of claim 20, wherein the user interface is configured to allow a user to alter the layout planning recommendation.

23. The sizing tool of claim 20, wherein the user interface is configured to store the layout planning recommendation in memory.

24. The sizing tool of claim 18, wherein the plurality of workload groups each comprise storage resource requirements.

25. The sizing tool of claim 24, wherein the storage resource requirements comprises a processing resource requirement and a capacity resource requirement.

26. The sizing tool of claim 18, wherein the available physical resources comprise one or more storage devices, and one or more storage servers.

27. The sizing tool of claim 18, further comprising the configuration estimation module coupled to the layout planning recommendation module, wherein the configuration estimation module is configured to receive calls from one or more applications, and wherein the configuration estimation module is configured to receive the plurality of workload groups from an calling application program of one of the one or more applications.

28. The sizing tool of claim 27, further comprising the one or more applications coupled to the configuration estimation module.

29. The sizing tool of claim 28, wherein the one or more applications are sizing tools.

30. The sizing tool of claim 29, wherein each sizing tool is at least one of an Exchange Sizer, a Database Sizer, a Custom Application Sizer, or a Request-For-Extension Sizer.

31. A machine readable storage medium having instructions thereon, which instructions, when executed by a digital processing device, cause the digital processing device to perform the following, comprising:

receiving information indicative of a set of available physical resources and a plurality of workload groups;

based on the information, load balancing the plurality of workload groups to the set of available physical resources of a data storage system, wherein the load balancing is automated, wherein load balancing comprises mapping the plurality of workload groups to the set of available physical resources through heuristics using two or more packing algorithms and a least variance on one selected parameter of two or more packing parameters; and

generating a layout planning recommendation of the set of available physical resources that supports the plurality of workload groups based on the load balancing.

32. The machine readable storage medium of claim 31, having further instructions thereon, which further instructions, when executed by the digital processing device, cause the digital processing device to perform the following, comprising outputting a layout planning recommendation to a user.

33. The machine readable storage medium of claim 32, wherein the available physical resources comprise one or more storage servers and one or more corresponding storage devices, wherein a first parameter of the two or more packing parameters is the available processing resources on each storage server after mapping, and a second parameter of the two or more packing parameters is the available capacity resources of one or more storage devices corresponding to each storage server after mapping, and wherein each workload group comprises a processing requirement and a capacity requirement.

34. The machine readable storage medium of claim 31, wherein performing the first packing algorithm comprises:

selecting a workload group of the plurality of workload groups; and

mapping the workload group to a storage server of the one or more storage servers that has a maximum amount of available processing resource and that can support the capacity requirement of the selected workload group, and wherein performing the second packing algorithm comprises;

mapping one or more workload groups to a first storage server of the one or more storage servers until the first storage server either does not have enough available processing resources to support the processing requirement of a current workload group or cannot support the capacity requirement of the current workload group; and

mapping one or more workload groups to a second storage server when the first storage server does not have enough available processing resources to support the processing requirement of the current workload group or cannot support the capacity requirement of the current workload group.