Storage management for a cluster of integrated computing systems comprising integrated resource infrastructure using storage resource agents and synchronized inter-system storage priority map

• US 9,417,903 B2
• Filed: 06/21/2013
• Issued: 08/16/2016
• Est. Priority Date: 06/21/2013
• Status: Expired due to Fees

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Bello, Adekunle, Yedavilli, Aruna
• Primary Examiner(s): GHAFFARI, ABU Z
• Application Number:
  US13/923,863
• Publication Number:
  US 20140380303A1
• Time in Patent Office: 1,152 Days
• Field of Search: None
• US Class Current: 1/1
• CPC Class Codes: G06F 15/161 Computing infrastructure, e.g. computer clusters, blade chassis or hardware partitioning casings, cabinets, racks or drawers for data centers H05K5/00
• CPC Class Codes: G06F 15/17343 wherein the interconnection is dynamically configurable, e.g. having loosely coupled nearest neighbor architecture reconfigurable processors arrays G06F15/7867
• CPC Class Codes: G06F 16/1827 Management specifically adapted to NAS management of storage area networks [SAN] G06F3/067
• CPC Class Codes: G06F 2009/4557 Distribution of virtual machine instances; Migration and load balancing
• CPC Class Codes: G06F 2009/45579 I/O management, e.g. providing access to device drivers or storage
• CPC Class Codes: G06F 3/0604 Improving or facilitating administration, e.g. storage management
• CPC Class Codes: G06F 3/067 Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
• CPC Class Codes: G06F 9/45558 Hypervisor-specific management and integration aspects
• CPC Class Codes: H04L 67/1097 for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]

Abstract

Integrated computing systems with independently managed infrastructures including compute nodes and storage nodes form a cluster. Storage resource agents manage storage resources in the cluster. The resource agents identify storage requirements associated with allocation sets for resource consumers dispatched in the cluster, communicate with each other to locate inter-system storage resources that primarily satisfy locality criteria associated with resource consumer workloads, secondarily satisfy allocation set activity criteria associated with the allocation sets, and allocate the storage resources to the resource consumers to satisfy the storage requirements. The storage resource agents may base storage assignments on data placement information from a priority map. Data may be later relocated to alternate storage resources in satisfaction of cluster-wide storage policies, priority determinations, and data access rate determinations.

Claims

1. A method for managing data storage resources in a cluster of integrated systems, each integrated system having an integrated resource infrastructure, the integrated resource infrastructure including an at least one compute node and intra-system storage, the intra-system storage including data storage resources in an at least one storage node, the integrated resource infrastructure of each integrated system managed independently of the integrated resource infrastructure of each of the other integrated systems in the cluster, the method comprising:

associating a storage resource agent with each integrated system, each storage resource agent included in a plurality of distributed storage resource agents, the plurality of distributed storage resource agents communicatively interconnected and managing inter-system storage, the inter-system storage including the data storage resources;

forming, by at least two storage resource agents associated with the integrated systems, an inter-system storage priority map, the contents of the inter-system storage priority map synchronized by the at least two storage resource agents, the inter-system storage priority map including data set allocation criteria associated with each data allocation set, the data set allocation criteria associated with each data allocation set including at least one of a storage directive, an inter-system storage priority for accessing data storage resources in the inter-system storage, and data allocation set attributes descriptive of the data allocation set;

identifying, by a first storage resource agent associated with a first integrated system in the cluster, a first data storage requirement for a first resource consumer, the first data storage requirement associated with a first data allocation set, the first data allocation set associated with a workload of the first resource consumer, the first data allocation set based at least in part on data associated with the first data storage requirement, the first resource consumer dispatched on a first virtual machine hosted on a first compute node included in the first integrated system;

locating, by the first storage resource agent and an at least one second storage resource agent associated with an integrated system that is not the first integrated system, an at least one candidate data storage resource in the inter-system storage, the at least one candidate data storage resource within the data storage resources of the intra-system storage of at least one integrated system within the cluster;

selecting a data storage resource from the at least one candidate data storage resource, the selecting utilizing first data set allocation criteria included in the inter-system storage priority map, the first data set allocation criteria associated with the first data allocation set, the selected data storage resource satisfying a data allocation criterion included in the first data set allocation criteria; and

allocating the selected data storage resource to the first resource consumer to satisfy the first data storage requirement.

2. The method of claim 1, wherein the method further includes the at least one candidate data storage resource satisfying a locality criterion associated with the workload, wherein the at least one candidate data storage resource is associated with a locality measurement based at least in part on the length of a transmission path between the at least one candidate storage resource and the first compute node, wherein the locality measurement satisfies the locality criterion, wherein the selecting the data storage resource further includes the selected data storage resource satisfying a data allocation set activity criterion, wherein the data allocation set activity criterion includes a previous association, wherein the previous association is based at least in part on the data allocation set attributes included in the inter-system storage priority map, and wherein the data allocation set activity criterion is selected from the group consisting of:

a first requirement that the data allocation set has been previously hosted by the selected data storage resource, the data allocation set not presently hosted by the selected data storage resource;

a second requirement that the workload be previously associated with the selected data storage resource, the previous association including the selected data storage resource hosting, within a defined period of time, any data allocation set associated with the workload; and

a third requirement that first resource consumer be previously associated with the selected data storage resource, the previous association including the selected data storage resource hosting, within a defined period of time, any data allocation set associated with the first resource consumer.

3. The method of claim 1, wherein the first resource consumer has a first inter-system storage priority for accessing data storage resources in the inter-system storage, wherein a second resource consumer dispatched on a second virtual machine hosted on a second compute node in a second integrated system has a second inter-system storage priority for accessing data storage resources in the inter-system storage, and wherein the selected data storage resource is concurrently selected to satisfy a second data storage requirement for the second resource consumer, the method further comprising:

determining, by the storage resource agents associated with the integrated systems, using the inter-system storage priority map, that the first inter-system storage priority is higher than the second inter-system storage priority.

4. The method of claim 3, wherein a plurality of resource consumers is dispatched on a plurality of virtual machines hosted on the plurality of compute nodes in the cluster of integrated systems, wherein each resource consumer has a consumer inter-system storage priority for accessing data storage resources in the-inter-system storage, relative to all resource consumers in the cluster, wherein each virtual machine has a virtual machine inter-system storage priority for accessing data storage resources in the-inter-system storage, relative to all virtual machines in the cluster, and wherein each integrated system has a system inter-system storage priority for accessing data storage resources in the-inter-system storage, relative to all integrated systems in the cluster, wherein each of the consumer, virtual machine, and system inter-system storage priorities for each resource consumer, virtual machine, and integrated system in the cluster are included the inter-system storage priority map, the method further comprising:

identifying, from the inter-system storage priority map, a first consumer inter-system storage priority for the first resource consumer;

identifying, from the inter-system storage priority map, a first virtual machine inter-system storage priority for the first virtual machine;

identifying, from the inter-system storage priority map, a first system inter-system storage priority for the first integrated system;

identifying, from the inter-system storage priority map, a second consumer inter-system storage priority for the second resource consumer;

identifying, from the inter-system storage priority map, a second virtual machine inter-system storage priority for the second virtual machine;

identifying, from the inter-system storage priority map, a second system inter-system storage priority for the second integrated system;

calculating the first inter-system storage priority according to a weighted formula based on at least one of the first consumer inter-system storage priority, the first virtual machine inter-system storage priority, and the first system inter-system storage priority; and

calculating the second inter-system storage priority according to the weighted formula based on at least one of the second consumer inter-system storage priority, the second virtual machine inter-system storage priority, and the second system inter-system storage priority.

5. The method of claim 1, wherein the selected data storage resource is not included in the intra-system storage in the first integrated system.

6. The method of claim 1, wherein the data set allocation criteria includes an inter-system storage criterion, wherein the allocating the selected data storage resource to the first resource consumer is based at least in part on the selected data storage resource satisfying the inter-system storage criterion, wherein the selected data storage resource satisfying the inter-system storage criterion is based at least in part on the data allocation set attributes included in the inter-system storage priority map, and wherein the inter-system storage criterion is selected from the group consisting of:

a first directive indicating that the first resource consumer is eligible to use data storage resources included in the inter-system data storage, the first directive associated with the first resource consumer;

a second directive indicating that the data associated with the first data storage requirement is eligible to use data storage resources included in the inter-system data storage, the second directive associated with the first resource consumer;

a first determination that the first resource consumer has an inter-system storage priority for accessing data storage resources in the inter-system storage below an inter-system storage priority threshold; and

a second determination that the first resource consumer has a first inter-system storage priority for accessing data storage resources in the inter-system storage higher than a second inter-system storage priority for accessing data storage resources in the inter-system storage, a second resource consumer having the second inter-system storage priority.

7. The method of claim 1, wherein the first resource consumer has a first inter-system storage priority for accessing data storage resources in the inter-system storage, and wherein the allocating the selected data storage resource comprises:

determining that the selected data storage resource is allocated to a second resource consumer, the second resource consumer having a second inter-system storage priority for accessing data storage resources in the inter-system storage, the second inter-system storage priority lower than the first inter-system storage priority;

deallocating the selected data storage resource from the second resource consumer; and

allocating, subsequent to the de-allocating, the selected data storage resource to the first resource consumer.

8. The method of claim 1, further comprising:

locating, using the inter-system storage map, subsequent to the allocating the selected data storage resource to the first resource consumer, an alternate data storage resource in the inter-system storage; and

relocating the first data allocation set from the selected data storage resource to the alternate data storage resource, the relocating satisfying an inter-system storage relocation criterion.

9. The method of claim 8, wherein the inter-system storage relocation criterion is selected from the group consisting of:

a determination that the first resource consumer has an-inter-system storage priority, included in the inter-system priority map, for accessing data storage resources in the-inter-system storage above an inter-system storage priority threshold;

a first inter-system storage policy of prioritizing allocation of intra-system data storage resources over allocation of data storage resources in inter-system storage; and

a second inter-system storage policy of balancing distribution of available data storage resources in inter-system and intra-system storage in the cluster.

10. A computer program product for managing data storage resources in a cluster of integrated systems, each integrated system having an integrated resource infrastructure, the integrated resource infrastructure including an at least one compute node and intra-system storage, the intra-system storage including data storage resources in an at least one storage node, the integrated resource infrastructure of each integrated system managed independently of the integrated resource infrastructure of each of the other integrated systems in the cluster, the computer program product comprising a non-transitory computer-readable storage medium having program code embodied therewith, the program code executable by a computer system to perform a method comprising:

associating a storage resource agent with each integrated system, each storage resource agent included in a plurality of distributed storage resource agents, the plurality of distributed storage resource agents communicatively interconnected and managing inter-system storage, the inter-system storage including the data storage resources;

forming, by at least two storage resource agents associated with the integrated systems, an inter-system storage priority map, the contents of the inter-system storage priority map synchronized by the at least two storage resource agents, the inter-system storage priority map including data set allocation criteria associated with each data allocation set, the data set allocation criteria associated with each data allocation set including at least one of a storage directive, an inter-system storage priority for accessing data storage resources in the inter-system storage, and data allocation set attributes descriptive of the data allocation set;

identifying, by a first storage resource agent associated with a first integrated system in the cluster, a first data storage requirement for a first resource consumer, the first data storage requirement associated with a first data allocation set, the first data allocation set associated with a workload of the first resource consumer, the first data allocation set based at least in part on data associated with the first data storage requirement, the first resource consumer dispatched on a first virtual machine hosted on a first compute node included in the first integrated system,

locating, by the first storage resource agent and an at least one second storage resource agent associated with an integrated system that is not the first integrated system, an at least one candidate data storage resource in the inter-system storage, the at least one candidate data storage resource within the data storage resources of the intra-system storage of at least one integrated system within the cluster;

selecting a data storage resource from the at least one candidate data storage resource, the selecting utilizing first data set allocation criteria included in the inter-system storage priority map, the first data set allocation criteria associated with the first data allocation set, the selected data storage resource satisfying a data allocation criterion included in the first data set allocation criteria; and

allocating the selected data storage resource to the first resource consumer to satisfy the first data storage requirement.

11. The computer program product of claim 10, wherein the first resource consumer has a first inter-system storage priority for accessing data storage resources in the inter-system storage, wherein a second resource consumer dispatched on a second virtual machine hosted on a second compute node in a second integrated system has a second inter-system storage priority for accessing data storage resources in the inter-system storage, and wherein the selected data storage resource is concurrently selected to satisfy a second data storage requirement for the second resource consumer, the method further comprising:

determining, by the storage resource agents associated with the integrated systems, using the inter-system storage priority map, that the first inter-system storage priority is higher than the second inter-system storage priority.

12. The computer program product of claim 11, wherein a plurality of resource consumers is dispatched on a plurality of virtual machines hosted on the plurality of compute nodes in the cluster of integrated systems, wherein each resource consumer has a consumer inter-system storage priority for accessing data storage resources in the-inter-system storage, relative to all resource consumers in the cluster, wherein each virtual machine has a virtual machine inter-system storage priority for accessing data storage resources in the-inter-system storage, relative to all virtual machines in the cluster, and wherein each integrated system has a system inter-system storage priority for accessing data storage resources in the-inter-system storage, relative to all integrated systems in the cluster, wherein each of the consumer, virtual machine, and system inter-system storage priorities for each resource consumer, virtual machine, and integrated system in the cluster are included the inter-system storage priority map, the method further comprising:

identifying, from the inter-system storage priority map, a first consumer inter-system storage priority for the first resource consumer;

identifying, from the inter-system storage priority map, a first virtual machine inter-system storage priority for the first virtual machine;

identifying, from the inter-system storage priority map, a first system inter-system storage priority for the first integrated system;

identifying, from the inter-system storage priority map, a second consumer inter-system storage priority for the second resource consumer;

identifying, from the inter-system storage priority map, a second virtual machine inter-system storage priority for the second virtual machine;

identifying, from the inter-system storage priority map, a second system inter-system storage priority for the second integrated system;

calculating the first-inter-system storage priority according to a weighted formula based on at least one of the first consumer inter-system storage priority, the first virtual machine inter-system storage priority, and the first system inter-system storage priority; and

calculating the second-inter-system storage priority according to the weighted formula based on at least one of the second consumer inter-system storage priority, the second virtual machine inter-system storage priority, and the second system inter-system storage priority.

13. The computer program product of claim 10, wherein the data set allocation criteria includes an inter-system storage criterion, wherein the allocating the selected data storage resource to the first resource consumer is based at least in part on the selected data storage resource satisfying the inter-system storage criterion, wherein the selected data storage resource satisfying the inter-system storage criterion is based at least in part on the data allocation set attributes included in the inter-system storage priority map, and wherein the inter-system storage criterion is selected from the group consisting of:

a first directive indicating that the first resource consumer is eligible to use data storage resources included in the inter-system storage, the first directive associated with the first resource consumer;

a second directive indicating that data associated with the first storage requirement is eligible to use data storage resources included in the inter-system storage, the second directive associated with the first resource consumer;

a first determination that the first resource consumer has an inter-system storage priority for accessing data storage resources in the inter-system storage below an inter-system storage priority threshold; and

a second determination that the first resource consumer has a first inter-system storage priority for accessing data storage resources in the inter-system storage higher than a second inter-system storage priority for accessing data storage resources in the inter-system storage, a second resource consumer having the second inter-system storage priority.

14. The computer program product of claim 10, the method further comprising:

locating, using the inter-system storage map, subsequent to the allocating the selected data storage resource to the first resource consumer, an alternate data storage resource in the inter-system storage; and

relocating the first data allocation set from the selected data storage resource to the alternate data storage resource, the relocating satisfying an inter-system storage relocation criterion.

15. The computer program product of claim 14, wherein the inter-system storage relocation criterion is selected from the group consisting of:

a determination that the first resource consumer has an inter-system storage priority, included in the inter-system priority map, for accessing data storage resources in the inter-system storage above an inter-system storage priority threshold;

a first inter-system storage policy of prioritizing allocation of intra-system data storage resources over allocation of inter-system data storage resources; and

a second inter-system storage policy of balancing distribution of available inter-system and intra-system data storage resources in the cluster.

16. The computer program product of claim 10, wherein the method further includes the at least one candidate data storage resource satisfying a locality criterion associated with the workload, wherein the at least one candidate data storage resource is associated with a locality measurement based at least in part on the length of a transmission path between the at least one candidate storage resource and the first compute node, wherein the locality measurement satisfies the locality criterion, wherein the selecting the data storage resource further includes the selected data storage resource satisfying a data allocation set activity criterion, wherein the data allocation set activity criterion includes a previous association, wherein the previous association is at least in part based on the data allocation set attributes included in the inter-system storage priority map, and wherein the data allocation set activity criterion is selected from the group consisting of:

a first requirement that the data allocation set has been previously hosted by the selected data storage resource, the data allocation set not presently hosted by the selected data storage resource;

a second requirement that the workload be previously associated with the selected data storage resource, the previous association including the selected data storage resource hosting, within a defined period of time, any data allocation set associated with the workload; and

a third requirement that the first resource consumer be previously associated with the selected data storage resource, the previous association including the selected data storage resource hosting, within a defined period of time, any data allocation set associated with the first resource consumer.

17. A computer system for managing data storage resources in a cluster of integrated systems, each integrated system having an integrated resource infrastructure, the integrated resource infrastructure including an at least one compute node and intra-system storage, the intra-system storage including data storage resources in an at least one storage node, the integrated resource infrastructure of each integrated system managed independently of the integrated resource infrastructure of each of the other integrated systems in the cluster, the computer system comprising:

a memory; and

at least one processor in communication with the memory, wherein the computer system is configured to perform a method, the method comprising;

associating a storage resource agent with each integrated system, each storage resource agent included in a plurality of distributed storage resource agents, the plurality of distributed storage resource agents communicatively interconnected and managing inter-system storage, the inter-system storage including the data storage resources;

forming, by at least two storage resource agents associated with the integrated systems, an inter-system storage priority map, the contents of the inter-system storage priority map synchronized by the at least two storage resource agents, the inter-system storage priority map including data set allocation criteria associated with each data allocation set, the data set allocation criteria associated with each data allocation set including at least one of a storage directive, an inter-system storage priority for accessing data storage resources in the inter-system storage, and data allocation set attributes descriptive of the data allocation set;

identifying, by a first storage resource agent associated with a first integrated system in the cluster, a first data storage requirement for a first resource consumer, the first data storage requirement associated with a first data allocation set, the first data allocation set associated with a workload of the first resource consumer, the first data allocation set based at least in part on data associated with the first data storage requirement, the first resource consumer dispatched on a first virtual machine hosted on a first compute node included in the first integrated system;

locating, by the first storage resource agent and an at least one second storage resource agent associated with an integrated system that is not the first integrated system, an at least one candidate data storage resource in the inter-system storage, the at least one candidate data storage resource within the data storage resources of the intra-system storage of at least one integrated system within the cluster;

selecting a data storage resource from the at least one candidate data storage resource, the selecting utilizing first data set allocation criteria included in the inter-system storage priority map, the first data set allocation criteria associated with the first data allocation set, the selected data storage resource satisfying a data allocation criterion included in the first data set allocation criteria; and

allocating the selected data storage resource to the first resource consumer to satisfy the first data storage requirement.

18. The computer system of claim 17, wherein the method further includes the at least one candidate data storage resource satisfying a locality criterion associated with the workload, wherein the at least one candidate data storage resource is associated with a locality measurement based at least in part on the length of a transmission path between the at least one candidate storage resource and the first compute node, wherein the locality measurement satisfies the locality criterion, wherein the selecting the data storage resource further includes the selected data storage resource satisfying a data allocation set activity criterion, wherein the data allocation set activity criterion includes a previous association, wherein the previous association is at least in part based on the data allocation set attributes included in the inter-system storage priority map, and wherein the data allocation set activity criterion is selected from the group consisting of:

a first requirement that the data allocation set be previously associated with the selected data storage resource, the previous association including the selected data storage resource hosting, within a defined period of time, the data allocation set, the data allocation set not presently hosted by the selected data storage resource;

a second requirement that the workload be previously associated with the selected data storage resource, the previous association including the selected data storage resource hosting, within a defined period of time, any data allocation set associated with the workload; and

a third requirement that the first resource consumer be previously associated with the selected data storage resource, the previous association including the selected data storage resource hosting, within a defined period of time, any data allocation set associated with the first resource consumer.

19. The computer system of claim 17, wherein the first resource consumer has a first inter-system storage priority for accessing data storage resources in the inter-system storage, wherein a second resource consumer dispatched on a second virtual machine hosted on a second compute node in a second integrated system has a second inter-system storage priority for accessing data storage resources in the inter-system storage, wherein the selected data storage resource is concurrently selected to satisfy a second data storage requirement for the second resource consumer, and wherein the method further comprises:

determining, by the storage resource agents associated with the integrated systems, using the inter-system storage priority map, that the first inter-system storage priority is higher than the second inter-system storage priority.

20. The computer system of claim 17, wherein the method further includes the at least one candidate data storage resource satisfying a locality criterion associated with the workload, wherein the at least one candidate data storage resource is associated with a locality measurement based at least in part on the length of a transmission path between the at least one candidate storage resource and the first compute node, wherein the locality measurement satisfies the locality criterion, wherein the selecting the data storage resource further includes the selected data storage resource satisfying a data allocation set activity criterion, wherein the data allocation set activity criterion includes a previous association, wherein the previous association is at least in part based on the data allocation set attributes included in the inter-system storage priority map, and wherein the data allocation set activity criterion is selected from the group consisting of:

a first requirement that the data allocation set has been previously hosted by the selected data storage resource, the data allocation set not presently hosted by the selected data storage resource;

a second requirement that the workload be previously associated with the selected data storage resource, the previous association including the selected data storage resource hosting, within a defined period of time, any data allocation set associated with the workload; and

a third requirement that the first resource consumer be previously associated with the selected data storage resource, the previous association including the selected data storage resource hosting, within a defined period of time, any data allocation set associated with the first resource consumer.