Methods systems, and computer program products for managing quality-of-service associated with storage shared by computing grids and clusters with a plurality of nodes

• US 8,473,566 B1
• Filed: 06/30/2006
• Issued: 06/25/2013
• Est. Priority Date: 06/30/2006
• Status: Active Grant

• Current Assignee: Emc IP Holding Company LLC (Dell Technologies Inc.)
• Original Assignee: EMC Corporation (Dell Technologies Inc.)
• Inventors: Cardente, John T., Compton, James T., Evans, Michael B., Fridella, Stephen A., Gupta, Uday K.
• Primary Examiner(s): Hamza, Faruk
• Assistant Examiner(s): OH, ANDREW CHUNG SUK
• Application Number:
  US11/479,939
• Time in Patent Office: 2,552 Days
• Field of Search: None
• US Class Current: 709/216
• CPC Class Codes: G06F 3/061 Improving I/O performance
• CPC Class Codes: G06F 3/0647 Migration mechanisms
• CPC Class Codes: G06F 3/0653 Monitoring storage devices or systems
• CPC Class Codes: G06F 3/067 Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
• CPC Class Codes: H04L 67/1097 for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]

Abstract

Methods, systems, and computer program products for managing quality of service for shared storage accessed by a plurality of nodes are disclosed. According to one method, at a node of a plurality of nodes sharing access to a first disk array, quality-of-service (QoS) metadata associated with a portion of the disk array are requested from a server, wherein the QoS metadata includes at least one performance criterion for the portion of the disk array. I/O performance is monitored for the portion of the disk array in response to receiving the QoS metadata. The monitored I/O performance is compared with the QoS metadata. Based upon the comparison of the monitored I/O performance with the QoS metadata, it is determined whether a QoS violation has occurred. In response to determining that a QoS violation has occurred, the server is requested to migrate data associated with the portion of the disk array.

Claims

1. A method for managing quality of service for shared storage accessed by a plurality of nodes, the method comprising:

at a client node of a plurality of nodes separate from and sharing access to a first disk array;

requesting, from a server, first quality-of-service (QoS) metadata associated with a first portion of the first disk array, wherein the first QoS metadata includes at least one performance criterion for the first portion of the first disk array;

obtaining, from the server, a map including physical disk access information for the first portion of the first disk array, wherein the physical disk access information includes a physical address of a location within a disk of the disk array;

receiving the first QoS metadata;

monitoring I/O performance for the first portion of the first disk array, wherein monitoring I/O performance for the first portion of the first disk array includes monitoring the first portion of the first disk array directly without interaction with the server using the physical disk access information included within the map;

comparing the monitored I/O performance with the first QoS metadata;

determining, without interaction with the server and based upon the comparison of the monitored I/O performance with the first QoS metadata, whether a QoS violation has occurred; and

in response to determining that a QoS violation has occurred, effecting migration of data associated with the first portion of the first disk array, wherein each of the plurality of client nodes is adapted to send a request for the first QoS metadata to the server and wherein the request includes, for a sending node, identifying information for the sending node and wherein the server is adapted to store the identifying information to create an association between the sending node and the first portion of the first disk array to indicate that the sending node has an I/O mapping to the first portion of the first disk array.

2. The method of claim 1 wherein determining whether a QoS violation has occurred includes determining whether at least a portion of the data associated with the first portion of the first disk array resides on disparately performing storage devices associated with the first disk array.

3. The method of claim 1 wherein determining whether a QoS violation has occurred includes determining whether at least a first portion of the data associated with the first portion of the first disk array is accessed more frequently than a second portion of the data.

4. The method of claim 1 wherein determining whether a QoS violation has occurred includes determining whether at least a portion of the data associated with the first portion of the first disk array is stored on storage devices with different cost attributes.

5. The method of claim 1 wherein determining whether a QoS violation has occurred includes determining whether at least a portion of the data associated with the first portion of the first disk array is stored on storage devices with different security attributes.

6. The method of claim 1 wherein effecting migration of data associated with the first portion of the disk array includes using the server to copy the data associated with the first portion of the first disk array to a second portion of the first disk array.

7. The method of claim 1 wherein effecting migration of data associated with the first portion of the disk array includes using the server to copy the data associated with the first portion of the first disk array to a first portion of a second disk array.

8. The method of claim 1 wherein effecting migration of data associated with the first portion of the disk array includes using the client node to copy the data associated with the first portion of the disk array.

9. The method of claim 8, comprising receiving, at the client node, as part of a request from the server that the client node migrate the data, a source location including first physical disk access information and a destination location including second physical disk access information and wherein effecting migration of data includes copying the data from the source location using the first physical disk access information to the destination location using the second physical disk access information.

10. The method of claim 8 comprising receiving, at the client node, as part of a request from the server that the client node migrate the data, a source location including first physical disk access information, a destination location including second physical disk access information, and second QoS metadata associated with the destination location and wherein the effecting migration of data includes copying the data from the source location using the first physical disk access information to the destination location using the second physical disk access information and using the second QoS metadata to monitor I/O performance for the destination location.

11. The method of claim 1 wherein the at least one performance criterion for the first portion of the first disk array includes at least one of a maximum average response time for read operations, a maximum average response time for write operations, a maximum response time for read operations, a maximum response time for write operations, a minimum throughput for read operations per unit time, and a minimum throughput for write operations per unit time.

12. The method of claim 1 comprising maintaining a variance tolerance for the at least one performance criterion and calculating a weighted average for a plurality of monitored I/O performance values and wherein comparing the monitored I/O performance with the first QoS metadata includes comparing the weighted average with the at least one performance criterion and wherein determining whether a QoS violation has occurred includes identifying a variance associated with the weighted average that is larger than the variance tolerance.

13. The method of claim 1 wherein the at least one performance criterion comprises a threshold and wherein comparing the monitored I/O performance with the first QoS metadata includes comparing the monitored I/O performance with the threshold and wherein determining whether a QoS violation has occurred includes determining whether the threshold has been exceeded.

14. The method of claim 1 comprising maintaining a fuzzy logic algorithm based upon the at least one performance criterion and wherein comparing the monitored I/O performance with the first QoS metadata includes executing the fuzzy logic algorithm on the monitored I/O performance and wherein determining whether a QoS violation has occurred includes determining whether the result of executing the fuzzy logic algorithm indicates that the at least one performance criterion is in danger of being exceeded.

15. A method for managing quality of service for shared storage accessed by a plurality of nodes, the method comprising:

at a server serving a plurality of client nodes separate from and sharing access to a first disk array;

receiving a request for quality-of-service (QoS) metadata from a first client node of the plurality of nodes, wherein the request for QoS metadata is associated with a first portion of the first disk array;

providing, to the first client node, a map including physical disk access information for the first portion of the first disk array, wherein the physical disk access information includes a physical address of a location within a disk of the disk array;

in response to receiving the first request for QoS metadata, sending, to the first client node, first QoS metadata including at least one performance criterion for the first portion of the first disk array, wherein the first client node monitors the first portion of the first disk array directly without interaction with the server using the physical disk access information included within the map and uses the first QoS metadata in determining, without interaction with the server, whether to initiate a request for data migration;

receiving a request from the first client node to migrate data associated with the first portion of the first disk array; and

effecting migration of the data associated with the first portion of the first disk array, wherein each of the plurality of client nodes is adapted to send a request for the first QoS metadata to the server and wherein the request includes, for a sending node, identifying information for the sending node and wherein the server is adapted to store the identifying information to create an association between the sending node and the first portion of the first disk array to indicate that the sending node has an I/O mapping to the first portion of the first disk array.

16. The method of claim 15 wherein the first request for QoS metadata includes identifying information for the first client node of the plurality of client nodes and wherein the method further comprises storing the identifying information to create an association between the first client node and the first portion of the first disk array.

17. The method of claim 15 comprising sending, in response to receiving the request to migrate data associated with the portion of the first disk array, a revoke message to nodes within the plurality of client nodes with input/output (I/O) associations with the first portion of the first disk array.

18. The method of claim 17 wherein effecting migration of the data includes, in response to determining to delegate migration to a node of the plurality of nodes, selecting a node from the plurality of client nodes to migrate the data associated with the first portion of the first disk array.

19. The method of claim 18 wherein effecting migration of the data includes, in response to selecting a node from the plurality of nodes to migrate the data, sending a migrate request to the selected node including first physical disk access information for the first portion of the first disk array and second physical disk access information identifying a second portion of the first disk array as a destination location for data migration.

20. The method of claim 18 wherein effecting migration of the data includes, in response to selecting a node from the plurality of client nodes to migrate the data, sending a migrate request to the selected node including first physical disk access information for the first portion of the first disk array and second physical disk access information identifying a first portion of a second disk array as a destination location for data migration.

21. The method of claim 18 wherein effecting migration of the data includes, in response to selecting the first client node from the plurality of client nodes to migrate the data, sending a migrate request to the first client node including first physical disk access information for the first portion of the first disk array, second physical disk access information identifying a second portion of the first disk array as a destination location for data migration, and second QoS metadata for the second portion of the first disk array.

22. The method of claim 18 wherein effecting migration of the data includes, in response to selecting the first client node from the plurality of client nodes to migrate the data, sending a migrate request to the first client node including first physical disk access information for the first portion of the first disk array, second physical disk access information identifying a first portion of a second disk array as a destination location for data migration, and second QoS metadata for the first portion of the second disk array.

23. The method of claim 15 wherein effecting migration of the data associated with the first portion of the first disk array includes determining whether to migrate the data at the server or to delegate migration to a node within the plurality of client nodes.

24. The method of claim 23 wherein effecting migration of the data includes, in response to determining to migrate the data at the server, using the server to copy the data from the first portion of the first disk array to a second portion of the first disk array.

25. The method of claim 23 wherein effecting migration of the data includes, in response to determining to migrate the data at the server, using the server to copy the data from the first portion of the first disk array to a first portion of a second disk array.

26. The method of claim 15 wherein the at least one performance criterion for the portion includes at least one of a maximum average response time for read operations, a maximum average response time for write operations, a maximum response time for read operations, a maximum response time for write operations, a minimum throughput of read operations per unit time, and a minimum throughput of write operations per unit time.

27. A system for managing quality of service for shared storage accessed by a plurality of nodes, the system comprising:

a first disk array; and

a server for providing first quality-of-service (QoS) metadata associated with a first portion of the first disk array, wherein the first QoS metadata includes at least one performance criterion for the first portion of the first disk array, and a map including physical disk access information for the first portion of the first disk array, wherein the physical disk access information includes a physical address of a location within a disk of the disk array; and

a plurality of client nodes for detecting QoS violations related to the first portion of the first disk array, each of the plurality of client nodes being separate from the first disk array and adapted to;

request, from the server, the first QoS metadata;

obtain, from the server, the map;

receive the first QoS metadata;

monitor input/output (I/O) performance for the first portion of the first disk array, wherein monitor I/O performance for the first portion of the first disk array includes monitoring the first portion of the first disk array directly without interaction with the server using the physical disk access information included within the map;

compare the monitored I/O performance with the first QoS metadata;

and determine, without interaction with the server and based upon the comparison of the monitored I/O performance with the first QoS metadata, whether a QoS violation has occurred, wherein each of the plurality of client nodes is adapted to send a request for the first QoS metadata to the server and wherein the request includes, for a sending node, identifying information for the sending node and wherein the server is adapted to store the identifying information to create an association between the sending node and the first portion of the first disk array to indicate that the sending node has an I/O mapping to the first portion of the first disk array.

28. The system of claim 27 wherein each of the plurality of client nodes is adapted to request, in response to determining that a QoS violation has occurred, that the server migrate data associated with the first portion of the first disk array.

29. The system of claim 28 wherein the server is adapted to send, in response to receiving a request to migrate data associated with the first portion of the first disk array from the sending node, a revoke message to nodes of the plurality of nodes with input/output (I/O) associations with the first portion of the first disk array.

30. The system of claim 28 wherein the server is adapted to determine, in response to receiving a request that the server migrate data associated with the first portion of the first disk array from a requesting node, whether to migrate the data at the server or to select a node from the plurality of nodes to migrate the data.

31. The system of claim 30 wherein the server is adapted to copy, in response to determining to migrate the data at the server, the data from the first portion of the first disk array to a second portion of the first disk array.

32. The system of claim 30 wherein the server is adapted to copy, in response to determining to migrate the data at the server, the data from the first portion of the first disk array to a first portion of a second disk array.

33. The system of claim 30 wherein the server is adapted to send, in response to selecting a node of the plurality of nodes to migrate the data, a migrate request to the selected node including first physical disk access information for the first portion of the first disk array and second physical disk access information identifying a second portion of the first disk array as a destination location for data migration.

34. The system of claim 30 wherein the server is adapted to send, in response to selecting a node of the plurality of nodes to migrate the data, a migrate request to the selected node including first physical disk access information for the first portion of the first disk array and second physical disk access information identifying a first portion of a second disk array as a destination location for data migration.

35. The system of claim 30 wherein the server is adapted to send, in response to selecting the requesting node to migrate the data, a migrate request to the requesting node including first physical disk access information for the first portion of the first disk array, second physical disk access information identifying a second portion of the first disk array as a destination location for data migration, and second QoS metadata for the second portion of the first disk array.

36. The system of claim 30 wherein the server is adapted to send, in response to selecting the requesting node to migrate the data, a migrate request to the requesting node including first physical disk access information for the first portion of the first disk array, second physical disk access information identifying a first portion of a second disk array as a destination location for data migration, and second QoS metadata for the first portion of the second disk array.

37. The system of claim 27 wherein, in determining whether a QoS violation has occurred, each client node is adapted to determine whether at least a portion of the data associated with the first portion of the first disk array resides on disparately performing storage devices associated with the first disk array.

38. The system of claim 27 wherein, in determining whether a QoS violation has occurred, each client node is adapted to determine whether at least a first portion of the data associated with the first portion of the first disk array is accessed more frequently than a second portion of the data.

39. The system of claim 27 wherein, in determining whether a QoS violation has occurred, each client node is adapted to determine whether at least a portion of the data associated with the first portion of the first disk array is stored on storage devices with different cost attributes.

40. The system of claim 27 wherein, in determining whether a QoS violation has occurred, each client node is adapted to determine whether at least a portion of the data associated with the first portion of the first disk array is stored on storage devices with different security attributes.

41. The system of claim 27 wherein the at least one performance criterion for the first portion of the first disk array includes at least one of a maximum average response time for read operations, a maximum average response time for write operations, a maximum response time for read operations, a maximum response time for write operations, a minimum throughput for read operations per unit time, and a minimum throughput for write operations per unit time.

42. The system of claim 27 wherein each of the plurality of client nodes is adapted to store a variance tolerance for the at least one performance criterion, to calculate a weighted average for a plurality of monitored I/O performance values, to compare the weighted average with the at least one performance criterion and wherein, in determining whether a QoS violation has occurred, each client node is adapted to identify a variance associated with the weighted average that is larger than the variance tolerance.

43. The system of claim 27 wherein each of the plurality of client nodes is adapted to use the at least one performance criterion as a threshold and, in comparing the monitored I/O performance with the first QoS metadata, each client node is adapted to compare the monitored I/O performance with the threshold and wherein, in determining whether a QoS violation has occurred, each client node is adapted to determine whether the threshold has been exceeded.

44. The system of claim 27 wherein each of the plurality of client nodes is adapted to maintain a fuzzy logic algorithm based upon the at least one performance criterion and wherein, in comparing the monitored I/O performance with the first QoS metadata, each client node is adapted to execute the fuzzy logic algorithm on the monitored I/O performance and wherein, in determining whether a QoS violation has occurred, each client node is adapted to determine whether the result of executing the fuzzy logic algorithm indicates that the at least one performance criterion is in danger of being exceeded.

45. A non-transitory computer readable medium having stored thereon executable instructions that when executed by the processor of a computer control the computer to perform steps comprising:

at a client node of a plurality of client nodes separate from and sharing access to a first disk array;

requesting first quality-of-service (QoS) metadata associated with a first portion of the first disk array from a server, wherein the QoS metadata includes at least one performance criterion for the first portion of the first disk array;

obtaining, from the server, a map including physical disk access information for the first portion of the first disk array, wherein the physical disk access information includes a physical address of a location within a disk of the disk array;

monitoring I/O performance for the first portion of the first disk array, wherein monitoring I/O performance for the first portion of the first disk array includes monitoring the first portion of the first disk array directly without interaction with the server using the physical disk access information included within the map;

comparing the monitored I/O performance with the QoS metadata;

determining, without interaction with the server and based upon the comparison of the monitored I/O performance with the QoS metadata, whether a QoS violation has occurred; and

in response to determining that a QoS violation has occurred, effecting migration of data associated with the first portion of the first disk array, wherein each of the plurality of client nodes is adapted to send a request for the first QoS metadata to the server and wherein the request includes, for a sending node, identifying information for the sending node and wherein the server is adapted to store the identifying information to create an association between the sending node and the first portion of the first disk array to indicate that the sending node has an I/O mapping to the first portion of the first disk array.