Distributed storage medium management for heterogeneous storage media in high availability clusters

• US 9,063,939 B2
• Filed: 11/03/2011
• Issued: 06/23/2015
• Est. Priority Date: 11/03/2011
• Status: Active Grant

• Current Assignee: Zettaset Incorporated
• Original Assignee: Zettaset Incorporated
• Inventors: Dalton, Michael W.
• Primary Examiner(s): Shah, Sanjiv
• Assistant Examiner(s): Chappell, Daniel C
• Application Number:
  US13/373,142
• Publication Number:
  US 20130117225A1
• Time in Patent Office: 1,328 Days
• Field of Search: 711/148
• US Class Current: 1/1
• CPC Class Codes: G06F 16/119 Details of migration of file systems migration mechanisms in storage systems G06F3/0647
• CPC Class Codes: G06F 9/5083 Techniques for rebalancing the load in a distributed system

Abstract

A high availability cluster and method of storage medium management in such high availability cluster. A number k of nodes belonging to the cluster, where k≧2 and includes a master node, are provisioned with high-access-rate and low-access-rate storage media. A file is written to the high-access-rate medium of a serving node selected from among k−1 nodes excluding the master node. The file is also written to low-access-rate medium of each of k−2 nodes excluding the master node and the serving node. A distributed storage medium management protocol supervises file migration from the low-access-rate medium of a back-up node to the high-access-rate medium of the back-up node upon disruption of file availability on the serving node. File rebalancing relies on parameters including popularity, write- and read-requests, capacity, processing load or cost.

Claims

1. A method of storage medium management in nodes of a high availability cluster that serves one or more complete files to a network client, said method comprising:

a) provisioning a number k of said nodes, where k≧

2 and includes a master node, each of said nodes having a high-access-rate non-transitory storage medium and a low-access-rate non-transitory storage medium;

b) selecting a file from said one or more complete files, and assigning an individual node from among k−

1 nodes excluding said master node, to be the serving node of said file and writing said file to said high-access-rate non-transitory storage medium of said serving node;

c) writing said file to said low-access-rate non-transitory storage medium of each of k−

2 nodes excluding said master node and said serving node;

d) monitoring, with a storage medium management protocol distributed among said k nodes, an availability to said network client of said file on said high-access-rate non-transitory storage medium of said serving node;

e) migrating said file from said low-access-rate non-transitory storage medium of a back-up node selected from among said k−

1 nodes to said high-access-rate non-transitory storage medium of said back-up node upon disruption of said availability on said serving node, and assigning said back-up node to be the new serving node;

f) updating on said master node a metadata about said file and a path to said file, whereby said master node can direct future requests by said network client to said file via said updated path;

g) and re-distributing said file to said high-access-rate non-transitory storage medium of at least one supplementary serving node selected from among said k−

1 nodes, wherein said re-distributing further comprises breaking said file into segments and re-distributing said segments to said at least one supplementary serving node;

thereby providing for high availability of said file to said network client.

2. The method of claim 1, further comprising:

a) monitoring a popularity of said file on said high-access-rate non-transitory storage medium of said serving node;

b) removing said file from said high-access-rate non-transitory storage medium when said popularity of said file drops below a predetermined threshold.

3. The method of claim 2, further comprising migrating a second file from said low-access-rate non-transitory storage medium to said high-access-rate non-transitory storage medium of said serving node when a popularity of said second file exceeds said predetermined threshold.

4. The method of claim 1, further comprising re-balancing the distribution of said one or more complete files among said high-access-rate non-transitory storage medium and said low-access-rate non-transitory storage medium of at least one among said k−

1 nodes.

5. The method of claim 4, wherein said re-balancing is based on a parameter selected from the group consisting of a number of write-requests, a number of read-requests, a capacity of said high-access-rate non-transitory storage medium, a processing load and a processing cost.

6. The method of claim 5, further comprising re-distributing said file to said high-access-rate non-transitory storage medium of a supplementary serving node selected from among said k−

1 nodes when said capacity of said high-access-rate non-transitory storage medium of said serving node is exceeded.

7. The method of claim 5, wherein said parameter is a cluster-wide parameter computed jointly for said k−

1 nodes.

8. The method of claim 1, further comprising mirroring said file between said low-access-rate non-transitory storage medium of said serving node and an imaging node of said high availability cluster.

9. The method of claim 8, wherein said imaging node is said back-up node.

10. The method of claim 1, wherein said storage medium management protocol is incorporated in ZooKeeper.

11. The method of claim 1, wherein said high-access-rate non-transitory storage medium comprises a flash storage drive.

12. The method of claim 1, wherein said low-access-rate non-transitory storage medium comprises a hard disk drive.

13. A high availability cluster of nodes employing heterogeneous storage media in serving one or more complete files to a network client, said high availability cluster comprising:

a) a number k of said nodes, where k≧

2 and includes a master node, and where a file is selected from said one or more complete files, and one of k−

1 nodes excluding said master node is assigned to be the serving node for serving said file to said network client;

b) a high-access-rate non-transitory storage medium associated with each of said k nodes, wherein said file is written to said high-access-rate non-transitory storage medium of said serving node;

c) a low-access rate non-transitory storage medium associated with each of said k nodes, wherein said file is written to said low-access-rate non-transitory storage medium of each of k−

2 nodes excluding said master node and said serving node;

d) a distributed storage medium management protocol residing on said k nodes for monitoring an availability to said network client of said file on said high-access-rate non-transitory storage medium of said serving node, and for migrating said file from said low-access-rate non-transitory storage medium of a back-up node selected among said k nodes to said high-access-rate non-transitory storage medium of said back-up node upon disruption of said availability on said serving node, and assigning said back-up node to be the new serving node of said file, said distributed storage medium management protocol further providing for updating on said master node a metadata about said file and a path to said file, whereby said master node can direct future requests by said network client to said file via said updated path and said distributed storage medium management protocol further comprising re-distributing said file to said high-access-rate non-transitory storage medium of at least one supplementary serving node selected from among said k−

1 nodes, wherein said re-distributing further comprises breaking said file into segments and re-distributing said segments to said at least one supplementary serving node;

thereby providing for high availability of said file to said network client.

14. The high availability cluster of claim 13, wherein said high-access-rate non-transitory storage medium comprises a flash storage drive.

15. The high availability cluster of claim 13, wherein said low-access-rate non-transitory storage medium comprises a hard disk drive.

16. The high availability cluster of claim 13, further comprising a re-balancing module associated with each of said k nodes.

17. The high availability cluster of claim 16, wherein said re-balancing module is configured to monitor said availability of said file.

18. The high availability cluster of claim 16, wherein said re-balancing module is configured to monitor a popularity of said file.

19. A method of storage medium management in nodes of a high availability cluster that serves one or more complete files to a network client, said method comprising:

a) provisioning a number k of said nodes, where k≧

2 and includes a master node, each of said nodes having a high-access-rate non-transitory storage medium and a low-access-rate storage medium;

b) selecting a file from said one or more complete files, and assigning an individual node from among k−

1 nodes excluding said master node, to be the serving node of said file and writing said file to said high-access-rate non-transitory storage medium of said serving node;

c) writing said file to said low-access-rate non-transitory storage medium of each of k−

2 nodes excluding said master node and said serving node;

d) monitoring, with a storage medium management protocol distributed among said k nodes, an availability to said network client of said file on said high-access-rate non-transitory storage medium of said serving node;

e) migrating said file from said low-access-rate non-transitory storage medium of a back-up node selected from among said k−

1 nodes to said high-access-rate non-transitory storage medium of said back-up node upon disruption of said availability on said serving node, and assigning said back-up node to be the new serving node of said file;

f) updating on said master node a metadata about said file and a path to said file, whereby said master node can direct future requests by said network client to said file via said updated path;

g) re-balancing the distribution of said one or more complete files among said high-access-rate non-transitory storage medium and said low-access-rate non-transitory storage medium of at least one among said k−

1 nodes;

h) basing said re-balancing on a parameter selected from the group consisting of a number of write-requests, a number of read-requests, a capacity of said high-access-rate non-transitory storage medium, a processing load and a processing cost;

i) and re-distributing said file to said high-access-rate non-transitory storage medium of at least one supplementary serving node selected from among said k−

1 nodes, wherein said re-distributing further comprises breaking said file into segments and re-distributing said segments to said at least one supplementary serving node;

thereby providing for high availability of said file to said network client.

20. A method of storage medium management in nodes of a high availability cluster that serves one or more complete files to a network client, said method comprising:

a) provisioning a number k of said nodes, where k≧

2 and includes a master node, each of said nodes having a high-access-rate non-transitory storage medium and a low-access-rate non-transitory storage medium;

b) selecting each file from said one or more complete files, and assigning an individual node from among k−

1 nodes excluding said master node, to be the serving node of said file and writing said file to said high-access-rate non-transitory storage medium of said serving node;

c) writing said file to said low-access-rate non-transitory storage medium of each of k−

2 nodes excluding said master node and said serving node;

d) monitoring, with a storage medium management protocol distributed among said k nodes, an availability to said network client of said file on said high-access-rate non-transitory storage medium of said serving node;

e) migrating said file from said low-access-rate non-transitory storage medium of a back-up node selected from among said k−

1 nodes to said high-access-rate non-transitory storage medium of said back-up node upon disruption of said availability on said serving node, and assigning said back-up node to be the new serving node of said file;

f) updating on said master node a metadata about said file and a path to said file, whereby said master node can direct future requests by said network client to said file via said updated path;

g) re-balancing the distribution of said one or more complete files among said high-access-rate non-transitory storage medium and said low-access-rate storage medium of at least one among said k−

1 nodes;

h) basing said re-balancing on a parameter selected from the group consisting of a number of write-requests, a number of read-requests, a capacity of said high-access-rate non-transitory storage medium, a processing load and a processing cost;

i) mirroring said file between said low-access-rate non-transitory storage medium of said serving node and an imaging node of said high availability cluster, and selecting said imaging node to be said back-up node;

j) and re-distributing said file to said high-access-rate non-transitory storage medium of at least one supplementary serving node selected from among said k−

1 nodes, wherein said re-distributing further comprises breaking said file into segments and re-distributing said segments to said at least one supplementary serving node;

thereby providing for high availability of said file to said network client.