Efficient data management in a cluster file system

• US 20070179981A1
• Filed: 01/31/2006
• Published: 08/02/2007
• Est. Priority Date: 01/31/2006
• Status: Abandoned Application

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Vincent, Pradeep
• Application Number:
  US11/343,305
• Publication Number:
  US 20070179981A1
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 3/0605 by facilitating the interaction with a user or administrator
• CPC Class Codes: G06F 3/0643 Management of files
• CPC Class Codes: G06F 3/067 Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]

Abstract

Methods and systems manage datasets in a cluster file system. A request is received from a client to perform a file system operation on a specified dataset stored in one of a plurality of nodes in a cluster. The specified dataset is retrieved from a first node through a backbone switch and stored in a cache in a second node. The requested file system operation is performed on the specified dataset and, upon completion of the requested operation, metadata is modified to indicate that the specified dataset is stored in the second node. The specified dataset is not returned through the backbone switch to the first node.

Claims

1. A cluster file system accessible to clients through a network, comprising:

a plurality of file system nodes in a cluster, including a first node and a second node;

a backbone switch interconnecting the first node and the second node;

a metadata structure identifying the node on which datasets are stored; and

the first node comprising a first cache and a dataset controller configured to, if a specified dataset is stored on the second node;

receive a request from a client to perform a file system operation on the specified dataset;

access the metadata structure to determine the node on which the specified dataset is stored;

retrieve through the backbone switch from the second node that a first portion of the specified dataset to which the file system operation is directed and leave a remainder portion of the specified dataset stored in the second node;

store the retrieved first portion in the first cache; and

upon completion of the file system operation, modify the metadata structure to indicate that at least the first portion of the specified dataset is stored in the first node, whereby the first portion is not returned through the backbone switch to the second node.

2. The system of claim 1, wherein:

the first node and the second node each comprise a virtual front-end server and a virtual back-end server; and

the metadata structure identifies the virtual server and the node on which datasets are stored.

3. The system of claim 1, wherein the dataset controller is further configured to:

upon completion of the file system operation, retrieve through the backbone switch the remainder portion of the specified dataset;

modify the metadata structure to indicate that the entire specified dataset is stored in the first node; and

store the entire specified dataset in the first node.

4. The system of claim 1, wherein the dataset controller is further configured to:

divide the specified dataset into a plurality of subsets, each having a size wherein the first portion and the remainder portion of the specified dataset each comprise at least one subset;

modify the metadata structure to indicate that subsets comprising the first portion are stored in the first node and subsets comprising the remainder portion are stored in the second node; and

store the subsets of the first portion in the first node.

5. The system of claim 4, wherein the dataset controller is further configured to, during a time in which the backbone switch is at a reduced level of activity:

transfer the subsets comprising the first portion from the first node through the backbone switch to the second node;

combine the at least one subset of the first portion with the at least one subset of the remainder portion to reform the specified dataset;

store the reformed specified dataset in the second node; and

modify the metadata structure to indicate that the specified dataset is stored in the second node.

6. The system of claim 1, wherein the dataset controller is further configured to, during a time in which the backbone switch is at a reduced level of activity:

transfer the first portion from the second node through the backbone switch to the first node;

combine the first portion with the remainder portion to reform the specified dataset;

store the reformed specified dataset in the first node; and

modify the metadata structure to indicate that the specified dataset is stored in the first node.

7. A method for managing datasets in a cluster file system, comprising:

receiving a request from a client to perform a file system operation on a specified dataset stored in one of a plurality of nodes in a cluster;

retrieving the specified dataset from a first node through a backbone switch;

storing the retrieved specified dataset in a cache in a second node;

performing the requested file system operation on the specified dataset; and

upon completion of the requested operation, modifying metadata to indicate that the specified dataset is stored in the second node, whereby the specified dataset is not returned through the backbone switch to the first node.

8. The method of claim 7, wherein:

the file system operation is requested to be performed on a first portion of the specified dataset; and

retrieving the specified dataset comprises retrieving the first portion through the backbone switch whereby a second portion remains stored in the first node.

9. The method of claim 8, wherein modifying the metadata comprises modifying the metadata to indicate that the first portion of the specified dataset is stored in the second node and the second portion is stored in the first node.

10. The method of claim 8, wherein:

the method further comprises dividing the specified dataset into a plurality of subsets wherein the first portion and the second portion each comprise at least one subset; and

modifying the metadata comprises modifying the metadata to indicate that subsets comprising the first portion are stored in the second node and subsets comprising the second portion are stored in the first node.

11. The method of claim 10, further comprising, during a time in which the backbone switch is at a reduced level of activity:

transferring the at least one subset of the first portion from the second node through the backbone switch to the first node;

combining the at least one subset of the first portion with the at least one subset of the second portion to reform the specified dataset;

storing the reformed specified dataset in the first node; and

modifying the metadata structure to indicate that the specified dataset is stored in the first node.

12. The method of claim 7, further comprising, during a time in which the backbone switch is at a reduced level of activity:

transferring the first portion from the second node through the backbone switch to the first node;

combining the first portion with the second portion to reform the specified dataset;

storing the reformed specified dataset in the first node; and

modifying the metadata structure to indicate that the specified dataset is stored in the first node.

13. A computer program product of a computer readable medium usable with a programmable computer, the computer program product having computer-readable code embodied therein for managing datasets in a cluster file system, the computer-readable code comprising instructions for:

receiving a request from a client to perform a file system operation on a specified dataset stored in one of a plurality of nodes in a cluster;

retrieving the specified dataset from a first node through a backbone switch;

storing the retrieved specified dataset in a cache in a second node;

performing the requested file system operation on the specified dataset; and

upon completion of the requested operation, modifying metadata to indicate that the specified dataset is stored in the second node, whereby the specified dataset is not returned through the backbone switch to the first node.

14. The computer program product of claim 13, wherein:

the file system operation is requested to be performed on a first portion of the specified dataset; and

the instructions for retrieving the specified dataset comprise instructions for retrieving the first portion through the backbone switch whereby a second portion remains stored in the first node.

15. The computer program product of claim 14, wherein:

the instructions further comprise instructions for dividing the specified dataset into a plurality of subsets wherein the first portion and the second portion each comprise at least one subset; and

the instructions for modifying the metadata comprise instructions for modifying the metadata to indicate that subsets comprising the first portion are stored in the second node and subsets comprising the second portion are stored in the first node.

16. The computer program product of claim 15, further comprising instructions for, during a time in which the backbone switch is at a reduced level of activity:

transferring the at least one subset of the first portion from the second node through the backbone switch to the first node;

combining the at least one subset of the first portion with the at least one subset of the second portion to reform the specified dataset;

storing the reformed specified dataset in the first node; and

modifying the metadata structure to indicate that the specified dataset is stored in the first node.

17. The computer program product of claim 13, further comprising instructions for, during a time in which the backbone switch is at a reduced level of activity:

transferring the first portion from the second node through the backbone switch to the first node;

combining the first portion with the second portion to reform the specified dataset;

storing the reformed specified dataset in the first node; and

modifying the metadata structure to indicate that the specified dataset is stored in the first node.

18. A file system node in a multi-node cluster file system, comprising:

means for interconnecting the node to at least a second node through a backbone switch;

a cache;

a metadata structure identifying the node on which datasets are stored;

means for receiving a request from a client to perform a file system operation on a specified dataset;

means for accessing the metadata structure to determine the node on which the specified dataset is stored;

if the specified dataset is stored on the second node, means for retrieving through the backbone switch that first portion of the specified dataset to which the file system operation is directed and leaving a remainder portion of the specified dataset stored in the second node;

means for storing the retrieved first portion in the first cache; and

means for modifying the metadata structure upon completion of the file system operation to indicate that at least the first portion of the specified dataset is stored in the first node, whereby the first portion is not returned through the backbone switch to the second node.

19. The file system node of claim 18, further comprising:

means for retrieving through the backbone switch the remainder portion of the specified dataset upon completion of the file system operation;

modifying the metadata structure to indicate that the entire specified dataset is stored in the first node; and

storing the entire specified dataset in the first node.

20. The file system node of claim 18, further comprising:

means for dividing the specified dataset into a plurality of subsets, each having a size wherein the first portion and the remainder portion of the specified dataset each comprise at least one subset;

means for modifying the metadata structure to indicate that subsets comprising the first portion are stored in the first node and subsets comprising the remainder portion are stored in the second node; and

means for storing the subsets of the first portion in the first node.

21. The file system node of claim 18, further comprising:

means for transferring the first portion from the second node through the backbone switch to the first node during a time in which the backbone switch is at a reduced level of activity;

means for combining the first portion with the remainder portion to reform the specified dataset;

means for storing the reformed specified dataset in the first node; and

means for modifying the metadata structure to indicate that the specified dataset is stored in the first node.