Modifying data structures in distributed file systems

• US 8,972,345 B1
• Filed: 09/27/2006
• Issued: 03/03/2015
• Est. Priority Date: 09/27/2006
• Status: Active Grant

• Current Assignee: Hewlett Packard Enterprise Development LP (Hewlett-Packard Enterprise Company)
• Original Assignee: Hewlett-Packard Development Company, L.P. (HP Inc.)
• Inventors: Aguilera, Marcos K.
• Primary Examiner(s): Alam, Hosain
• Assistant Examiner(s): HARPER, ELIYAH STONE
• Application Number:
  US11/527,877
• Time in Patent Office: 3,079 Days
• Field of Search: 707/104.1, 707/10, 707/100, 707/999.106, 707/999.203, 711/118
• US Class Current: 707/615
• CPC Class Codes: G06F 16/182 Distributed file systems

Abstract

Embodiments include methods, apparatus, and systems for modifying data structures in distributed file systems. One method of software execution includes using minitransactions that include a set of compare items and a set of write items to atomically modify data structures in a distributed file system.

Claims

1. A method of software execution by a computer, comprising:

using, by the computer, minitransactions that include a set of compare items and a set of write items stored in a host to atomically modify data structures in storage nodes of a distributed file system;

validating, by the computer, cached data stored in the host by comparing a version number of an inode stored in the host with a version number of the inode stored in one of the storage nodes; and

allowing, by the computer, the host to write to the data structures with the cached data when the version number of the inode stored in the host matches the version number of the inode stored in the one of the storage nodes.

2. The method of claim 1 further comprising, including with the set of compare and write items (1) a memory identification that identifies a storage node to be atomically modified, (2) an address specifying a location in the storage node, and (3) data.

3. The method of claim 1 further comprising, incrementing the version number of the inode stored in the one of the storage nodes each time data in the inode changes.

4. The method of claim 1, wherein the inode includes at least fields of size, type, and owner.

5. The method of claim 1 further comprising, modifying the data structures by updating an inode area and a data area in the data structures.

6. The method of claim 1 further comprising, reloading the inode stored in the one of the storage nodes into the host when the version number of the inode stored in the host does not match the version number of the inode stored in the one of the storage nodes.

7. The method of claim 1 further comprising, comparing data in the set of compare items for equality against a virtual address specified in the set of compare items such that if a comparison succeeds, then copying data from the set of write items to a storage node in the distributed file system.

8. A non-transitory computer readable medium having instructions for causing a computer to execute a method, comprising:

forming at least one storage node that includes (1) an inode area having attributes of files in the storage node and (2) a data block area storing contents of the files;

adding the storage node to a distributed file system in which a set of cluster nodes share files;

storing contents of the inode area in one of the cluster nodes;

determining that data stored in a cache of the one of the cluster nodes is valid by comparing contents of the inode area stored in the one of cluster nodes with contents of the inode area stored in the storage node;

using the data stored in the cache to perform a write operation in the distributed file system when the data stored in the cache is valid; and

changing attributes of the inode area by atomically modifying attributes of the inode area with a single minitransaction that verifies a version number in the inode area against a cached inode version number.

9. The non-transitory computer readable medium of claim 8 further comprising, forming the storage node with (3) a bitmap area that indicates which data blocks are being used in the storage node, and (4) a superblock region that includes static information about a volume of the storage node.

10. The non-transitory computer readable medium of claim 8 further comprising, using minitransactions that include a set of compare items and a set of write items to atomically modify a data structure of the distributed file system.

11. The non-transitory computer readable medium of claim 8 further comprising:

issuing minitransactions to check an iversion of a cache inode, check free status of a new storage block, update iversion and dversion numbers of an inode, and populate the new storage block with data.

12. The non-transitory computer readable medium of claim 8 further comprising, storing an inode, a chaining list for the inode, and file contents in a same storage node.

13. The non-transitory computer readable medium of claim 8 further comprising, atomically marking the data block in the bitmap area as not available for write operations, and adding the data block to a chaining list in the inode area.

14. The non-transitory computer readable medium of claim 8 further comprising creating a new file in the distributed file system by (1) allocating a new inode, (2) allocating a new directory entry for the new file, (3) populating the new directory entry, and (4) issuing a minitransaction to verify that the new file does not already exist in the distributed file system.

15. A computer system, comprising:

a memory for storing an algorithm; and

a processor for executing the algorithm to;

transmit minitransactions between cluster nodes and storage nodes in a distributed file system, wherein the minitransactions include a set of compare items and a set of write items that atomically modify data structures in the distributed file system, and the minitransactions compare data of two inodes to validate contents of cached data stored in the cluster nodes by comparing a version of an inode stored in a host of the distributed file system with a version number of the inode stored in one of the storage nodes to allow the cluster nodes to modify the data structures with the contents of the cached data when the version number of the inode stored in the host matches the version number of the inode stored in the one of the storage nodes.

16. The computer system of claim 15, wherein the processor further executes the algorithm to implement a network-attached storage (NAS) service with multiple heads for each one of plural file systems.

17. The computer system of claim 15, wherein a minitransaction aborts if either (1) comparisons between the compare items and cached data do not match or (2) another minitransaction attempts to concurrently execute a conflicting transaction to a file system data structure.

18. The computer system of claim 15, wherein the storage nodes include (1) a superblock region for storing static information about the file system, (2) a bitmap region for indicating if blocks in a data region are available, (3) an inode list for storing contents of an inode, and (4) a file data region for storing data content of inodes and chaining lists.

19. The computer system of claim 15, wherein the minitransactions protect two different hosts from performing conflicting updates to the storage nodes by (1) determining if an inode and directory entry are free, (2) marking the inode and directory entry as being used, and (3) populating the inode and directory entry.