Namespace consistency for a wide-area file system

• US 8,311,980 B2
• Filed: 12/01/2003
• Issued: 11/13/2012
• Est. Priority Date: 12/09/2002
• Status: Active Grant

• Current Assignee: Hewlett Packard Enterprise Development LP (Hewlett-Packard Enterprise Company)
• Original Assignee: Hewlett-Packard Development Company, L.P. (HP Inc.)
• Inventors: Saito, Yasushi, Karamanolis, Christos
• Primary Examiner(s): LEROUX, ETIENNE PIERRE
• Application Number:
  US10/726,041
• Publication Number:
  US 20040172421A1
• Time in Patent Office: 3,270 Days
• Field of Search: 707/100, 707/204
• US Class Current: 707/610
• CPC Class Codes: G06F 16/1834 implemented based on peer-to-peer networks, e.g. gnutella p2p communication protocols H04L67/104
• CPC Class Codes: G06F 16/184 implemented as replicated file system

Abstract

The invention is a system for and a method of maintaining namespace consistency in a wide-area file system. In one embodiment, a wide-area file system has a plurality of replicas for a file. Each replica of a file and parent directories for the file are at a node. An update to a replica of the file is propagated to other replicas of the file. In response to receiving a propagated update to a replica at a node, the replica for the file at the node is updated. A directory operation (e.g., rename, link or unlink) may affect a backpointer for a replica. When a backpointer for a replica at a node is not consistent with the parent directories for the replica at the node, the the parent directories are modified to be consistent with the backpointer. A modification may be performed at a node while an earlier inconsistent modification may be ignored. As a result, consistency is maintained among the replicas.

Claims

1. A method for a wide-area file system, including a plurality of nodes storing replicas of objects, the objects being files and file directories, wherein for each replica of an object at a node, a parent directory for the object is replicated at the node, the method comprising:

propagating an update to a replica of a file directory to other replicas of the file directory via a graph, wherein each replica of the file directory has edges to only a subset of the other replicas such that all the replicas of the file directory are connected via the graph; and

in response to receiving a propagated update to a replica of the file directory at a node, updating the replica for the file directory at the node.

2. The method according to claim 1, wherein each replica of an object has a backpointer including an identification of a parent directory for the object and a name of the object in the parent directory.

3. The method according to claim 2, wherein the parent directories are modified when the backpointer for a replica of an object at a node is not consistent with the parent directories for the replica of the object at the node.

4. The method according to claim 3, wherein modifying the parent directories occurs only after a delay.

5. The method according to claim 3, wherein multiple modifications to the parent directories at the node are performed according to an order in which corresponding updates occur.

6. The method according to claim 3, wherein a modification is performed at the node and an earlier inconsistent modification is ignored.

7. The method according to claim 2, wherein a directory operation affects the backpointer for the object.

8. The method according to claim 7, wherein the directory operation is selected from a group consisting of rename, link and unlink.

9. The method according to claim 8, wherein when the backpointer for a replica of an object at a node is not consistent with the parent directories for the replica of the object at the node, further comprising modifying the parent directories to be consistent with the backpointer.

10. The method according to claim 1, wherein the replicas of the file directory include core replicas and non-core replicas, the parent directory for the file directory having edges only to the core replicas of the file directory and each core replica of the file directory having edges to one or more of the non-core replicas of the file directory.

11. The method according to claim 10, wherein in response to a user accessing an object at a node when no replica of the object exists at the node, the method further comprises steps of forming a non-core replica of the parent directory for the object at the node and forming a non-core replica of the object at the node.

12. The method according to claim 10, wherein a minimum number of core replicas are maintained according to a minimum replication factor.

13. The method according to claim 1, wherein a replica of an object is deleted by marking the replica as invalid.

14. The method according to claim 13, wherein said marking the replica as invalid comprises removing the backpointer for the replica.

15. The method according to claim 13, further comprising periodically removing replicas marked as invalid.

16. A system including:

a plurality of nodes that store replicas of objects, the objects being files and file directories,

wherein for each replica of an object at a node, the node stores a replica of a parent directory for the object and a backpointer having an identification of the parent directory for the object, and

wherein each replica of a file directory has edges to only a subset of the other replicas of the file directory such that all the replicas of the file directory are connected via the graph and the nodes are configured to propagate updates to replicas of each file directory to other replicas of the file directory via the graph.

17. The system according to claim 16, wherein in response to receiving a propagated update to a replica of the file directory at a node, the node updates the parent directories for the file at the node.

18. The system according to claim 16, wherein when a backpointer for a replica of an object at a node is not consistent with the parent directories for the replica of the object at the node, the node modifies the parent directories to be consistent with the backpointer.

19. The system according to claim 18, wherein the node modifies the parent directories to be consistent with the backpointer only after a delay.

20. The system according to claim 18, wherein multiple modifications to the parent directories at the node are performed according to an order in which corresponding updates occur.

21. The system according to claim 18, wherein a modification is performed at the node and an earlier inconsistent modification is ignored.

22. The system according to claim 16, wherein a directory operation is affects the backpointer for the object.

23. The system according to claim 22, wherein the directory operation is selected from a group consisting of rename, link and unlink.

24. The system according to claim 22, wherein when a backpointer for a replica of an object at a node is not consistent with the parent directories for the replica of the object at the node, the node modifies the parent directories to be consistent with the backpointer.

25. The system according to claim 16, wherein the replicas of the file directory include core replicas and non-core replicas, the parent directory for the file directory having edges only to the core replicas of the file directory and each core replica of the file directory having edges to one or more of the non-core replicas of the file directory.

26. The system according to claim 16, wherein in response to a user accessing an object at a node when no replica of the object exists at the node, a non-core replica of the object and a non-core replica of the parent directory for the object are formed at the node.

27. The system according to claim 16, wherein a replica of an object is deleted by marking the replica as invalid.

28. The system according to claim 27, wherein said marking the replica as invalid comprises removing the backpointer for the replica.

29. The system according to claim 27, further comprising periodically removing replicas marked as invalid.