File replication method for distributed file systems

• US 20040205152A1
• Filed: 01/30/2004
• Published: 10/14/2004
• Est. Priority Date: 01/30/2003
• Status: Active Grant

• Current Assignee: Hitachi, Ltd.
• Original Assignee: Hitachi, Ltd.
• Inventors: Okitsu, Jun, Kawamoto, Shinichi, Yasuda, Yoshiko, Ebata, Atsushi
• Granted Patent: US 7,590,667 B2
• Time in Patent Office: Days
• US Class Current: 709/217
• CPC Class Codes: G06F 16/184 implemented as replicated file system
• CPC Class Codes: G06F 16/188 Virtual file systems
• CPC Class Codes: Y10S 707/99931 Database or file accessing
• CPC Class Codes: Y10S 707/99938 Concurrency, e.g. lock management in shared database
• CPC Class Codes: Y10S 707/99952 Coherency, e.g. same view to multiple users
• CPC Class Codes: Y10S 707/99953 Recoverability

Abstract

A file replication method which allows, between a plurality of virtualized-and-unified file systems, real-time creation of a partial copy of that one of the virtualized-and-unified file systems as a target of replication is provided. The replication-source virtualized-and-unified file system is provided with a unified management directory for managing a structure of files and directories and the attribute information thereof, replication information for setting the file as an object to be copied, a request reception unit for receiving a request from a client in a unified manner, and a replication unit for judging whether or not replication should be performed in accordance with the unified management directory and the replication information and simultaneously transferring, based on the result of the judgment, the file access request to the replication-source virtualized-and-unified file system and to the replication-destination virtualized-and-unified file system.

Claims

1. A file replication method for creating, in a distributed file system including a plurality of network storage apparatus and a replication system each connected to a network wherein the replication system has a management table for managing attribute information of all files and directories in the network storage apparatus as a replication source, a partial copy of data stored in the network storage apparatus as the replication source in the network storage apparatus as a replication destination, said method comprising the steps of:

preliminarily recording replication information for specifying a file as a target of replication in said replication system;

receiving a file access request from a client;

judging whether or not a replicating operation should be performed with execution of said file access request by using said management table and said replication information; and

simultaneously transferring, if a result of said judgment is such that the replicating operation should be performed, said file access request to said network storage apparatus as the replication source and to said network storage apparatus as the replication destination.

2. A file replication method according to claim 1, wherein said replication system contains synchronization information indicative of whether or not contents of a file and a directory, each as an object to be copied, maintain consistency between the network storage apparatus as the replication source and the network storage apparatus as the replication destination and judges that the replicating operation should be performed under a condition that said synchronization information indicates consistency in said judgment step.

3. A file replication method according to claim 1, wherein the judgment is performed in said judgment step that the replicating operation should be performed under a condition that the received file access request is a write request.

4. A replication system for performing file replication between a plurality of network storage apparatus connected to a network, said replication system comprising:

a management table for managing attribute information of all files and directories in the network storage apparatus as a replication source;

a unit for holding replication information indicative of whether or not replication should be performed;

a request reception unit for receiving, a file access request from a client; and

a replication unit for simultaneously transferring the file access request to the plurality of network storage apparatus, said replication unit including

a judgment unit for judging whether or not replication should be performed by using the management table and the replication information and

a request processing unit for simultaneously transferring, in accordance with a result of the judgment, the file access request to the network storage apparatus as the replication source and to the network storage apparatus as a replication destination.

5. A replication system according to claim 4, wherein said replication unit further includes a consistency unit for maintaining consistency of all files and directories, each as an object to be copied, between the network storage apparatus as the replication source and the network storage apparatus as the replication destination.

6. A replication system according to claim 4, wherein said judgment unit judges that replication should be performed if the received file access request is a write request.

7. A replication system according to claim 4, further having a unit for holding synchronization information indicating that contents of a file and a directory, each as an object to be copied, maintain consistency between the network storage apparatus as the replication source and the network storage apparatus as the replication destination and said judgment unit judges whether or not replication should be performed by also using the synchronization information.

8. A replication system according to claim 4, wherein said replication information is at least one rule indicating that a file having a specified user or group identifier, a file belonging to a specified directory, or a file having a specified file identifier is an object to be copied.

9. A file replication method for creating, in an external network storage, a partial copy of data stored in a virtualized-and-unified file system including a plurality of network storages and a unification virtualizing system for managing, in a unified manner, a structure of files and directories present in distributed relation in the network storages and attribute information thereof and allowing a unified access to the network storages from an outside, said method comprising the steps of:

preliminarily recording, in said unification virtualizing system, replication information specifying the file as a target of replication;

receiving, in a unified manner, a file access request from a client to said virtualized-and-unified file system;

specifying the network storage storing therein the file as a target of said file access request by using a mapping unit for determining and setting the network storage for each of the files;

judging, by using said replication information, whether or not a replicating operation should be performed with execution of said file access request; and

transferring said file access request to the specified network storage and, if a result of said judgment is such that the replicating operation should be performed, transferring the file access request also to the external network storage as a replication destination, and thereby causing each of the network storage storing therein said target file and the external network storage as the replication destination to execute a file access requested by the file access request.

10. A file replication method according to claim 9, wherein it is judged in said judgment step that the replicating operation should be performed under a condition that the received file access request is a write request.

11. A file replication method according to claim 9, said method further comprising the step of:

collecting a response to the file access request from said network storage device storing therein said file and a response to the file access request from said network storage as the replication destination and returning the collected responses as one response to said client.

12. A unification virtualizing system for a plurality of network storages, said system virtually showing the plurality of network storages connected to a network as a single file system and comprising:

a unified management directory for managing a structure of all files and directories present in said virtualized-and-unified file system and attribute information thereof;

a unit for holding replication information for specifying the file to be replicated by assuming that an external file system is a replication destination;

a request reception unit for receiving a file access request from a client;

a mapping unit for determining the network storage of a file access target of said file access request;

a judgment unit for judging whether or not the file access target of said file access request should be replicated in said external file system by using the unified management directory and the replication information; and

a request transfer unit for simultaneously transferring, if the judgment unit judges that replication should be performed, said file access request to the external file system as the replication destination and to the network storage determined by the mapping unit.

13. A unification virtualizing system for a plurality of network storages according to claim 12, further comprising:

a consistency unit for maintaining consistency of a file and an directory, each as an object to be copied, between said virtualized-and-unified file system and said external file system as the replication destination.

14. A unification virtualizing system for a plurality of network storages according to claim 12, further comprising:

a response collection unit for collecting a response to the file access request from each of the network storages belonging to said virtualized-and-unified file system and a response to the file access request from said external file system as the replication destination and returning the collected responses as one response to the client.

15. A unification virtualizing system for a plurality of network storages according to claim 12, further comprising:

a unit for holding synchronization information indicating that all files and directories, each as an object to be copied, maintain consistency between said virtualized-and-unified file system and said file system as the replication destination, wherein

said judgment unit judges whether or not replication should be performed by also using the synchronization information.

16. A unification virtualizing system for a plurality of network storages according to claim 12, wherein said judgment unit judges that replication should not be performed if the file access request is a read request and the file access request is not transferred to the external file system as the replication destination.

17. A unification virtualizing system for a plurality of network storages according to claim 12, wherein said replication information is at least one rule indicating that a file having a specified user or group identifier, a file subordinate to a specified directory, or a file having a specified file identifier is an object to be copied.

18. A unification virtualizing system for a plurality of network storages according to claim 12, further comprising:

a unit for holding master information indicating that the files and directories managed by the unified management directory are masters, wherein

said judgment unit judges whether or not replication should be performed in accordance also with the master information.

19. A unification virtualizing system for a plurality of network storages according to claim 17, wherein said replication information includes not only the rule but also information for identifying the virtualized-and-unified file system to which the rule is applied.

22. A unification virtualizing system for a plurality of network storages according to claim 12, further comprising:

a capacity management unit for periodically acquiring respective disk capacities and amounts of disk use of said virtualized-and-unified file system and said external file system as the replication destination and determining, from said disk capacities and amounts of disk use, a disk capacity and an amount of disk use which allow for replication.

20. A virtualized-and-unified file system comprised of a plurality of network storages and a unification virtualizing system, wherein

the unification virtualizing system has a replication unit, said virtualized-and-unified file system including:

a plurality of virtualized-and-unified file systems, of which a first virtualized-and-unified file system responds to an access request from a client, activates the replication unit, and creates in real time a partial copy of the first virtualized-and-unified file system in another virtualized-and-unified file system connected to a network.

21. A virtualized-and-unified file system connected to a network and comprised of a plurality of network storages and a unification virtualizing system, wherein

the unification virtualizing system has a replication unit, said virtualized-and-unified file system including:

a plurality of virtualized-and-unified file systems, of which a first virtualized-and-unified file system responds to an access request from a client, activates the replication unit, and creates in real time a partial copy of the first virtualized-and-unified file system in another network storage connected to the network.