System, method, and service for federating and optionally migrating a local file system into a distributed file system while preserving local access to existing data

• US 20050114291A1
• Filed: 11/25/2003
• Published: 05/26/2005
• Est. Priority Date: 11/25/2003
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Becker-Szendy, Ralph Attila, Sivan-Zimet, Miriam
• Granted Patent: US 7,243,089 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 16/10 File systems; File servers
• CPC Class Codes: Y10S 707/99931 Database or file accessing
• CPC Class Codes: Y10S 707/99945 Object-oriented database structure processing

Abstract

Indirect access to local file systems is provided using storage tank protocols, allowing federation of a local file system into a distributed file system while preserving local access to the existing data in the local file system. The goal of the present system is to federate and migrate the data on a computer system with minimum disruption to applications operating on the computer system. Existing applications on a computer system continue to operate during data federation and migration and require little or no reconfiguration either when the data migration starts or when it ends. Data consistency is maintained: existing applications may modify data in the file system during migration or federation. During federation, other computer systems (or hosts) may modify the data in the file system if access control information allows them to do so. All changes in the file system are seen consistently on all hosts. Minimal downtime is required to install the present system and reconfigure the existing applications to communicate with the present system.

Claims

1. A method of federating a local file system into a distributed file system while preserving local access to an existing data in the local file system, comprising:

adding a federation layer that allows both a local client and a plurality of distributed clients to access the existing data;

allowing local applications to access both the data exposed in the local file system and data in other parts of the distributed file system; and

wherein the federation layer establishes a detour between the local applications and the local file system, to provide access to a distributed mechanism.

2. The method of claim 1, wherein the federation layer comprises a virtual server that serves file data and metadata from the local file system to the distributed file system.

3. The method of claim 2, wherein the virtual server comprises:

a virtual metadata server that serves file metadata;

a virtual storage server that serves file content; and

wherein the virtual metadata and the virtual storage server serve data from the local file system to the distributed file system.

4. The method of claim 3, wherein the virtual storage server is a virtual object storage server.

5. The method of claim 1, wherein the federation layer contains an object ID database for mapping between a file name and an object ID number that is unambiguous and stable.

6. The method of claim 2, wherein the federation layer contains a distributed file system client for transferring and translating communications from a local application on the local computer system to the virtual server.

7. The method of claim 2, further comprises installing the virtual file server in the local computer system.

8. The method of claim 3, further comprises installing the virtual metadata server in the local computer system.

9. The method of claim 4, further comprises installing. the virtual object storage server in the local computer system.

10. The method of claim 6, further comprises installing the distributed file system client in the local computer system.

11. The method of claim 2, further comprises configuring the virtual file server to communicate with the distributed file system.

12. The method of claim 11, further comprises:

temporarily disconnecting the local applications on the local computer system; and

reconfiguring the local applications to communicate with the distributed file system client.

13. The method of claim 5, further comprises assigning an object ID number to each object found in the local file system.

14. The method of claim 13, further comprises generating the object ID number by counting the objects that were found in the local file system.

15. The method of claim 13, further comprises:

generating the object ID number as a tuple <

I,G>

of an inode number of the object found in the local file system, and a generation number;

wherein the generation number is increased anytime a same inode number is found for a different object; and

maintaining a deleted flag for objects that have been deleted, so the occurrence of the same inode number for different objects can be recognized.

16. The method of claim 13, further comprises:

detecting multiple hard links to a same file by comparing an inode number of files;

if one file has more than one hard link, using a same object ID number for the multiple names for that object; and

deleting only the object when a last hard link has been unlinked.

17. The method of claim 6, further comprises using a shared memory between the distributed file system client and the virtual server to enhance communication between the distributed file system client and the virtual server.

18. The system of claim 6, further comprises moving the virtual server into a kernel space for efficiency purpose.

19. A computer program product having instruction codes for federating a local file system into a distributed file system while preserving local access to an existing data in the local file system, comprising:

a first set of instruction codes for adding a federation layer that allows both a local client and a plurality of distributed clients to access the existing data;

a second set of instruction codes for allowing local applications to access both the data exposed in the local file system and data in other parts of the distributed file system; and

wherein the federation layer establishes a detour between the local applications and the local file system, to provide access to a distributed mechanism.

20. The computer program product of claim 19, wherein the federation layer comprises a virtual server that serves file data and metadata from the local file system to the distributed file system.

21. The computer program product of claim 20, wherein the virtual server comprises:

a virtual metadata server that serves file metadata;

a virtual storage server that serves file content; and

wherein the virtual metadata and the virtual storage server serve data from the local file system to the distributed file system.

22. The computer program product of claim 21, wherein the virtual storage server is a virtual object storage server.

23. The computer program product of claim 19, wherein the federation layer contains an object ID database for mapping between a file name and an object ID number that is unambiguous and stable.

24. The computer program product of claim 20, wherein the federation layer contains a distributed file system client for transferring and translating communications from a local application on the local computer system to the virtual server.

25. The computer program product of claim 20, further comprises a third set of instruction codes for installing the virtual file server in the local computer system.

26. The computer program product of claim 21, further comprises a fourth set of instruction codes for installing the virtual metadata server and the virtual in the local computer system.

27. The computer program product of claim 22, further comprises a fifth set of instruction codes for installing the virtual object storage server in the local computer system.

28. The computer program product of claim 23, further comprises a sixth set of instruction codes for installing the distributed file system client in the local computer system.

29. The computer program product of claim 20, further comprises a seventh set of instruction codes for configuring the virtual file server to communicate with the distributed file system.

30. The computer program product of claim 29, further comprises:

an eight set of instruction codes for temporarily disconnecting the local applications on the local computer system; and

a ninth set of instruction codes for reconfiguring the local applications to communicate with the distributed file system client.

31. A service for federating a local file system into a distributed file system while preserving local access to an existing data in the local file system, comprising:

an addition of a federation layer that allows both a local client and a plurality of distributed clients to access the existing data;

an allowance of local applications to access both the data exposed in the local file system and data in other parts of the distributed file system; and

wherein the federation layer establishes a detour between the local applications and the local file system, to provide access to a distributed mechanism.

32. The service of claim 31, wherein the federation layer comprises a virtual server that serves file data and metadata from the local file system to the distributed file system.

33. The service of claim 32, wherein the virtual server comprises:

a virtual metadata server that serves file metadata;

a virtual storage server that serves file content; and

wherein the virtual metadata and the virtual storage server serve data from the local file system to the distributed file system.

34. The service of claim 33, wherein the virtual storage server is a virtual object storage server.

35. The service of claim 31, wherein the federation layer contains an object ID database for mapping between a file name and an object ID number that is unambiguous and stable.

36. The service of claim 32, wherein the federation layer contains a distributed file system client for transferring and translating communications from a local application on the local computer system to the virtual server.

37. The service of claim 32, further comprises an installation of the virtual file server in the local computer system.

38. The service of claim 33, further comprises an installation of the virtual metadata server and the virtual in the local computer system.

39. The service of claim 34, further comprises an installation of the virtual object storage server in the local computer system.

40. The service of claim 36, further comprises an installation of the distributed file system client in the local computer system.