Concurrent file across at a target file server during migration of file systems between file servers using a network file system access protocol

• US 6,938,039 B1
• Filed: 06/30/2000
• Issued: 08/30/2005
• Est. Priority Date: 06/30/2000
• Status: Expired due to Term

• Current Assignee: Emc IP Holding Company LLC (Dell Technologies Inc.)
• Original Assignee: EMC Corporation (Dell Technologies Inc.)
• Inventors: Vahalia, Uresh, Alexander, Jeffrey L., Gupta, Uday K., Bober, Paul M., John, Aju
• Primary Examiner(s): Coby, Frantz
• Assistant Examiner(s): Filipczyk, Marcin
• Application Number:
  US09/608,469
• Time in Patent Office: 1,887 Days
• Field of Search: 707 1- 10, 707/103, 707200-206, 711/2, 711/4, 711/168, 714/5, 395/704
• US Class Current: 707/704
• CPC Class Codes: G06F 16/119 Details of migration of file systems migration mechanisms in storage systems G06F3/0647
• CPC Class Codes: Y10S 707/99938 Concurrency, e.g. lock management in shared database
• CPC Class Codes: Y10S 707/99952 Coherency, e.g. same view to multiple users

Abstract

A file system is migrated from a source file server to a target file server in a data network while permitting clients to have concurrent read/write access to the file system. The target file server issues directory read requests and file read requests to the source file server in accordance with a network file access protocol to transfer the file system from the source file server to the target file server. Concurrent with the transfer of the file system from the source file server to the target file server, the target file server responds to client read/write requests for access to the file system. In a preferred embodiment, the target file server maintains a hierarchy of on-line nodes off-line nodes. The online nodes represent file system objects that have been completely migrated, and the offline nodes representing file system objects that have not been completely migrated. The target file server executes a background process that walks through the hierarchy in order to migrate the objects of the offline nodes. When an object has been completely migrated, the target file server changes the offline node for the object to an online node for the object.

Claims

1. A method of migrating a file system from a source file server to a target file server in a data network while permitting clients to have concurrent read/write access to the file system, said method comprising:

the target file server issuing directory read requests and file read requests to the source file server in accordance with a network file access protocol to transfer the file system from the source file server to the target file server, and

concurrent with the transfer of the file system from the source file server to the target file server, the target file server responding to client read/write requests for access to the file system.

2. The method as claimed in claim 1, wherein the source server initially has an Internet Protocol (IP) address for receiving client requests for access to the file system, and wherein the method includes taking the source file server offline with respect to the client requests and directing subsequent client requests to the target file server instead of the source file server by the target file server assuming the Internet Protocol (IP) address of the source file server.

3. The method as claimed in claim 1, wherein the target file server responds to a client request for access to an object of the file system by checking whether the object is in the target file server, and if the object is not in the target file server, fetching the object from the source file server and then accessing the object in the target file server once the object has been transferred to the target file server.

4. The method as claimed in claim 3, wherein the object is accessed in the target file server by calling an object accessor function of a file system program in the target file server, and wherein the checking of whether the object is in the target file server includes calling an object accessor function of migration code programming that is integrated with the file system program.

5. The method as claimed in claim 4, wherein the object accessor function of the migration code programming intercepts a call from the file system program that would, in the absence of migration being performed, call the object accessor function of the file system program.

6. A method of migrating a file system from a source file server to a target file server in a data network while permitting clients to have concurrent read/write access to the file system, said method comprising:

the target file server issuing directory read requests and file read requests to the source file server in accordance with a network file access protocol to transfer the file system from the source file server to the target file server, and

concurrent with the transfer of the file system from the source file server to the target file server, the target file server responding to client read/write requests for access to the file system;

wherein the target file server responds to a client request for access to an object of the file system by checking whether the object is in the target file server, and if the object is not in the target file server, fetching the object from the source file server and then accessing the object in the target file server once the object has been transferred to the target file server; and

wherein the file system includes nodes in a hierarchy of directories, the file system in the target file server includes online nodes and offline nodes, the online nodes representing objects that have been transferred from the source file server to the target file server, the offline nodes representing objects that have not yet been transferred from the source file server to the target file server, and wherein the target file server checks whether the object is in the target file server by checking whether the target file server has an online node for the object.

7. The method as claimed in claim 6, wherein the target file server does not have an online node for the object, and fetches data for the object from the source file server, writes the data in an offline node for the object, and then changes the offline node to an online node.

8. The method as claimed in claim 7, wherein the offline node is changed to an online node by switching a flag for the offline node, the flag indicating either an offline node or an online node.

9. The method as claimed in claim 6, wherein the object is a file, and wherein the target file server fetches, from the source file server, file data from the beginning of the file up to the last byte of the file data to be accessed in response to the client request for access.

10. The method as claimed in claim 9, wherein the target file server places the file data in a migration node in the target file server, and once all of the file data are placed in the migration node, an offline node for the file is overwritten with the migration node, and then the offline node for the file is changed to an online node for the file.

11. The method as claimed in claim 6, wherein the object is a file directory including at least one file, and the method includes the target file system checking a link count attribute for the file, and upon finding that the link count attribute has a value greater than one, accessing a migration directory to find a node for the file, and placing a reference to the node for the file in the file directory.

12. The method as claimed in claim 6, wherein the target file server executes a routine that walks through the hierarchy of directories to read the last byte of at least each offline file.

13. The method as claimed in claim 6, wherein the target file server maintains a count of the number of offline nodes for the file system, the count being decremented when an offline node is changed to an online node, and the target file server recognizing that migration of the file system is finished when the count becomes zero.

14. A method of migrating a file system from a source file server to a target file server in a data network while permitting clients to have concurrent read/write access to the file system, said method comprising:

taking the source file server offline with respect to the client requests and directing subsequent client requests to the target file server instead of the source file server;

the target file server maintaining a hierarchy of nodes representing file system objects, the nodes including online nodes and offline nodes, the online nodes representing file system objects that have been transferred from the source file server to the target file server, the offline nodes representing file system objects that have not yet been transferred from the source file server to the target file server,

the target file server walking through the hierarchy of nodes and issuing directory read requests and file read requests to the source file server in accordance with a network file access protocol to transfer the file system objects represented by offline nodes from the source file server to the target file server, and when a file system object has been transferred from the source file server to the target file server, the target file system changing the offline node for the object to an online node for the object, and

concurrent with the transfer of the file system from the source file server to the target file server, the target file server responding to client read/write requests for access to the file system, the target file server responding by accessing objects in the target file server when the target file server has online nodes for the objects, and by fetching objects from the source file server when the target file server has offline nodes for the objects.

15. The method as claimed in claim 14, wherein the source server initially has an Internet Protocol (IP) address for receiving client requests for access to the file system, and the source file server is taken offline with respect to the client requests and subsequent client requests are directed to the target file server instead of the source file server by the target file server assuming the Internet Protocol (IP) address of the source file server.

16. The method as claimed in claim 14, wherein the objects are accessed in the target file server by calling an object accessor function of a file system program in the target file server, and wherein the target file server responds to the client read/write requests by calling an object accessor function of migration code programming that is integrated with the file system program.

17. The method as claimed in claim 16 wherein the object accessor function of the migration code programming intercepts calls from the file system program that would, in the absence of migration being performed, call the object accessor function of the file system program.

18. The method as claimed in claim 14, wherein the offline node for the object is changed to an online node for the object by switching a flag for the offline node, the flag indicating either an offline node or an online node.

19. The method as claimed in claim 14, wherein the object is a file, and wherein the target file server fetches, from the source file server, file data from the beginning of the file up to the last byte of the file data to be accessed in response to the client request for access.

20. The method as claimed in claim 19 wherein the target file server places the file data in a migration node in the target file server, and once all of the file data are placed in the migration node, an offline node for the file is overwritten with the migration node, and then the offline node for the file is changed to an online node for the file.

21. The method as claimed in claim 14, wherein the object is a file directory including at least one file, and the method includes the target file system checking a link count attribute for the file, and upon finding that the link count attribute has a value greater than one, accessing a migration directory to find a node for the file, and placing a reference to the node for the file in the file directory.

22. The method as claimed in claim 14, wherein the target file server executes a background routine that walks through the hierarchy of nodes to find nodes representing files, and upon finding a node representing a file, performs a read operation including a read of the last byte of the file.

23. The method as claimed in claim 14 wherein the target file server maintains a count of the number of offline nodes for the file system, the count being decremented when an offline node is changed to an online node, and the target file server recognizing that migration of the file system is finished when the count becomes zero.

24. A network file server comprising data storage and at least one processor coupled to the data storage for storing and retrieving data from the data storage, the network file server being adapted for coupling to a data network for communication with network clients, the network file server being programmed with a file system program for maintaining files of data in the data storage and responding to client read/write requests for access to the files, the network file server also being programmed with a migration program for issuing directory read requests and file read requests to a source file server in accordance with a network file access protocol to migrate a file system from the source file server to the network file server while permitting the network clients to have concurrent read/write access to the file system.

25. The network file server as claimed in claim 24, wherein the file system program is a Unix-based file system program, and the migration program is integrated with the file system program.

26. The network file server as claimed in claim 24, wherein the network file server is programmed to respond to a client request for access to an object of the file system by checking whether the object is in the network file server, and if the object is not in the network file server, fetching the object from the source file server and then accessing the object in the network file server once the object has been transferred to the network file server.

27. The network file server as claimed in claim 26, wherein the network file server is programmed to access the object by calling an object accessor function of the file system program, and to check whether the object is in the network file server by calling an object accessor function of the migration program.

28. The network file server as claimed in claim 27, wherein the object accessor function of the migration program intercepts calls from the file system program that would, in the absence of migration being performed, call the object accessor function of the file system program.

29. A network file server comprising data storage and at least one processor coupled to the data storage for storing and retrieving data from the data storage, the network file server being adapted for coupling to a data network for communication with network clients, the network file server being programmed with a file system program for maintaining files of data in the data storage and responding to client read/write requests for access to the files, the network file server also being programmed with a migration program for issuing directory read requests and file read requests to a source file server in accordance with a network file access protocol to migrate a file system from the source file server to the network file server while permitting the network clients to have concurrent read/write access to the file system;

wherein the network file server is programmed to respond to a client request for access to an object of the file system by checking whether the object is in the network file server, and if the object is not in the network file server, fetching the object from the source file server and then accessing the object in the network file server once the object has been transferred to the network file server; and

wherein the file system includes nodes in a hierarchy of directories, the file system in the network file server includes online nodes and offline nodes, the online nodes representing objects that have been transferred from the source file server to the network file server, the offline nodes representing objects that have not yet been transferred from the source file server to the network file server, and wherein the network file server is programmed to check whether the object is in the network file server by checking whether the network file server has an online node for the object.

30. The network file server as claimed in claim 29, wherein the network file server is programmed to fetch data for offline nodes from the source file server, and to write the fetched data in the offline node for the object, and then change the offline node to an online node.

31. The network file server as claimed in claim 30, wherein the network file server is programmed to change the offline node to an online node by switching a flag for the offline node, the flag indicating either an offline node or an online node.

32. The network file server as claimed in claim 29, wherein the object is a file, and wherein the network file server is programmed to fetch, from the source file server, file data from the beginning of the file up to the last byte of the file data to be accessed in response to the client request for access.

33. The network file server as claimed in claim 32, wherein the network file server is programmed to place the fetched file data in a migration node in the network file server, and once all of the file data are placed in the migration node, an offline node for the file is overwritten with the migration node, and then the offline node for the file is changed to an online node for the file.

34. The network file server as claimed in claim 29, wherein the object is a file directory including at least one file, and the network file server is programmed to check a link count attribute for the file, and upon finding that the link count attribute has a value greater than one, to access a migration directory to find a node for the file, and place a reference to the node for the file in the file directory.

35. The network file server as claimed in claim 29, wherein the network file server is programmed for walking through the hierarchy of directories and for performing read operations including a read of the last byte of at least each offline file.

36. The network file server as claimed in claim 29, wherein the network file server is programmed for maintaining a count of the number of offline nodes for the file system, for decrementing the count when an offline node is changed to an online node, and for recognizing that migration of the file system is finished when the count becomes zero.

37. A network file server comprising data storage and at least one processor coupled to the data storage for storing and retrieving data from the data storage, the network file server being adapted for coupling to a data network for communication with network clients, the network file server being programmed with a file system program for maintaining files of data in the data storage and responding to client read/write requests for access to the files, the network file server also being programmed with a migration program integrated with the file system program for issuing directory read requests and file read requests to a source file server in accordance with a network file access protocol to migrate a file system from the source file server to the network file server while permitting the network clients to have concurrent read/write access to the file system, the migration program being executable by the network file server for:

maintaining a hierarchy of nodes representing file system objects, the nodes including online nodes and offline nodes, the online nodes representing file system objects that have been transferred from the source file server to the network file server, the offline nodes representing file system objects that have not yet been transferred from the source file server to the network file server,

walking through the hierarchy of nodes and issuing directory read requests and file read requests to the source file server in accordance with a network file access protocol to transfer the file system objects represented by the offline nodes from the source file server to the network file server, and when a file system object has been transferred from the source file server to the network file server, f6r changing the offline node for the object to an online node for the object, and

responding to client read/write requests for access to the file system, concurrent with the transfer of the file system from the source file server to the network file server, by accessing objects in the network file server when the network file server has online nodes for the objects, and by fetching objects from the source file server when the network file server has offline nodes for the objects.

38. The network file server as claimed in claim 37, wherein the file system program has an object accessor function, and the migration program has an object accessor function that intercepts calls from the file system program that would, in the absence of migration being performed, call the object accessor function of the file system program.

39. The network file server as claimed in claim 37, wherein the migration program changes the offline node for the object to an online node for the object by switching a flag for the offline node, the flag indicating either an offline node or an online node.

40. The network file server as claimed in claim 37, wherein the network file server is programmed to respond to a client request for access to a file in the file system by fetching, from the source file server, file data from the beginning of the file up to the last byte of the file data to be accessed in response to the client request for access.

41. The network file server as claimed in claim 40 wherein the network file server is programmed to place the file data in a migration node in the network file server, and once all of the file data are placed in the migration node, to overwrite an offline node for the file with the migration node, and then to change the offline node for the file to an online node for the file.

42. The network file server as claimed in claim 37, wherein the network file server is programmed to migrate a file directory by checking a link count attribute for the file, and upon finding that the link count attribute has a value greater than one, accessing a migration directory to find a node for the file, and placing a reference to the node for the file in the file directory.

43. The network file server as claimed in claim 37, wherein the network file server is programmed for walking through the hierarchy of directories to find nodes representing files, and upon finding a node representing a file, for performing a read operation including a read of the last byte of the file.

44. The network file server as claimed in claim 37, wherein the network file server is programmed for maintaining a count of the number of offline nodes for the file system, decrementing the count when an offline node is changed to an online node, and recognizing that migration of the file system is finished when the count becomes zero.

45. A program storage device containing a file system migration program for a network file server, the network file server including data storage and at least one processor coupled to the data storage for storing and retrieving data from the data storage, the network file server being adapted for coupling to a data network for communication with network clients, the network file server being programmed with a file system program for maintaining files of data in the data storage and responding to client read/write requests for access to the files, wherein the migration program is executable by the network file server for issuing directory read requests and file read requests to a source file server in accordance with a network file access protocol to migrate a file system from the source file server to the network file server while permitting the network clients to have concurrent read/write access to the file system.

46. The program storage device as claimed in claim 45, wherein the file system program is a Unix-based file system program, and the migration program is integrated with the file system program.

47. The program storage device as claimed in claim 45, wherein the migration program is executable by the network file server for responding to a client request for access to an object of the file system by checking whether the object is in the network file server, and if the object is not in the network file server, fetching the object from the source file server and then accessing the object in the network file server once the object has been transferred to the network file server.

48. The program storage device as claimed in claim 47, wherein the migration program has an object accessor function executable by the network file server for accessing the object by calling an object accessor function of the file system program, and for checking whether the object is in the network file server by calling an object accessor function of the migration program.

49. The program storage device as claimed in claim 48, wherein the object accessor function of the migration program intercepts calls from the file system program that would, in the absence of migration being performed, call the object accessor function of the file system program.

50. A program storage device containing a file system migration program for a network file server, the network file server including data storage and at least one processor coupled to the data storage for storing and retrieving data from the data storage, the network file server being adapted for coupling to a data network for communication with network clients, the network file server being programmed with a file system program for maintaining files of data in the data storage and responding to client read/write requests for access to the files, wherein the migration program is executable by the network file server for issuing directory read requests and file read requests to a source file server in accordance with a network file access protocol to migrate a file system from the source file server to the network file server while permitting the network clients to have concurrent read/write access to the file system;

wherein the migration program is executable by the network file server for responding to a client request for access to an object of the file system by checking whether the object is in the network file server, and if the object is not in the network file server, fetching the object from the source file server and then accessing the object in the network file server once the object has been transferred to the network file server; and

wherein the file system includes nodes in a hierarchy of directories, the file system in the network file server includes online nodes and offline nodes, the online nodes representing objects that have been transferred from the source file server to the network file server, the offline nodes representing objects that have not yet been transferred from the source file server to the network file server, and wherein the migration program is executable by the network file server for checking whether the object is in the network file server by checking whether the network file server has an online node for the object.

51. The program storage device as claimed in claim 50, wherein the migration program is executable by the network file server for fetching data for offline nodes from the source file server, and writing the fetched data in the offline node for the object, and then changing the offline node to an online node.

52. The program storage device as claimed in claim 51, wherein the migration program is executable by the network file server for changing the offline node to an online node by switching a flag for the offline node, the flag indicating either an offline node or an online node.

53. The program storage device as claimed in claim 50, wherein the object is a file, and wherein the migration program is executable by the network file server for fetching, from the source file server, file data from the beginning of the file up to the last byte of the file data to be accessed in response to the client request for access.

54. The program storage device as claimed in claim 53, wherein the migration program is executable by the network file server for placing the fetched file data in a migration node in the network file server, and once all of the file data are placed in the migration node, for overwriting an offline node for the file with the migration node, and then changing the offline node for the file to an online node for the file.

55. The program storage device as claimed in claim 50, wherein the object is a file directory including at least one file, and the migration program is executable by the network file server for checking a link count attribute for the file, and upon finding that the link count attribute has a value greater than one, for accessing a migration directory to find a node for the file, and placing a reference to the node for the file in the file directory.

56. The program storage device as claimed in claim 50, wherein the migration program is executable by the network files server for walking through the hierarchy of directories to read the last byte of at least each offline file.

57. The program storage device as claimed in claim 50, wherein the migration program is executable by the network file server for maintaining a count of the number of offline nodes for the file system, for decrementing the count when an offline node is changed to an online node, and for recognizing that migration of the file system is finished when the count becomes zero.

58. A program storage device containing a file system migration program for a network file server, the network file server including data storage and at least one processor coupled to the data storage for storing and retrieving data from the data storage and, the network file server being adapted for coupling to a data network for communication with network clients, the network file server being programmed with a file system program for maintaining files of data in the data storage and responding to client read/write requests for access to the files, the migration program being executable by the network file server and being integrated with the file system program for issuing directory read requests and file read requests to a source file server in accordance with a network file access protocol to migrate a file system from the source file server to the network file server while permitting the network clients to have concurrent read/write access to the file system, the migration program being executable by the network file server for:

maintaining a hierarchy of nodes representing file system objects, the nodes including online nodes and offline nodes, the online nodes representing file system objects that have been transferred from the source file server to the network file server, the offline nodes representing file system objects that have not yet been transferred from the source file server to the network file server,

walking through the hierarchy of nodes for issuing directory read requests and file read requests to the source file server in accordance with a network file access protocol to transfer the file system objects represented by the offline nodes from the source file server to the network file server, and when a file system object has been transferred from the source file server to the network file server, for changing the offline node for the object to an online node for the object, and

responding to client read/write requests for access to the file system, concurrent with the transfer of the file system from the source file server to the network file server, by accessing objects in the network file server when the network file server has online nodes for the objects, and by fetching objects from the source file server when the network file server has offline nodes for the objects.

59. The program storage device as claimed in claim 58, wherein the file system program has-an object accessor function, and the migration program has an object accessor function that intercepts calls from the file system program that would, in the absence of migration being performed, call the object accessor function of the file system program.

60. The program storage device as claimed in claim 58, wherein the migration program is executable by the network file server for changing the offline node for the object to an online node for the object by switching a flag for the offline node, the flag indicating either an offline node or an online node.

61. The program storage device as claimed in claim 58, wherein the migration program is executable by the network file server for responding to a client request for access to a file in the file system by fetching, from the source file server, file data from the beginning of the file up to the last byte of the file data to be accessed in response to the client request for access.

62. The program storage device as claimed in claim 61 wherein the migration program is executable by the network file server for placing the file data in a migration node in the network file server, and once all of the file data are placed in the migration node, to overwrite an offline node for the file with the migration node, and then to change the offline node for the file to an online node for the file.

63. The program storage device as claimed in claim 58, wherein the migration program is executable by the network file server for migrating a file directory by checking a link count attribute for the file, and upon finding that the link count attribute has a value greater than one, accessing a migration directory to find a node for the file, and placing a reference to the node for the file in the file directory.

64. The program storage device as claimed in claim 58, wherein the migration program is executable by the network file server for walking through the hierarchy of directories to find nodes representing files, and upon finding a node representing a file, for performing a read operation including a read of the last byte of the file.

65. The program storage device as claimed in claim 58, wherein the migration program is executable by the network file server for maintaining a count of the number of offline nodes for the file system, decrementing the count when an offline node is changed to an online node, and recognizing that migration of the file system is finished when the count becomes zero.