System and Method for Event-Based Synchronization of Remote and Local File Systems

• US 20140040196A1
• Filed: 08/02/2013
• Published: 02/06/2014
• Est. Priority Date: 08/03/2012
• Status: Active Grant

• Current Assignee: Egnyte Incorporated
• Original Assignee: Egnyte Incorporated
• Inventors: Wijayaratne, Ravi, Jassal, Amrit, Koos, Remus, White, Ray, Marathe, Manish, Mielityinen, Markku, Ram, Rajesh, Gopalan, Sriram, Aahz, Toivonen, Heikki J.
• Granted Patent: US 9,858,288 B2
• Time in Patent Office: Days
• US Class Current: 707/624
• CPC Class Codes: G06F 16/11 File system administration, e.g. details of archiving or snapshots file system backup G06F11/14
• CPC Class Codes: G06F 16/178 Techniques for file synchronisation in file systems

Abstract

A method for synchronizing a file system (FS) and a remote file system (RFS) includes monitoring the FS for FS events, generating FS event records, receiving RFS event records of RFS events, generating file system operations (FSOs) based on the FS and RFS event records, and communicating the FSOs to the FS and RFS to synchronize them. A method for generating the FSOs includes accessing a plurality of FS and/or RFS event records, processing the accessed records to generate processed event records, generating the FSOs based on the processed event records, and outputting the FSOs to cause synchronization of the FS and RFS. Systems are also described. The invention facilitates event-based, steady-state synchronization of local and remote file systems.

Claims

1. A method for synchronizing a file system (FS) and a remote file system (RFS) that is located remotely from said FS, said method comprising:

monitoring said FS for FS events, each of said FS events being indicative of a change made to said FS;

generating a plurality of event records based on said FS events, each of said event records being associated with one of said FS events;

receiving a plurality of RFS event records, each RFS event record being indicative of a remote event that changed said RFS;

generating file system operations based at least in part on said FS event records and said RFS event records; and

communicating said file system operations to said FS and said RFS to synchronize said FS and said RFS.

2. The method of claim 1, further comprising:

storing said plurality of event records in a first events database; and

storing said plurality of RFS event records in a second events database.

3. The method of claim 2, further comprising:

storing said event records in said first events database in chronological order; and

storing said RFS event records in said second events database in chronological order.

4. The method of claim 3, wherein said step of generating file system operations includes:

retrieving at least some of said event records from said first events database according to the hierarchy of said FS; and

retrieving at least some of said RFS event records from said second events database according to the hierarchy of said RFS.

5. The method of claim 3, further comprising:

copying at least some of said event records to a third events database prior to said step of generating; and

copying at least some of said RFS event records to a fourth events database prior to said step of generating.

6. The method of claim 1, further comprising:

establishing a connection with a remote file storage system having access to said RFS; and

receiving said plurality of RFS event records via said connection.

7. The method of claim 6, further comprising providing some of said file system operations to said remote file storage system via said connection.

8. The method of claim 1, further comprising:

defining an event synchronization time period; and

wherein said step of generating file system operations is based only on ones of said events and ones of said remote events that occurred during said event synchronization time period.

9. The method of claim 8, wherein the frequency of said event synchronization time period is adjustable.

10. The method of claim 1, further comprising synchronizing said FS and said RFS prior to said step of monitoring said FS for events.

11. The method of claim 1, further comprising:

establishing a client connection with a client desiring access to said FS; and

wherein said step of monitoring said FS for changes includes

monitoring calls going to said FS via said client connection and

executing a trap in response to each of said calls that will result in a change to said FS, said trap resulting in the generation of one of said event records.

12. The method of claim 1, wherein said step of generating file system operations is performed after each event record is generated.

13. The method of claim 1, further comprising:

processing said event records and said RFS event records to generate a set of processed event records; and

wherein

said step of generating said file system operations is based at least in part on said set of processed event records.

14. The method of claim 13, wherein said step of processing said event records includes:

identifying ones of said event records according to a type of said events; and

modifying at least some of said identified event records having said type.

15. The method of claim 14, wherein said step of processing said event records further includes deleting some of said event records.

16. The method of claim 15, further comprising:

accessing a first event record associated with a first event occurring on said FS;

accessing a second RFS event record associated with a second event occurring on said RFS; and

wherein

said first event record and said second event record are associated with a common file system object that is common to both said FS and said RFS; and

said step of processing said event records includes resolving a conflict between said first event and said second event.

17. A file storage system for synchronizing with a remote file storage system remotely located from said file storage system, said file storage system comprising:

memory for storing a file system including file system objects;

a client interface operative to enable a client to access said file system objects;

a file system module operative to monitor for changes being made to said file system by said client and to output event information about each of said changes;

a data monitor operative to generate event records responsive to event information received from said file system module;

a remote file server interface operative to establish a connection with said remote file storage system for communicating with said remote file storage system;

a synchronizer operative to receive a plurality of remote event records from said remote file storage system, said remote event records being indicative of changes made to a remote file system;

an event processor operative to generate file system operations based at least in part on said event records and said remote event records; and

an output for communicating said file system operations to said file system and said remote file system to synchronize said file system and said remote file system.

18. The system of claim 17, further comprising:

a first event database operative to store said event records; and

a second event database operative to store said remote event records.

19. The system of claim 18, wherein:

said event records are stored in said first event database in chronological order; and

said remote event records are stored in said second event database in chronological order.

20. The system of claim 19, wherein:

said first event database is operative to return event records according to the hierarchy of said file system; and

said second event database is operative to return remote event records according to the hierarchy of said remote file system.

21. The system of claim 19, wherein:

said data monitor is further operative to copy a portion of said event records from said first event database to a third event database;

said synchronizer is further operative to copy a portion of said remote event records from said second event database to a fourth event database; and

said event processor is operative to generate file system operations based only said event records and said remote event records stored in said third event database and said fourth event database.

22. The system of claim 17, wherein said synchronizer is further operative to:

request remote event records via said remote file server interface; and

store said remote event record in said second events database.

23. The system of claim 22, further comprising a sync action handler operative to:

cause some of said file system operations to be applied to said file system; and

cause others of said file system operations to be transmitted to said remote file storage system via said remote file server interface.

24. The system of claim 23, wherein said sync action handler utilizes application program interfaces to apply said file system operations to said file system and to transmit said file system operations to said remote file storage system.

25. The system of claim 17, wherein:

said synchronizer is further operative to define an event synchronization time period; and

said event processor is operative to generate file system operations based only on ones of said event records and ones of said remote event records that are associated with events occurring during said event synchronization time period.

26. The system of claim 25, wherein the duration of said event synchronization time period is adjustable.

27. The system of claim 17, wherein said file system and said remote file system are initially synchronized.

28. The system of claim 17, wherein said file system module is operative to:

monitor input/output (I/O) calls going to said file system; and

execute a trap for each of said I/O calls that will change said file system.

29. The system of claim 17, wherein said event processor is further operative to:

process said event records and said remote event records to generate a set of processed event records; and

generate file system operations based at least in part on said set of processed event records.

30. The server of claim 29, wherein, to generate said set of processed event records, said event processor is further operative to:

identify ones of said event records based on a type of event; and

modify at least some of said identified event records having said type.

31. The server of claim 30, wherein, to generate said set of processed event records, said event processor is further operative to delete some of said event records.

32. The server of claim 31, wherein, to generate said set of processed event records, said event processor is further operative to:

access a first event record associated with a first event occurring on a common file system object in said file system;

access a second event record associated with a second event occurring on said common file system object in said remote file system; and

generate file system operations that resolve a conflict between said first event and said second event.

33. A file storage system for use with a remote file storage system remotely located from said file storage system, said file storage system comprising:

memory for storing a file system including file system objects;

a client interface operative to enable a client to access said file system objects;

a file system module operative to monitor for changes to said file system by said client;

means for generating event records of changes being made to said file system as said changes occur;

means for receiving remote event records from said remote file storage system, said remote event records being indicative of changes made to a remote file system; and

means for generating file system operations that can be applied to said file system and said remote file system to synchronize said file system and said remote file system.

34. A method for synchronizing a file system (FS) and a remote file system (RFS) that is located remotely from said FS, said method comprising:

monitoring said FS for events, each of said events being indicative of a change made to said FS;

generating an event record in response to the occurrence of each of said events; and

transmitting said event record to a remote file storage system having access to said RFS.

35. A file storage system for use with a remote file storage system remotely located from said file storage system, said file storage system comprising:

memory for storing a file system including file system objects;

a client interface operative to enable a client to access said file system objects;

a file system module operative to monitor for changes being made said file system by said client;

a data monitor operative to generate an event record in response to a change being made to said file system;

a remote file server interface operative to establish a connection with said remote file storage system having access to a remote file system; and

a synchronizer operative to transmit said event record to said remote file storage system via said remote file server interface.

36. A file storage system comprising:

a local file storage system storing a local file system (LFS);

a remote file storage system storing a remote file system (RFS);

a local event monitor on said local file storage system, said local event monitor operative to generate local event records indicative of changes made to said LFS;

a remote event monitor on said remote file storage system operative to generate remote event records indicative of changes made to said RFS; and

an event processor on at least one of said local file storage system and said remote file storage system, said event processor being operative to use at least some of said local event records and at least some of said remote event records to synchronize said LFS and said RFS.