System and method for event-based synchronization of remote and local file systems

• US 10,558,620 B2
• Filed: 01/26/2018
• Issued: 02/11/2020
• Est. Priority Date: 08/03/2012
• Status: Active Grant

• Current Assignee: Egnyte Incorporated
• Original Assignee: Egnyte Incorporated
• Inventors: Koos, Remus, White, Ray, Marathe, Manish, Mielityinen, Markku, Jassal, Amrit, Ram, Rajesh, Gopalan, Sriram, Aahz, Toivonen, Heikki J., Wijayaratne, Ravi
• Primary Examiner(s): Beausoliel, Jr., Robert W
• Assistant Examiner(s): Ganger, Arthur
• Application Number:
  US15/881,269
• Publication Number:
  US 20180150476A1
• Time in Patent Office: 746 Days
• Field of Search: 707625
• CPC Class Codes: G06F 16/11 File system administration, e.g. details of archiving or snapshots file system backup G06F11/14
• CPC Class Codes: G06F 16/178 Techniques for file synchronisation in file systems

Abstract

A method for synchronizing a file system (FS) and a remote file system (RFS) includes monitoring the FS for FS events, generating FS event records, receiving RFS event records of RFS events, generating file system operations (FSOs) based on the FS and RFS event records, and communicating the FSOs to the FS and RFS to synchronize them. A method for generating the FSOs includes accessing a plurality of FS and/or RFS event records, processing the accessed records to generate processed event records, generating the FSOs based on the processed event records, and outputting the FSOs to cause synchronization of the FS and RFS. Systems are also described. The invention facilitates event-based, steady-state synchronization of local and remote file systems.

Claims

1. A method for generating file system operations for synchronizing a local file system (LFS) and a remote file system (RFS) that is remotely located from said LFS, said method comprising:

accessing a plurality of event records including LFS event records and RFS event records, each of said LFS event records corresponding to a change previously made to said LFS and each of said RFS event records corresponding to a change previously made to said RFS;

processing said LFS event records and said RFS event records together to generate a set of processed event records including both processed ones of said LFS event records and processed ones of said RFS event records;

generating file system operations based at least in part on said set of processed event records, said file system operations including LFS operations operative to cause changes to said LFS and RFS operations operative to cause changes to said RFS;

applying said LFS operations to said LFS; and

providing said RFS operations to a remote file storage system having access to said RFS, whereby said remote file storage system can apply said RFS operations to alter said RFS; and

wherein

portions of said LFS and said RFS are synchronized as said file system operations are applied to said LFS and said RFS.

2. The method of claim 1, wherein said step of processing said LFS event records and said RFS event records together comprises:

first, processing said LFS event records independently from said RFS event records; and

second, jointly processing said LFS and said RFS event records.

3. The method of claim 1, wherein said step of processing said LFS event records and said RFS event records together comprises:

identifying ones of said LFS and said RFS event records that are associated with particular file system paths; and

processing said ones of said LFS and said RFS event records that are associated with said particular file system paths on a path-by-path basis.

4. The method of claim 1, wherein said step of generating file system operations based at least in part on said set of processed event records comprises:

categorizing at least some of said set of processed event records; and

generating said file system operations based at least in part on said categorization.

5. The method of claim 4, wherein said step of categorizing at least some of said set of processed event records comprises categorizing ones of said LFS event records, each of said categorized LFS event records being associated with a common file system object synchronized on said LFS and said RFS and having been categorized based on a number of said RFS event records that are also associated with said common file system object.

6. The method of claim 5, wherein:

if none of said RFS event records are associated with said common file system object, said LFS event record is categorized as an independent LFS event record; and

said step of generating file system operations based at least in part on said categorization includes generating at least one file system operation based on said independent LFS event record alone.

7. The method of claim 5, wherein:

if only one of said RFS event records is associated with said common file system object, said LFS event record is categorized as a simple dependent LFS event record; and

said step of generating file system operations based at least in part on said categorization includes utilizing conflict resolution to generate at least one file system operation based on said simple dependent LFS event record and said one RFS event record associated with said common file system object.

8. The method of claim 7, wherein said step of utilizing conflict resolution to generate at least one file system operation based on said simple dependent LFS event record includes using a lookup table to determine at least one operation to generate for resolving a conflict between said simple dependent LFS event and said one RFS event record associated with said common file system object.

9. The method of claim 7, wherein said step of utilizing conflict resolution to generate at least one file system operation based on said simple dependent LFS event record includes:

determining that at least one of said simple dependent LFS event records corresponds to a combination of events that was in error and should not have occurred; and

executing a different file system re-synchronization process in response to said error.

10. The method of claim 5, wherein:

if more than one of said RFS event records is associated with said common file system object, said LFS event record is categorized as a complex dependent LFS event record; and

said step of generating file system operations based at least in part on said categorization includes initiating a limited rescan synchronization of a portion of said LFS and said RFS associated with said common file system object.

11. A server for generating file system operations for synchronizing a local file system (LFS) and a remote file system (RFS) that is located remotely from said server, said server comprising:

a processor operative to execute code;

memory for storing data and said code;

an event database storing a plurality of event records including LFS event records and RFS event records, each of said LFS event records corresponding to a change previously made to said LFS and each of said RFS event records corresponding to a change previously made to said RFS;

an event processor operative to

process said LFS event records and said RFS event records together to generate a set of processed event records including both processed ones of said LFS event records and processed ones of said RFS event records and

generate file system operations using said set of processed event records, said file system operations including LFS operations operative to cause changes to said LFS and RFS operations operative to cause changes to said RFS; and

a local file system handler operative to

apply said LFS operations to said LFS to alter said LFS and

provide said RFS operations to a remote file storage system having access to said RFS, whereby said remote file storage system can apply said RFS operations to alter said RFS; and

wherein

portions of said LFS and said RFS are synchronized as said file system operations are applied to said LFS and said RFS.

12. The server of claim 11, wherein said event processor is operative to:

first, process said LFS event records independently from said RFS event records; and

second, jointly process said LFS and said RFS event records.

13. The server of claim 11, wherein said event processor is operative to:

identify ones of said LFS and said RFS event records that are associated with particular file system paths; and

process said ones of said LFS and said RFS event records that are associated with said particular file system paths on a path-by-path basis.

14. The server of claim 11, wherein, to generate said file system operations, said event processor is operative to:

categorize at least some of said set of processed event records; and

generate said file system operations based at least in part on said categorization.

15. The server of claim 14, wherein said event processor is further operative to categorize ones of said LFS event records of said set of processed event records, each of said categorized LFS event records being associated with a common file system object synchronized on said LFS and said RFS and having been categorized based on a number of said RFS event records that are also associated with said common file system object.

16. The server of claim 15, wherein said event processor is further operative to:

categorize said LFS event record as an independent LFS event record if none of said RFS event records are associated with said common file system object; and

generate at least one file system operation based on said independent LFS event record alone.

17. The server of claim 15, wherein said event processor is further operative to:

categorize said LFS event record associated with said common file system object as a simple dependent LFS event record if only one of said RFS event records is associated with said common file system object; and

utilize conflict resolution to generate at least one file system operation based on said simple dependent LFS event record and said one RFS event record associated with said common file system object.

18. The server of claim 17, wherein said memory includes a lookup table accessible by said event processor to determine at least one action to take for resolving a conflict when utilizing conflict resolution.

19. The server of claim 17, wherein, to resolve a conflict when utilizing conflict resolution, said event processor is operative to:

determine that said simple dependent LFS event record is associated with a combination of events that was in error and should not have occurred; and

initiate a different file system re-synchronization process in response to said error.

20. The server of claim 15, wherein said event processor is further operative to:

categorize said LFS event record as a complex dependent LFS event record if more than one RFS event record is associated with said common file system object; and

initiate a limited rescan synchronization of a portion of said LFS and said RFS associated with said common file system object.

21. A server for generating file system operations for synchronizing a local file system (LFS) and a remote file system (RFS) that is located remotely from said server, said server comprising:

memory storing a plurality of event records including LFS event records and RFS event records, each of said LFS event records corresponding to a change previously made to said LFS and each of said RFS event records corresponding to a change previously made to said RFS;

means for processing said LFS event records and said RFS event records together to generate a set of processed event records including both processed ones of said LFS event records and processed ones of said RFS event records;

means for generating file system operations based at least in part on said set of processed event records by categorizing ones of said processed event records, said file system operations including LFS operations operative to cause changes to said LFS and RFS operations operative to cause changes to said RFS; and

a local file system handler operative to

apply said LFS operations to said LFS to alter said LFS and

provide said RFS operations to a remote file storage system having access to said RFS, whereby said remote file storage system can apply said RFS operations to alter said RFS; and

wherein

portions of said LFS and said RFS are synchronized as said file system operations are applied to said LFS and said RFS.