Sharing and synchronizing electronically stored files

• US 9,239,846 B2
• Filed: 04/23/2012
• Issued: 01/19/2016
• Est. Priority Date: 04/23/2012
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Besen, Adam, Catmull, David, Cheong, Hwi, DeNeui, Alexander, Mueller, Hendrik, Schneider, Ronald, Sharma, Rishi, Vasishth, Himanshu, Wurtz, David, Pape, Frank III
• Primary Examiner(s): LEWIS, CHERYL RENEA
• Application Number:
  US13/453,678
• Publication Number:
  US 20130282657A1
• Time in Patent Office: 1,366 Days
• US Class Current: 1/1
• CPC Class Codes: G06F 16/178 Techniques for file synchronisation in file systems
• CPC Class Codes: G06F 16/182 Distributed file systems
• CPC Class Codes: G06F 16/1844 Management specifically adapted to replicated file systems

Abstract

Aspects of the present disclosure are directed to architectures, methods and systems and structures that facilitate the sharing and synchronization of electronically stored files among and between cloud entities and a number of computers, systems, devices and/or users. One particular exemplary architectural aspect includes the concurrent determination of file system changes within a cloud file system and a client file system, the serial ordering of necessary file system operations in response to the determined file system changes, and the concurrent execution of file system operations such that the cloud file system and the client computer file system are synchronized.

Claims

1. A computer implemented system for synchronizing electronically stored files between a client file system and a cloud file system comprising:

a cloud watcher processor that receives a change log indicative of the status of the cloud file system, detects changes to the cloud file system, and generates work items in response to the detected cloud file system changes;

a local watcher processor that concurrently monitors the client file system and detects changes thereto and generates work items in response to the detected client file system changes;

a fetcher processor that serializes selected ones of the work items and assigns each of the selected ones of the work items to only one of a plurality of worker processors that concurrently perform assigned work items, wherein the fetcher processor references a dependency map prior to assigning selected work items to worker processors;

wherein the work items are file operations resulting in a synchronization between the client file system and the cloud file system.

2. The computer implemented system of claim 1 further comprising an event aggregator processor that aggregates a number of the detected client file system changes such that the aggregated changes are used to generate work items.

3. The computer implemented system of claim 1 further comprising a snapshot that indicates a current synchronization state between the client file system and the cloud file system.

4. The computer implemented system of claim 1 further comprising a blacklist of work items that are not to be assigned to worker processors.

5. The computer implemented system of claim 1 further comprising a local graph that indicates a state of the local file system.

6. The computer implemented system of claim 1 further comprising a cloud graph that indicates a state of the cloud file system.

7. The computer implemented system of claim 1 wherein the dependency map includes indications of particular inodes that are affected by current worker operations.

8. The computer implemented system of claim 1 wherein the dependency map includes indications of particular resourceIDs that are affected by current worker operations.

9. The computer implemented system of claim 1 wherein the dependency map includes filenames of files that are affected by current worker operations.

10. The computer implemented system of claim 1 wherein the fetcher processor includes a work queue of work items that are ordered from oldest to newest.

11. The computer implemented system of claim 1, wherein the fetcher processor assigns each of the selected ones of the work items to only one of the plurality of worker processors to prevent any two worker processors in the plurality of work processors from operating on conflicting or overlapping changes.

12. A computer implemented method for synchronizing electronically stored files between a client file system and a cloud file system comprising the steps of:

receiving a change log indicative of the status of the cloud file system, detecting changes to the cloud file system, and generating work items in response to the detected cloud file system changes;

concurrently monitoring the client file system, detecting changes to one or more files electronically stored in the client file system, and generating work items in response to the detected client file system changes;

serially fetching the generated work items such that each of the generated work items is assigned to only one of a plurality of worker processors, wherein a dependency map is referenced prior to assigning selected work items to worker processors; and

concurrently working the fetched work items at the assigned worker processors, such that changes detected to a file in one of the client file system or cloud file system are replicated to a file of the other file system.

13. The computer implemented system of claim 12 further comprising the step of aggregating a number of the detected client file system changes and then using the aggregated changes to generate work items.

14. The computer implemented method of claim 12 further comprising the step of comparing a current state of the cloud file system to a graph representing a previous state of the file system such that any changes to files in the file system are determined.

15. The computer implemented method of claim 12 wherein the fetching of work items includes ordering the work items in a work queue from oldest to newest order.

16. The computer implemented method of claim 12 wherein the fetching of work items includes determining whether a particular work item in the work queue can be processed.

17. The computer implemented method of claim 16 wherein the fetching of work items includes maintaining the ordering of the work items in the work queue from oldest to newest order.

18. A computer storage medium having computer executable instructions which when executed by a computer cause the computer to perform operations comprising:

receiving a change log indicative of the status of the cloud file system, detecting changes to the cloud file system, and generating file system operations in response to the detected cloud file system changes;

concurrently monitoring the client file system, detecting changes to one or more files electronically stored in the client file system, aggregating a number of the client file system changes, and using the aggregated changes to generate file system operations in response to the detected client file system changes;

serially ordering the file system operations from oldest to newest;

assigning each of the file system operations to only one of a plurality of worker processors, wherein a dependency map is referenced prior to assigning selected work items to worker processors; and

concurrently performing the file system operations at assigned worker processors, such that changes detected to a file in one of the file systems are replicated a file of the other file system.

19. The computer storage medium of claim 18 which further causes the computer to perform the following operation comprising:

comparing a current state of the cloud file system to a graph such that any changes to files in the cloud file system are determined.

20. The computer storage medium of claim 19 which further causes the computer to perform the following operation comprising:

comparing the determined changes to a snapshot structure that is indicative of a current synchronization state between the client file system and the cloud file system.