Synchronizing file updates between two cloud controllers of a distributed filesystem

• US 9,805,056 B2
• Filed: 06/24/2014
• Issued: 10/31/2017
• Est. Priority Date: 06/24/2014
• Status: Active Grant

• Current Assignee: Panzura, Inc.
• Original Assignee: Panzura, Inc.
• Inventors: Parkison, Brian Christopher, Davis, Andrew P., Taylor, John Richard
• Primary Examiner(s): Ruiz, Angelica
• Application Number:
  US14/313,703
• Publication Number:
  US 20150370827A1
• Time in Patent Office: 1,225 Days
• Field of Search: 707600-831, 707899, 707999001-999206
• CPC Class Codes: G06F 11/1435 using file system or storage system metadata
• CPC Class Codes: G06F 11/2089 Redundant storage control functionality
• CPC Class Codes: G06F 11/2094 Redundant storage or storage space G06F11/2056 takes precedence
• CPC Class Codes: G06F 16/178 Techniques for file synchronisation in file systems
• CPC Class Codes: G06F 16/1844 Management specifically adapted to replicated file systems
• CPC Class Codes: G06F 2201/85 Active fault masking without idle spares active fault masking without idle spare hardware where processing functionality is redundant G06F11/2035

Abstract

The disclosed embodiments disclose techniques for synchronizing file updates between two cloud controllers of a distributed filesystem. Two or more cloud controllers collectively manage distributed filesystem data that is stored in the cloud storage systems; the cloud controllers ensure data consistency for the stored data, and each cloud controller caches portions of the distributed filesystem. During operation, a cloud controller receives a request from a client to access a file in the distributed filesystem. The cloud controller sends a synchronization update request for the file to a second cloud controller and in response receives a synchronization update for the file from the second cloud controller.

Claims

1. A computer-implemented method for synchronizing file updates between two cloud controllers of a distributed filesystem, the method comprising:

collectively managing the data of the distributed filesystem using two or more cloud controllers, wherein collectively managing the data comprises storing the data for the distributed filesystem in one or more cloud storage systems, wherein the cloud controllers cache and ensure data consistency for data stored in the cloud storage systems;

receiving at a first cloud controller a client request to access a file in the distributed filesystem, wherein cloud controllers track version information for each file, wherein the tracked version information for the file on a cloud controller is updated whenever the cloud controller either updates the metadata for the file in response to a client write or receives a metadata update for the file from another cloud controller;

sending a synchronization update request for the file to a second cloud controller, wherein sending the synchronization update request comprises;

sending a version identifier for the most recent version of the file currently available on the first cloud controller to the second cloud controller; and

using the version identifier to determine that the file has been modified on the second cloud controller and that the version of the file currently available on the first cloud controller is out-of-date

wherein upon receiving the synchronization update request the second cloud controller is configured to;

use the version identifier to determine a set of metadata deltas that are needed to update the out-of-date version of the file on the first cloud controller to the updated version of the file on the second cloud controller;

compress the set of metadata deltas; and

send the compressed set of metadata deltas to the first cloud controller in a synchronization update; and

receiving the synchronization update for the file from the second cloud controller in response to the synchronization update request;

wherein using the version identifier to determine the set of metadata deltas facilitates determining the differences between the out-of-date file version and the current file version without needing to cache or download all of the data for the file on either cloud controller.

2. The computer-implemented method of claim 1, wherein upon receiving the synchronization update request the second cloud controller is further configured to:

determine a set of updated file data for the file that is referenced in the set of metadata deltas but not yet stored in a cloud storage system; and

send the set of updated file data to the first cloud controller in the synchronization update.

3. The computer-implemented method of claim 1:

wherein the client request is a request to write to the file;

wherein sending the synchronization update request for the file comprises determining that the second cloud controller is presently managing access to the file and sending the version identifier for the file to the second cloud controller in a write lock request for the file; and

wherein initiating the synchronization update by including the version identifier in the write lock request ensures that the client will be writing the most recent version of the file.

4. The computer-implemented method of claim 1, wherein sending a synchronization update request further comprises:

determining that the file needs strong read-after-write consistency;

sending a registration notification from the first cloud controller to the other cloud controllers of the distributed filesystem, wherein the registration notification requests that the first cloud controller receive immediate notification of any changes to the file; and

receiving a change notification message for the file from the second cloud controller, wherein the change notification message includes an updated version identifier for the file that is more recent than the version identifier on the first cloud controller;

wherein sending the synchronization update request to the second cloud controller facilitates ensuring that the client accesses the most recent version of the file.

5. The computer-implemented method of claim 4, wherein sending the synchronization update request comprises determining that the first cloud controller has not received an incremental metadata snapshot containing the metadata for the file that is associated with the updated version identifier.

6. The computer-implemented method of claim 5, wherein the method further comprises:

subsequently receiving an incremental metadata snapshot that includes the set of metadata deltas for the file; and

determining from an updated version identifier for the file that was received in the synchronization update that the current local metadata version of the file is up-to-date and that the incremental metadata snapshot for the file does not need to be applied;

wherein synchronization updates facilitate synchronizing file updates between cloud controllers for scenarios in which lazy incremental metadata snapshots do not synchronize file updates between cloud controllers quickly enough to support efficient collaborative editing of the file by distributed clients.

7. The computer-implemented method of claim 1, wherein the method further comprises:

determining a set of additional files that are related to the file and are likely to be modified in conjunction with the file; and

sending pre-emptive synchronization update requests for the set of additional files to the owning cloud controller for the related files to ensure that the first cloud controller has already synchronized the updated versions of the related files prior to a subsequent client request for the related files;

wherein pre-emptively synchronizing related files facilitates reducing time delays for distributed clients that are iteratively editing the related files.

8. The computer-implemented method of claim 7, wherein determining the set of additional files that are related to the file involves one or more of the following:

receiving a pre-specified file grouping that indicates a set of related files that are likely to be accessed in conjunction;

tracking client and client application behavior to determine a set of files that are likely to be accessed in conjunction;

determining a set of files that are likely to be accessed in conjunction based on spatial and temporal locality; and

scanning the file to determine a set of files that are referenced in the file'"'"'s contents and hence are likely to be accessed in conjunction with the file.

9. A non-transitory computer-readable storage medium storing instructions that when executed by a computer cause the computer to perform a method for synchronizing file updates between two cloud controllers of a distributed filesystem, the method comprising:

collectively managing the data of the distributed filesystem using two or more cloud controllers, wherein collectively managing the data comprises storing the data for the distributed filesystem in one or more cloud storage systems, wherein the cloud controllers cache and ensure data consistency for data stored in the cloud storage systems;

receiving at a first cloud controller a client request to access a file in the distributed filesystem, wherein cloud controllers track version information for each file, wherein the tracked version information for the file on a cloud controller is updated whenever the cloud controller either updates the metadata for the file in response to a client write or receives a metadata update for the file from another cloud controller;

sending a synchronization update request for the file to a second cloud controller, wherein sending the synchronization update request comprises;

sending a version identifier for the most recent version of the file currently available on the first cloud controller to the second cloud controller; and

using the version identifier to determine that the file has been modified on the second cloud controller and that the version of the file currently available on the first cloud controller is out-of-date

wherein upon receiving the synchronization update request the second cloud controller is configured to;

use the version identifier to determine a set of metadata deltas that are needed to update the out-of-date version of the file on the first cloud controller to the updated version of the file on the second cloud controller;

compress the set of metadata deltas; and

send the compressed set of metadata deltas to the first cloud controller in a synchronization update; and

receiving the synchronization update for the file from the second cloud controller in response to the synchronization update request;

wherein using the version identifier to determine the set of metadata deltas facilitates determining the differences between the out-of-date file version and the current file version without needing to cache or download all of the data for the file on either cloud controller.

10. The non-transitory computer-readable storage medium of claim 9, wherein upon receiving the synchronization update request the second cloud controller is further configured to:

determine a set of updated file data for the file that is referenced in the set of metadata deltas but not yet stored in a cloud storage system; and

send the set of updated file data to the first cloud controller in the synchronization update.

11. The non-transitory computer-readable storage medium of claim 9, wherein the client request is a request to write to the file;

wherein sending the synchronization update request for the file comprises determining that the second cloud controller is presently managing access to the file and sending the version identifier for the file to the second cloud controller in a write lock request for the file; and

wherein initiating the synchronization update by including the version identifier in the write lock request ensures that the client will be writing the most recent version of the file.

12. The non-transitory computer-readable storage medium of claim 9, wherein sending a synchronization update request further comprises:

determining that the file needs strong read-after-write consistency;

sending a registration notification from the first cloud controller to the other cloud controllers of the distributed filesystem, wherein the registration notification requests that the first cloud controller receive immediate notification of any changes to the file; and

receiving a change notification message for the file from the second cloud controller, wherein the change notification message includes an updated version identifier for the file that is more recent than the version identifier on the first cloud controller;

wherein sending the synchronization update request to the second cloud controller facilitates ensuring that the client accesses the most recent version of the file.

13. The non-transitory computer-readable storage medium of claim 12, wherein sending the synchronization update request comprises determining that the first cloud controller has not received an incremental metadata snapshot containing the metadata for the file that is associated with the updated version identifier.

14. A cloud controller that receives a synchronization update from a second cloud controller in a distributed filesystem, comprising:

a processor;

a storage mechanism that stores metadata for the distributed filesystem; and

a storage management mechanism;

wherein two or more cloud controllers collectively manage the data of the distributed filesystem;

wherein the cloud controller is configured to receive a request from a client to access a file in the distributed filesystem, wherein cloud controllers track version information for each file, wherein the tracked version information for the file on the cloud controller is updated whenever the cloud controller either updates the metadata for the file in response to a client write or receives a metadata update for the file from another cloud controller;

wherein the storage management mechanism sends a synchronization update request for the file to a second cloud controller, wherein sending the synchronization update request comprises;

sending a version identifier for the most recent version of the file currently available on the cloud controller to the second cloud controller; and

using the version identifier to determine that the file has been modified on the second cloud controller and that the version of the file currently available on the cloud controller is out-of-date;

wherein upon receiving the synchronization update request the second cloud controller is configured to;

use the version identifier to determine a set of metadata deltas that are needed to update the out-of-date version of the file on the cloud controller to the updated version of the file on the second cloud controller;

compress the set of metadata deltas; and

send the compressed set of metadata deltas to the cloud controller in a synchronization update;

wherein the storage management mechanism is configured to receive the synchronization update for the file from the second cloud controller in response to the synchronization update request; and

wherein using the version identifier to determine the set of metadata deltas facilitates determining the differences between the out-of-date file version and the current file version without needing to cache or download all of the data for the file on either cloud controller.