Sending interim notifications for namespace operations for a distributed filesystem

• US 10,291,705 B2
• Filed: 08/27/2015
• Issued: 05/14/2019
• Est. Priority Date: 09/10/2014
• Status: Active Grant

• Current Assignee: Panzura, Inc.
• Original Assignee: Panzura, Inc.
• Inventors: Jung, Steve Hyuntae, Mondal, Shishir, Taylor, John Richard, Lin, Yun
• Primary Examiner(s): Trost, IV, William G
• Assistant Examiner(s): Moorad, Imran
• Application Number:
  US14/838,037
• Publication Number:
  US 20160072888A1
• Time in Patent Office: 1,356 Days
• Field of Search: None
• CPC Class Codes: H04L 61/457 containing identifiers of data entities on a computer, e.g. file names
• CPC Class Codes: H04L 67/01 Protocols
• CPC Class Codes: H04L 67/06 specially adapted for file transfer, e.g. file transfer protocol [FTP]
• CPC Class Codes: H04L 67/1097 for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]

Abstract

The disclosed embodiments disclose techniques for sending interim notifications to clients of a distributed filesystem. Two or more cloud controllers collectively manage distributed filesystem data that is stored in one or more cloud storage systems; the cloud controllers ensure data consistency for the stored data, and each cloud controller caches portions of the distributed filesystem. During operation, a cloud controller receives a client request to perform a namespace operation upon a filesystem object. The cloud controller determines that it will need to contact at least one of another peer cloud controller or a cloud storage system to service the request, and sends an interim notification to the client to notify the client that the request is pending.

Claims

1. A computer-implemented method for sending interim notifications for a namespace operation to a client of a distributed filesystem, the method comprising:

collectively managing the data of the distributed filesystem using two or more cloud controllers, wherein collectively managing the data comprises storing the data for the distributed filesystem in one or more cloud storage systems, wherein the cloud controllers cache and ensure data consistency for data stored in the cloud storage systems;

receiving at a first cloud controller a request from the client to perform a namespace operation upon a filesystem object, wherein the client is configured to abort the request if the duration of the request exceeds a timeout interval;

determining that the first cloud controller needs to contact at least one of a second, distinct cloud controller or a cloud storage system to service the request by performing actions that comprise;

determining a specified level of consistency associated with the filesystem object and the type of namespace operation being requested by the client for the filesystem object; and

determining from at least one of the specified level of consistency and the type of filesystem object access that the request involves a remote request from the first cloud controller to at least one of the second cloud controller or the cloud storage system; and

sending an interim notification to the client to notify the client that the request is pending, wherein the first cloud controller is configured to send the interim notification to the client before the timeout interval expires to ensure that the client does not abort the request;

wherein sending the interim notification further comprises tracking a set of delays associated with accessing the second cloud controller, determining from the tracked information that the remote request will exceed the client timeout interval, and ensuring that the client receives the interim notification before the timeout interval is exceeded.

2. The computer-implemented method of claim 1,

wherein the namespace operation is a create operation and the filesystem object is a file;

wherein the first cloud controller contacts the second cloud controller to claim the file to ensure that the first cloud controller has exclusive access to the file for the create operation;

wherein the second cloud controller, upon determining that the file does not exist, creates a placeholder that reserves the requested namespace for the file and then notifies the first cloud controller that the namespace for the file has been reserved;

wherein the first cloud controller, upon receiving notification of the namespace reservation, grants a file handle for the file to the client, thereby allowing the client to write data to the new file, and

wherein sending the interim notification to the client facilitates ensuring that the remote operations can complete before the client abandons the namespace operation.

3. The computer-implemented method of claim 1,

wherein the namespace operation is a delete operation and the filesystem object is a file;

wherein the request from the client is a request to open the file with delete-on-close permissions;

wherein the first cloud controller contacts the second cloud controller to claim the file to ensure that the first cloud controller has exclusive access to the file for the delete operation;

wherein the second cloud controller sends confirmation to the first cloud controller that the file has been claimed;

wherein, upon receiving confirmation of the file being claimed from the second cloud controller, the first cloud controller grants a delete-on-close file handle to the client; and

wherein sending the interim notification to the client facilitates ensuring that the multiple remote operations can complete before the client abandons the namespace operation.

4. The computer-implemented method of claim 1,

wherein the namespace operation is a delete operation, wherein the filesystem object is a file;

wherein the request from the client is a request to open the file with delete-on-close permissions;

wherein the first cloud controller contacts the second cloud controller to ensure that the first cloud controller has exclusive access to the file for the delete operation;

wherein the second cloud controller determines that a third cloud controller is presently claiming the file and contacts the third cloud controller to determine whether any file handles are presently open for the file;

wherein the second cloud controller, upon receiving confirmation that no file handles are open for the file on the third cloud controller, marks the file for deletion, wherein a response from the third cloud controller that indicates that no file handles are open for the file further indicates that the third cloud controller now no longer claims the file;

wherein the second cloud controller notifies the first cloud controller that no other cloud controllers are presently claiming the file;

wherein the first cloud controller grants a delete-on-close file handle to the client; and

wherein sending the interim notification to the client facilitates ensuring that the multiple remote operations can complete before the client abandons the namespace operation.

5. The computer-implemented method of claim 1,

wherein the namespace operation is a rename operation;

wherein the filesystem object is a file;

wherein the namespace operation specifies the file as the source file and a target filename;

wherein the first cloud controller contacts the second cloud controller to ensure that a consistent delete operation can be performed for the file in the distributed filesystem and that a consistent create operation can be performed for the target filename in the distributed filesystem; and

wherein sending the interim notification to the client facilitates ensuring that the multiple remote operations can complete before the client abandons the namespace operation.

6. The computer-implemented method of claim 1,

wherein the namespace operation is a hierarchical operation that updates multiple filesystem objects that span multiple levels of a directory hierarchy in the distributed filesystem;

and

wherein sending the interim notification to the client facilitates ensuring that the multiple operations can complete before the client abandons the namespace operation.

7. The computer-implemented method of claim 1, wherein sending the interim notification to the client further comprises:

calculating an initial interval that maximizes the time available to perform the remote request that without exceeding the timeout interval; and

delaying sending the interim notification from the first cloud controller to the client until the initial interval has expired.

8. The computer-implemented method of claim 1, wherein sending the interim notification to the client further comprises:

determining that the remote request may involve delay that exceeds multiple client timeout intervals; and

sending multiple interim notifications to the client to indicate to the client that the request is still pending but needs additional time.

9. The computer-implemented method of claim 1,

wherein the client is configured to send asynchronous requests for namespace operations to the first cloud controller;

and

wherein, upon receiving the interim notification, the client proceeds to initiate other namespace operations via the first cloud controller instead of blocking on the request, thereby improving the performance of namespace operations for the client.

10. The computer-implemented method of claim 1,

wherein the two or more cloud controllers are configured to handle requests received from the client and the two or more cloud controllers asynchronously, thereby improving the throughput of namespace operations for the distributed filesystem;

and

wherein the first cloud controller does not block on remote operations and continues to process new incoming client requests, send interim notifications to clients, and initiate additional remote operations while waiting for a given remote request to complete.

11. A non-transitory computer-readable storage medium storing instructions that when executed by a computer cause the computer to perform a method for sending interim notifications for a namespace operation to a client of a distributed filesystem, the method comprising:

collectively managing the data of the distributed filesystem using two or more cloud controllers, wherein collectively managing the data comprises storing the data for the distributed filesystem in one or more cloud storage systems, wherein the cloud controllers cache and ensure data consistency for data stored in the cloud storage systems;

receiving at a first cloud controller a request from the client to perform a namespace operation upon a filesystem object, wherein the client is configured to abort the request if the duration of the request exceeds a timeout interval;

determining that the first cloud controller needs to contact at least one of a second, distinct cloud controller or a cloud storage system to service the request by performing actions that comprise;

determining a specified level of consistency associated with the filesystem object and the type of namespace operation being requested by the client for the filesystem object; and

determining from at least one of the specified level of consistency and the type of filesystem object access that the request involves a remote request from the first cloud controller to at least one of the second cloud controller or the cloud storage system; and

sending an interim notification to the client to notify the client that the request is pending, wherein the first cloud controller is configured to send the interim notification to the client before the timeout interval expires to ensure that the client does not abort the request;

wherein sending the interim notification further comprises tracking a set of delays associated with accessing the second cloud controller, determining from the tracked information that the remote request will exceed the client timeout interval, and ensuring that the client receives the interim notification before the timeout interval is exceeded.

12. A cloud controller that sends interim notifications for a namespace operation to a client of a distributed filesystem, comprising:

a processor;

a storage mechanism that stores metadata for the distributed filesystem; and

a storage management mechanism;

wherein two or more cloud controllers collectively manage the data of the distributed filesystem;

wherein the cloud controller is configured to receive a request from a client to perform a namespace operation upon a filesystem object, wherein the client is configured to abort the request if the duration of the request exceeds a timeout interval;

wherein the storage management mechanism determines that that the first cloud controller needs to contact at least one of a second, distinct cloud controller or a cloud storage system to service the request by performing actions that comprise;

determining a specified level of consistency associated with the filesystem object and the type of namespace operation being requested by the client for the filesystem object; and

determining from at least one of the specified level of consistency and the type of filesystem object access that the request involves a remote request from the cloud controller to at least one of the second cloud controller or the cloud storage system; and

wherein the cloud controller sends an interim notification to the client to notify the client that the request is pending, wherein the cloud controller is configured to send the interim notification to the client before the timeout interval expires to ensure that the client does not abort the request;

wherein sending the interim notification further comprises tracking a set of delays associated with accessing the second cloud controller, determining from the tracked information that the remote request will exceed the client timeout interval, and ensuring that the client receives the interim notification before the timeout interval is exceeded.