Sending interim notifications to a client of a distributed filesystem

• US 9,613,048 B2
• Filed: 09/10/2014
• Issued: 04/04/2017
• Est. Priority Date: 09/10/2014
• Status: Active Grant

• Current Assignee: Panzura, Inc.
• Original Assignee: Panzura, Inc.
• Inventors: Lin, Yun, Taylor, John Richard
• Primary Examiner(s): Tran, Jimmy H
• Application Number:
  US14/482,934
• Publication Number:
  US 20160072886A1
• Time in Patent Office: 937 Days
• CPC Class Codes: G06F 16/172 Caching, prefetching or hoarding of files
• CPC Class Codes: G06F 16/182 Distributed file systems
• CPC Class Codes: H04L 67/01 Protocols
• CPC Class Codes: H04L 67/1063 Discovery through centralising entities
• CPC Class Codes: H04L 67/1097 for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]
• CPC Class Codes: H04L 67/568 Storing data temporarily at an intermediate stage, e.g. caching

Abstract

The disclosed embodiments disclose techniques for sending interim notifications to a client of a distributed filesystem. Two or more cloud controllers collectively manage distributed filesystem data that is stored in one or more cloud storage systems; the cloud controllers ensure data consistency for the stored data, and each cloud controller caches portions of the distributed filesystem. During operation, a cloud controller receives a client request to access a file. The cloud controller determines that it will need to contact at least one of another peer cloud controller or a cloud storage system to service the request, and sends an interim notification to the client to notify the client that the request is pending.

Claims

1. A computer-implemented method for sending interim notifications to a client of a distributed filesystem, the method comprising:

collectively managing the data of the distributed filesystem using two or more cloud controllers, wherein collectively managing the data comprises storing the data for the distributed filesystem in one or more cloud storage systems, wherein the cloud controllers cache and ensure data consistency for data stored in the cloud storage systems;

receiving at a first cloud controller a request from the client to access a file, wherein the client is configured to abort the request if the duration of the request exceeds a timeout interval;

determining from at least one of a level of consistency associated with the file and the type of file access being requested by the client for the file that the first cloud controller needs to initiate a remote request to contact at least one of a second, distinct cloud controller or a cloud storage system to service the request, wherein the first cloud controller determines that the request comprises a write operation that accesses the most recently updated version of the file, wherein the second cloud controller is the owning cloud controller for the file, wherein the remote request comprises contacting the second cloud controller to synchronize recent updates for the file to the first cloud controller and to claim ownership for the file to facilitate client write access for the file; and

sending an interim notification from the first cloud controller to the client before the timeout interval expires to notify the client that the request is pending and to ensure that the client does not abort the request, wherein sending the interim notification comprises tracking a set of delays associated with accessing the second cloud controller, determining from the tracked information that the remote request is likely to exceed the client timeout interval, and ensuring that the client receives the interim notification before the timeout interval is exceeded.

2. The computer-implemented method of claim 1, wherein sending the interim notification to the client further comprises:

calculating an initial interval that gives the remote request as much additional time as possible while also ensuring that the timeout interval is not exceeded; and

delaying sending the interim notification from the first cloud controller to the client until the initial interval has expired.

3. The computer-implemented method of claim 1, wherein sending the interim notification to the client further comprises:

determining that the remote request may involve substantial delay that exceeds multiple client timeout intervals; and

sending multiple interim notifications to the client to indicate to the client that the request is still pending but needs additional time.

4. The computer-implemented method of claim 1,

wherein the client is configured to send asynchronous file access requests to the first cloud controller;

and

wherein, upon receiving the interim notification, the client proceeds to initiate other file operations and file requests to the first cloud controller instead of busy-waiting for the request, thereby improving file access performance for the client.

5. The computer-implemented method of claim 1,

wherein cloud controllers are configured to handle client and cloud controller requests asynchronously, thereby improving file access performance for the distributed filesystem;

and

wherein the first cloud controller does not block on remote operations and continues to process new incoming client requests, send interim notifications to clients, and initiate additional remote operations while waiting for a given remote request to complete.

6. A non-transitory computer-readable storage medium storing instructions that when executed by a computer cause the computer to perform a method for sending interim notifications to a client of a distributed filesystem, the method comprising:

collectively managing the data of the distributed filesystem using two or more cloud controllers, wherein collectively managing the data comprises storing the data for the distributed filesystem in one or more cloud storage systems, wherein the cloud controllers cache and ensure data consistency for data stored in the cloud storage systems;

receiving at a first cloud controller a request from the client to access a file, wherein the client is configured to abort the request if the duration of the request exceeds a timeout interval;

determining from at least one of a level of consistency associated with the file and the type of file access being requested by the client for the file that the first cloud controller needs to initiate a remote request to contact at least one of a second, distinct cloud controller or a cloud storage system to service the request, wherein the first cloud controller determines that the request comprises a write operation that accesses the most recently updated version of the file, wherein the second cloud controller is the owning cloud controller for the file, wherein the remote request comprises contacting the second cloud controller to synchronize recent updates for the file to the first cloud controller and to claim ownership for the file to facilitate client write access for the file; and

sending an interim notification from the first cloud controller to the client before the timeout interval expires to notify the client that the request is pending and to ensure that the client does not abort the request, wherein sending the interim notification comprises tracking a set of delays associated with accessing the second cloud controller, determining from the tracked information that the remote request is likely to exceed the client timeout interval, and ensuring that the client receives the interim notification before the timeout interval is exceeded.

7. The non-transitory computer-readable storage medium of claim 6, wherein sending the interim notification to the client further comprises:

calculating an initial interval that gives the remote request as much additional time as possible while also ensuring that the timeout interval is not exceeded; and

delaying sending the interim notification from the first cloud controller to the client until the initial interval has expired.

8. The non-transitory computer-readable storage medium of claim 6, wherein sending the interim notification to the client further comprises:

determining that the remote request may involve substantial delay that exceeds multiple client timeout intervals; and

sending multiple interim notifications to the client to indicate to the client that the request is still pending but needs additional time.

9. The non-transitory computer-readable storage medium of claim 6,

wherein the client is configured to send asynchronous file access requests to the first cloud controller;

and

wherein, upon receiving the interim notification, the client proceeds to initiate other file operations and file requests to the first cloud controller instead of busy-waiting for the request, thereby improving file access performance for the client.

10. A cloud controller that sends interim notifications to a client of a distributed filesystem, comprising:

a processor; and

a local storage device that stores a copy of the metadata for the distributed filesystem;

wherein two or more cloud controllers collectively manage the data of the distributed filesystem;

wherein the cloud controller;

receives a request from the client to access a file, wherein the client is configured to abort the request if the duration of the request exceeds a timeout interval;

determines from at least one of a level of consistency associated with the file and the type of file access being requested by the client for the file that the cloud controller needs to initiate a remote request to contact at least one of a second, distinct cloud controller or a cloud storage system to service the request, wherein the cloud controller determines that the request comprises a write operation that accesses the most recently updated version of the file, wherein the second cloud controller is the owning cloud controller for the file, wherein the remote request comprises contacting the second cloud controller to synchronize recent updates for the file to the cloud controller and to claim ownership for the file to facilitate client write access for the file; and

sends an interim notification to the client before the timeout interval expires to notify the client that the request is pending and to ensure that the client does not abort the request, wherein sending the interim notification comprises tracking a set of delays associated with accessing the second cloud controller, determining from the tracked information that the remote request is likely to exceed the client timeout interval, and ensuring that the client receives the interim notification before the timeout interval is exceeded.