Cloud-native global file system with multi-site support using push classes

• US 10,423,573 B1
• Filed: 07/24/2018
• Issued: 09/24/2019
• Est. Priority Date: 07/24/2018
• Status: Active Grant

• Current Assignee: Nasuni Corporation
• Original Assignee: Nasuni Corporation
• Inventors: Shaw, David M., Rodriguez, Andres
• Primary Examiner(s): Stevens, Robert
• Application Number:
  US16/043,194
• Time in Patent Office: 427 Days
• Field of Search: 707705
• CPC Class Codes: G06F 16/128 Details of file system snapshots on the file-level, e.g. snapshot creation, administration, deletion use of snapshots for error detection or correction G06F11/14, G06F11/16
• CPC Class Codes: G06F 16/144 Query formulation
• CPC Class Codes: G06F 16/17 Details of further file system functions
• CPC Class Codes: G06F 16/182 Distributed file systems
• CPC Class Codes: G06F 16/1873 Versioning file systems, temporal file systems, e.g. file system supporting different historic versions of files
• CPC Class Codes: G06F 16/907 Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
• CPC Class Codes: G06F 3/0604 Improving or facilitating administration, e.g. storage management
• CPC Class Codes: G06F 3/0665 at area level, e.g. provisioning of virtual or logical volumes
• CPC Class Codes: G06F 3/067 Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
• CPC Class Codes: G06F 9/542 Event management; Broadcasting; Multicasting; Notifications

Abstract

A technique for data sharing among multiple filers that share a volume in a private or public cloud object store is implemented. In this approach, a mechanism is provided to enable a local filer to determine whether other filers that are sharing the volume have a consistent view of new data being written to the cloud object store by the local filer. The begins by associating together a collection of one or more files in a “push class.” On demand, a push operation for the push class is initiated on the local filer. Preferably, the push is managed according to one or more push criteria associated with the push class. Typically, the push operation pushes file data and metadata associated with the one or more files of the push class in respective phases, with the file data being pushed to the cloud during a first phase and the metadata associated with that file data being pushed during a second phase that follows the first phase. After the push operation completes, a determination is made whether the new version of the file is available at one or more other filers that share the volume.

Claims

1. A method of data sharing among multiple filers that share a volume in a private or public cloud object store, comprising:

configuring a push class that defines a collection of one or more files of the volume that, according to a user-defined criteria of interest, are to be selectively pushed to the cloud object store distinct from any point-in-time snapshot on the volume;

associating together the collection of one or more files in the push class as configured;

upon a given occurrence, initiating a push operation for the push class according to one or more push criteria, wherein the push operation is initiated on a particular filer and pushes to the cloud object store file data and metadata associated with the one or more files in respective first and second phases; and

upon completion of the push operation, determining whether the one or more files are available at one or more other filers that share the volume.

2. The method as described in claim 1 wherein determining whether the one or more files are available at the one or more other filers comprises:

querying for a push version of metadata associated with the push class; and

receiving a response from at least one of the other filers, the response having been generated by the other filer by querying the push version of a root of the volume;

wherein the response received from the other filer is indicative that the one or more files are available at the other filer.

3. The method as described in claim 2 wherein the volume root is at or above the push version.

4. The method as described in claim 1 wherein determining whether the one or more files are available at the one or more other filers comprises:

after the second phase is initiated, querying for a payload identifier associated with the push class; and

receiving a response from at least one of the other filers, the response including a payload identifier returned from the other filer; and

comparing the payload identifier associated with the push class with the payload identifier returned from the other filer to determine whether the one or more files are available at the other filer.

5. The method as described in claim 4 wherein a value of the payload identifier determines whether the one or more files are available at the other filer.

6. The method as described in claim 1 wherein the one or more push criteria are configurable.

7. The method as described in claim 1 wherein the given ones of the multiple filers are located remote from one another.

8. The method as described in claim 1 wherein the file data is pushed to the cloud object store in the first phase, and wherein metadata associated with the file data is pushed to the cloud object store in the second phase.

9. A method carried out in association with a private or public cloud object store in which a volume is being shared among filers across a number of sites, comprising:

configuring a push class that defines a collection of one or more files of the volume that, according to a user-defined criteria of interest, are to be selectively pushed to the cloud object store distinct from any point-in-time snapshot on the volume;

assembling a payload to be pushed from a source filer to one or more target filers that share the volume with the source filer, wherein a payload is comprises the set of one or more files that satisfy the user-defined criteria of interest;

pushing the payload from the source filer distinct from any point-in-time snapshot on the volume; and

providing a notification indicating that the payload has arrived at a particular target filer in a consistent manner.

10. The method as described in claim 9 wherein the user-defined criteria of interest being defined by one of:

a directory pattern to follow, and a temporal constraint.

11. The method as described in claim 9 wherein the notification is provided via an intermediary positioned between the source filer and the one or more target filers.

12. Apparatus, comprising:

a hardware processor;

computer memory holding computer program instructions executed by the hardware processor to facilitate data sharing among multiple filers that share a volume in a private or public cloud object store, the computer program instructions comprising program code operative to;

configure a push class that defines a collection of one or more files of the volume that, according to a user-defined criteria of interest, are to be selectively pushed to the cloud object store distinct from any point-in-time snapshot on the volume;

associate together the collection of one or more files in the push class as configured;

upon a given occurrence, initiate a push operation for the push class according to one or more push criteria, wherein the push operation is initiated on a particular filer and pushes to the cloud object store file data and metadata associated with the one or more files in respective first and second phases; and

upon completion of the push operation, determine whether the one or more files are available at one or more other filers that share the volume.