Pre-fetching data for a distributed filesystem

• US 8,788,628 B1
• Filed: 11/14/2011
• Issued: 07/22/2014
• Est. Priority Date: 11/14/2011
• Status: Active Grant

• Current Assignee: Panzura, Inc.
• Original Assignee: Panzura, Inc.
• Inventors: Taylor, John Richard, Chou, Randy Yen-pang, Davis, Andrew P.
• Primary Examiner(s): Bates, Kevin
• Assistant Examiner(s): RAHMAN, SM AZIZUR
• Application Number:
  US13/295,844
• Time in Patent Office: 981 Days
• Field of Search: 709/219
• US Class Current: 709/219
• CPC Class Codes: G06F 16/172 Caching, prefetching or hoarding of files
• CPC Class Codes: G06F 16/182 Distributed file systems

Abstract

The disclosed embodiments provide a system that facilitates pre-fetching data for a distributed filesystem. During operation, a cloud controller (e.g., a computing device that caches data from the distributed filesystem) that maintains a set of metadata for the distributed filesystem receives a request to access a data block for a file. The cloud controller traverses the metadata to identify a metadata entry that is associated with the block, and then uses this metadata entry to download a cloud file containing the data block from a cloud storage system. While performing these operations, the cloud controller additionally determines that an additional cloud file in the cloud storage system includes data that is likely to be accessed in conjunction with the data block, and proceeds to pre-fetch this additional cloud file from the cloud storage system.

Claims

1. A computer-implemented method for pre-fetching data in a distributed filesystem, the method comprising:

collectively managing data coherency for the data of the distributed filesystem using two or more cloud controllers by;

collectively presenting a unified namespace for the distributed filesystem to the clients of the distributed filesystem via the two or more cloud controllers, wherein the clients can only access the distributed filesystem via the cloud controllers, wherein the file data for the distributed filesystem is stored in a remote cloud storage system using encrypted cloud files, wherein each cloud controller caches a subset of the file data from the remote cloud storage system that is being actively accessed by that cloud controller'"'"'s respective clients, wherein all new file data received by each cloud controller from its clients is written to the remote cloud storage system via the receiving cloud controller;

maintaining at each cloud controller a copy of the complete metadata for all of the files stored in the distributed filesystem, wherein each cloud controller communicates any changes to the metadata for the distributed filesystem to the full set of cloud controllers for the distributed filesystem to ensure that the clients of the distributed filesystem share a consistent view of each file in the distributed filesystem;

upon receiving in a cloud controller new file data from a client, storing the new file data for the distributed filesystem as cloud files in the remote cloud storage system;

upon receiving confirmation that the new cloud files have been successfully stored in the remote cloud storage system, sending from the cloud controller an incremental metadata snapshot that includes new metadata for the distributed filesystem that describes the new file data and links to the new cloud files, wherein the incremental metadata snapshot is received by the other cloud controllers of the distributed filesystem and used to ensure data coherency for the distributed filesystem;

receiving at the cloud controller a request from a client to access a data block for a file;

traversing the cloud controller'"'"'s copy of the metadata for the distributed filesystem to identify a metadata entry associated with the data block;

using the metadata entry to download a cloud file containing the data block from the remote cloud storage system to the cloud controller;

determining that an additional cloud file in the remote cloud storage system includes data that is likely to be accessed in conjunction with the data block; and

pre-fetching the additional cloud file from the remote cloud storage system to the cloud controller.

2. The computer-implemented method of claim 1,

wherein cloud files comprise logical storage volumes in the cloud storage system that store data and meta-data for the distributed filesystem;

and

wherein the cloud controller manages the layout of data being written into cloud files to improve the performance of subsequent accesses.

3. The computer-implemented method of claim 2,

wherein each cloud controller participating in the distributed filesystem maintains a targeted set of meta-data for the distributed filesystem;

wherein managing the layout of data comprises storing meta-data and data in separate cloud files; and

wherein one or more additional cloud controllers download a meta-data cloud file that includes new meta-data created by the first cloud controller to ensure that the targeted set of meta-data tracked by each cache controller is up-to-date.

4. The computer-implemented method of claim 3,

wherein data created in the cloud controller is encrypted prior to being written to a target cloud file in the remote cloud storage system;

wherein the target cloud file is serially encrypted;

wherein an accessing cloud controller that needs only a small piece of data from the target cloud file needs to wait for the entire file to be downloaded and decrypted before it can access a desired data block from the file; and

wherein the distributed filesystem is managed by one or more cloud controllers, and the cloud storage system is unaware of the organization and structure of the distributed filesystem.

5. The computer-implemented method of claim 4,

wherein data in the distributed filesystem is indexed using a global address space;

wherein each cloud file is uniquely indexed in the global address space; and

wherein using the metadata entry to download the cloud file further comprises;

determining from the metadata entry that the data block is not presently stored in the cloud controller;

using a global address stored in the metadata entry to identify the cloud file that includes the data block; and

using an offset stored in the metadata entry to determine the location of the data block in the cloud file.

6. The computer-implemented method of claim 5, wherein the method further comprises:

determining additional data blocks and files that are likely to be referenced in close temporal proximity to the data block; and

pre-fetching additional cloud files containing temporally proximate data to reduce the download latency associated with downloading cloud files from the remote cloud storage system.

7. The computer-implemented method of claim 6:

wherein determining the additional data blocks and files that are likely to be referenced in close temporal proximity further comprises receiving user feedback that indicates expected file characteristics and access patterns; and

wherein determining the additional cloud file further comprises;

receiving an indication that the data blocks for the file will be accessed sequentially;

determining the set of sequential data blocks subsequent to the requested data block in the file;

using the metadata for the distributed filesystem to determine a set of cloud files that contain the subsequent sequential data blocks, wherein the set of cloud files includes the additional cloud file; and

pre-fetching the set of additional cloud files to reduce the file access latency for the file for the client.

8. The computer-implemented method of claim 6, wherein determining the additional cloud files comprises:

determining that one or more additional data blocks for the file are not presently stored in the cloud controller; and

determining that the one or more additional data blocks for the file are stored in the additional cloud file.

9. The computer-implemented method of claim 8,

wherein writes in a local transactional filesystem result in modifications for the file being distributed across multiple cloud files;

wherein the method further comprises grouping files and data blocks that are likely to be accessed together into the same cloud file at the time that new data is written to the remote cloud storage system to reduce the number of cloud files that need to be loaded at a subsequent time to access the grouped files and data blocks; and

wherein pre-fetching the additional cloud file from the remote cloud storage system facilitates ensuring that all of the grouped data blocks for that are associated with the file are downloaded to the cloud controller.

10. The computer-implemented method of claim 6, wherein determining the additional cloud file comprises:

determining that one or more additional files that are in the same directory in the distributed filesystem as the requested file are likely to be accessed in conjunction with the data block;

determining that the one or more additional files are not presently stored in the cloud controller; and

determining that the one or more additional files are stored in the additional cloud file;

wherein pre-fetching cloud files that contain additional files that are in the same directory as a requested file facilitates reducing file access latency for the client.

11. The computer-implemented method of claim 6, wherein the method further involves:

creating the file in a local transactional filesystem on the cloud controller;

creating metadata for the file on the cloud controller;

uploading the cloud file containing the data block to the remote cloud storage system;

removing the data block from the cloud controller; and

downloading the cloud file from the remote cloud storage system when the data block is needed on the cloud controller.

12. The computer-implemented method of claim 11, wherein downloading the cloud file from the remote cloud storage system further comprises stitching the data block back into the local transactional filesystem.

13. A non-transitory computer-readable storage medium storing instructions that when executed by a computer cause the computer to perform a method for pre-fetching data in a distributed filesystem, the method comprising:

collectively managing data coherency for the data of the distributed filesystem using two or more cloud controllers by;

collectively presenting a unified namespace for the distributed filesystem to the clients of the distributed filesystem via the two or more cloud controllers, wherein the clients can only access the distributed filesystem via the cloud controllers, wherein the file data for the distributed filesystem is stored in a remote cloud storage system using encrypted cloud files, wherein each cloud controller caches a subset of the file data from the remote cloud storage system that is being actively accessed by that cloud controller'"'"'s respective clients, wherein all new file data received by each cloud controller from its clients is written to the remote cloud storage system via the receiving cloud controller;

maintaining at each cloud controller a copy of the complete metadata for all of the files stored in the distributed filesystem, wherein each cloud controller communicates any changes to the metadata for the distributed filesystem to the full set of cloud controllers for the distributed filesystem to ensure that the clients of the distributed filesystem share a consistent view of each file in the distributed filesystem;

upon receiving in a cloud controller new file data from a client, storing the new file data for the distributed filesystem as cloud files in the remote cloud storage system;

upon receiving confirmation that the new cloud files have been successfully stored in the remote cloud storage system, sending from the cloud controller an incremental metadata snapshot that includes new metadata for the distributed filesystem that describes the new file data and links to the new cloud files, wherein the incremental metadata snapshot is received by the other cloud controllers of the distributed filesystem and used to ensure data coherency for the distributed filesystem;

receiving at the cloud controller a request from a client to access a data block for a file;

traversing the cloud controller'"'"'s copy of the metadata for the distributed filesystem to identify a metadata entry associated with the data block;

using the metadata entry to download a cloud file containing the data block from the remote cloud storage system to the cloud controller;

determining that an additional cloud file in the remote cloud storage system includes data that is likely to be accessed in conjunction with the data block; and

pre-fetching the additional cloud file from the remote cloud storage system to the cloud controller.

14. The non-transitory computer-readable storage medium of claim 13,

wherein data in the distributed filesystem is indexed using a global address space;

wherein each cloud file is uniquely indexed in the global address space; and

wherein using the metadata entry to download the cloud file further comprises;

determining from the metadata entry that the data block is not presently stored in the cloud controller;

using a global address stored in the metadata entry to identify the cloud file that includes the data block; and

using an offset stored in the metadata entry to determine the location of the data block in the cloud file.

15. The non-transitory computer-readable storage medium of claim 14, wherein the method further comprises:

determining additional data blocks and files that are likely to be referenced in close temporal proximity to the data block; and

pre-fetching additional cloud files containing temporally proximate data to reduce the download latency associated with downloading cloud files from the remote cloud storage system.

16. The non-transitory computer-readable storage medium of claim 15:

wherein determining the additional data blocks and files that are likely to be referenced in close temporal proximity further comprises receiving user feedback that indicates expected file characteristics and access patterns; and

wherein determining the additional cloud file further comprises;

receiving an indication that the data blocks for the file will be accessed sequentially;

determining the set of sequential data blocks subsequent to the requested data block in the file;

using the metadata for the distributed filesystem to determine a set of cloud files that contain the subsequent sequential data blocks, wherein the set of cloud files includes the additional cloud file; and

pre-fetching the set of additional cloud files to reduce the file access latency for the file for the client.

17. The non-transitory computer-readable storage medium of claim 15, wherein determining the additional cloud files comprises:

determining that one or more additional data blocks for the file are not presently stored in the cloud controller; and

determining that the one or more additional data blocks for the file are stored in the additional cloud file.

18. The non-transitory computer-readable storage medium of claim 17,

wherein writes in a local transactional filesystem result in modifications for the file being distributed across multiple cloud files;

wherein the method further comprises grouping files and data blocks that are likely to be accessed together into the same cloud file at the time that new data is written to the remote cloud storage system to reduce the number of cloud files that need to be loaded at a subsequent time to access the grouped files and data blocks; and

wherein pre-fetching the additional cloud file from the remote cloud storage system facilitates ensuring that all of the grouped data blocks that are associated with the file are downloaded to the cloud controller.

19. The non-transitory computer-readable storage medium of claim 15, wherein determining the additional cloud file comprises:

determining that one or more additional files that are in the same directory in the distributed filesystem as the requested file are likely to be accessed in conjunction with the data block;

determining that the one or more additional files are not presently stored in the cloud controller; and

determining that the one or more additional files are stored in the additional cloud file;

wherein pre-fetching cloud files that contain additional files that are in the same directory as a requested file facilitates reducing file access latency for the client.

20. A cloud controller that facilitates pre-fetching data in a distributed filesystem, comprising:

a processor;

a storage mechanism that stores metadata for the distributed filesystem;

a receiving mechanism configured to receive a request from a client to access a data block for a file; and

a storage management mechanism;

wherein two or more cloud controllers are configured to collectively manage data coherency for the data of the distributed filesystem by;

collectively presenting a unified namespace for the distributed filesystem to the clients of the distributed filesystem via the two or more cloud controllers, wherein the clients can only access the distributed filesystem via the cloud controllers, wherein the file data for the distributed filesystem is stored in a remote cloud storage system using encrypted cloud files, wherein each cloud controller caches a subset of the file data from the remote cloud storage system that is being actively accessed by that cloud controller'"'"'s respective clients, wherein all new file data received by each cloud controller from its clients is written to the remote cloud storage system via the receiving cloud controller;

maintaining in the storage mechanism of each cloud controller a copy of the complete metadata for all of the files stored in the distributed filesystem, wherein each cloud controller communicates any changes to the metadata for the distributed filesystem to the full set of cloud controllers for the distributed filesystem to ensure that the clients of the distributed filesystem share a consistent view of each file in the distributed filesystem;

upon receiving in a given cloud controller new file data from a client, storing the new file data for the distributed filesystem as cloud files in the remote cloud storage system;

upon receiving confirmation that the new cloud files have been successfully stored in the remote cloud storage system, sending from the given cloud controller an incremental metadata snapshot that includes new metadata for the distributed filesystem that describes the new file data and links to the new cloud files, wherein the incremental metadata snapshot is received by the other cloud controllers of the distributed filesystem and used to ensure data coherency for the distributed filesystem;

wherein the storage management mechanism is configured to;

traverse the stored metadata in the storage mechanism to identify a metadata entry associated with the data block;

download a cloud file containing the data block from the remote storage system;

determine that an additional cloud file in the remote cloud storage system includes data that is likely to be accessed in conjunction with the data block; and

pre-fetch the additional cloud file from the remote cloud storage system.