Using a local cache to store, access and modify files tiered to cloud storage

• US 9,727,470 B1
• Filed: 09/30/2014
• Issued: 08/08/2017
• Est. Priority Date: 09/30/2014
• Status: Active Grant

• Current Assignee: Emc IP Holding Company LLC (Dell Technologies Inc.)
• Original Assignee: Emc IP Holding Company LLC (Dell Technologies Inc.)
• Inventors: Cande, Edward G., Wang, Lijun, Walton, Jonathan M.
• Primary Examiner(s): Lo, Ann J
• Assistant Examiner(s): Taeuber, Michelle
• Application Number:
  US14/501,928
• Time in Patent Office: 1,043 Days
• CPC Class Codes: G06F 12/0833 in combination with broadcast means (e.g. for invalidation or updating)
• CPC Class Codes: G06F 12/084 with a shared cache
• CPC Class Codes: G06F 12/0866 for peripheral storage systems, e.g. disk cache
• CPC Class Codes: G06F 16/176 Support for shared access to files; File sharing support
• CPC Class Codes: G06F 16/1774 Locking methods, e.g. locking methods for file systems allowing shared and concurrent access to files
• CPC Class Codes: G06F 16/182 Distributed file systems
• CPC Class Codes: G06F 16/2336 Pessimistic concurrency control approaches, e.g. locking or multiple versions without time stamps
• CPC Class Codes: G06F 16/2343 Locking methods, e.g. distributed locking or locking implementation details
• CPC Class Codes: G06F 2212/154 Networked environment
• CPC Class Codes: G06F 2212/2515 being configurable for different purposes, e.g. as cache or non-cache memory
• CPC Class Codes: G06F 2212/263 Network storage, e.g. SAN or NAS
• CPC Class Codes: G06F 2212/621 Coherency control relating to peripheral accessing, e.g. from DMA or I/O device
• CPC Class Codes: G06F 3/0622 in relation to access
• CPC Class Codes: G06F 3/0643 Management of files
• CPC Class Codes: G06F 3/067 Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]

Abstract

Systems and methods are provided herein for efficient local caching of data tiered to cloud storage to help reduce the bandwidth cost of repeated reads and writes to the same region of a stubbed file, increase the performance of write operations, and increase performance of read operations to portions of a stubbed file accessed repeatedly. When operations are directed toward data tiered to the cloud, the data can be read from cloud storage and stored within a local cache. A cache tracking tree can be generated and used to track file regions of a stub file, cached states associated with regions of the stub file, a set of cache flags, and other file and mapping data. For example, the cache state of regions of a stub file can be tracked including a cached data state, a non-cached state, a modified state, or a truncated state. Operations directed toward stubbed files can then look to the cache tracking tree to determine the most efficient way to access, retrieve, or operate on the data that maximizes local file system performance while reducing network activity.

Claims

1. A method comprising:

generating a cache tracking tree based on a stub file, wherein the cache tracking tree tracks file regions of the stub file, cached states associated with regions of the stub file, a set of cache flags, a version, a file size, a region size, a data offset, a last region, and a range map;

associating the cache tracking tree with the stub file;

receiving a read request associated with a set of regions of the stub file;

activating a shared cache tracking tree lock on the cache tracking tree wherein the set of cache flags are set to indicate the shared cache tracking tree lock;

activating a shared region lock for the set of regions, wherein the set of cache flags are set to indicate the shared region lock;

determining a cached state associated with the set of regions of the stub file;

in response to determining the cached state associated with the set of regions of the stub file do not have a state of not cached, upgrading the shared region lock to an exclusive region lock;

reading file data from the set of regions; and

deactivating the shared cache tracking tree lock and the exclusive region lock.

2. The method of claim 1, further comprising:

receiving a fill request associated with a set of regions of the stub file;

activating an exclusive cache tracking tree lock on the cache tracking tree, wherein the set of cache flags are set to indicate the exclusive cache tracking tree lock is active;

determining a cached state associated with the set of regions of the stub file;

in response to determining the cached state associated with the set of regions of the stub file are not dirty, downgrading the exclusive cache tracking tree lock to a shared cache tracking tree lock, wherein the set of cache flags are set to indicate the shared cache tracking tree lock is active;

updating the cached states associated with the set of regions to cached;

activating a shared region lock for the set of regions wherein the set of cache flags are set to indicate the shared region lock;

reading file data from cloud storage associated with the set of regions;

storing the file data in local cache storage based on the cache tracking tree; and

deactivating the shared cache tracking tree lock and the shared region lock.

3. The method of claim 1, further comprising:

receiving a write operation associated with a set of regions of the stub file and a set of write data;

activating an exclusive lock on the cache tracking tree wherein the set of cache flags are set to indicate the exclusive lock;

in response to determining that the file size of the write is greater than the file size, updating at least one of the file size, the last region, and the range map;

downgrading the exclusive lock to a cache tracking tree shared lock, wherein the set of cache flags are set to indicate the cache tracking tree shared lock;

activating an exclusive region lock for the set of regions wherein the set of cache flags are set to indicate the exclusive region lock;

in response to determining that regions of the set of regions have a cached state of not cached, filling regions of the set of regions;

updating the cached states of the set of regions to dirty;

writing the write data into the set of regions; and

deactivating the cache tracking tree shared lock and the exclusive region lock.

4. A system comprising at least one storage device and at least one hardware processor configured to:

generating a cache tracking tree based on a stub file, wherein the cache tracking tree tracks file regions of the stub file, cached states associated with regions of the stub file, a set of cache flags, a version, a file size, a region size, a data offset, a last region, and a range map;

associating the cache tracking tree with the stub file;

receiving a read request associated with a set of regions of the stub file;

activating a shared cache tracking tree lock on the cache tracking tree wherein the set of cache flags are set to indicate the shared cache tracking tree lock;

activating a shared region lock for the set of regions, wherein the set of cache flags are set to indicate the shared region lock;

determining a cached state associated with the set of regions of the stub file;

in response to determining the cached state associated with the set of regions of the stub file do not have a state of not cached, upgrading the shared region lock to an exclusive region lock;

reading file data from the set of regions; and

deactivating the shared cache tracking tree lock and the exclusive region lock.

5. The system of claim 4, further configured to:

receiving a fill request associated with a set of regions of the stub file;

activating an exclusive cache tracking tree lock on the cache tracking tree, wherein the set of cache flags are set to indicate the exclusive cache tracking tree lock is active;

determining a cached state associated with the set of regions of the stub file;

in response to determining the cached state associated with the set of regions of the stub file are not dirty, downgrading the exclusive cache tracking tree lock to a shared cache tracking tree lock, wherein the set of cache flags are set to indicate the shared cache tracking tree lock is active;

updating the cached states associated with the set of regions to cached;

activating a shared region lock for the set of regions wherein the set of cache flags are set to indicate the shared region lock;

reading file data from cloud storage associated with the set of regions;

storing the file data in local cache storage based on the cache tracking tree; and

deactivating the shared cache tracking tree lock and the shared region lock.

6. The system of claim 4, further configured to:

receiving a write operation associated with a set of regions of the stub file and a set of write data;

activating an exclusive lock on the cache tracking tree wherein the set of cache flags are set to indicate the exclusive lock;

in response to determining that the file size of the write is greater than the file size, updating at least one of the file size, the last region, and the range map;

downgrading the exclusive lock to a cache tracking tree shared lock, wherein the set of cache flags are set to indicate the cache tracking tree shared lock;

activating an exclusive region lock for the set of regions wherein the set of cache flags are set to indicate the exclusive region lock;

in response to determining that regions of the set of regions have a cached state of not cached, filling regions of the set of regions;

updating the cached states of the set of regions to dirty;

writing the write data into the set of regions; and

deactivating the cache tracking tree shared lock and the exclusive region lock.

7. A non-transitory computer readable medium with program instructions stored thereon to perform the following acts:

generating a cache tracking tree based on a stub file, wherein the cache tracking tree tracks file regions of the stub file, cached states associated with regions of the stub file, a set of cache flags, a version, a file size, a region size, a data offset, a last region, and a range map;

associating the cache tracking tree with the stub file;

receiving a read request associated with a set of regions of the stub file;

activating a shared cache tracking tree lock on the cache tracking tree wherein the set of cache flags are set to indicate the shared cache tracking tree lock;

activating a shared region lock for the set of regions, wherein the set of cache flags are set to indicate the shared region lock;

determining a cached state associated with the set of regions of the stub file;

in response to determining the cached state associated with the set of regions of the stub file do not have a state of not cached, upgrading the shared region lock to an exclusive region lock;

reading file data from the set of regions; and

deactivating the shared cache tracking tree lock and the exclusive region lock.

8. The non-transitory computer readable medium of claim 7, with program instructions stored thereon to further perform the following acts:

receiving a fill request associated with a set of regions of the stub file;

activating an exclusive cache tracking tree lock on the cache tracking tree, wherein the set of cache flags are set to indicate the exclusive cache tracking tree lock is active;

determining a cached state associated with the set of regions of the stub file;

in response to determining the cached state associated with the set of regions of the stub file are not dirty, downgrading the exclusive cache tracking tree lock to a shared cache tracking tree lock, wherein the set of cache flags are set to indicate the shared cache tracking tree lock is active;

updating the cached states associated with the set of regions to cached;

activating a shared region lock for the set of regions wherein the set of cache flags are set to indicate the shared region lock;

reading file data from cloud storage associated with the set of regions;

storing the file data in local cache storage based on the cache tracking tree; and

deactivating the shared cache tracking tree lock and the shared region lock.

9. The non-transitory computer readable medium of claim 7, with program instructions stored thereon to further perform the following acts:

receiving a write operation associated with a set of regions of the stub file and a set of write data;

activating an exclusive lock on the cache tracking tree wherein the set of cache flags are set to indicate the exclusive lock;

in response to determining that the file size of the write is greater than the file size, updating at least one of the file size, the last region, and the range map;

downgrading the exclusive lock to a cache tracking tree shared lock, wherein the set of cache flags are set to indicate the cache tracking tree shared lock;

activating an exclusive region lock for the set of regions wherein the set of cache flags are set to indicate the exclusive region lock;

in response to determining that regions of the set of regions have a cached state of not cached, filling regions of the set of regions;

updating the cached states of the set of regions to dirty;

writing the write data into the set of regions; and

deactivating the cache tracking tree shared lock and the exclusive region lock.