File system snapshots over thinly provisioned volume file in mapped mode

• US 9,256,629 B1
• Filed: 06/28/2013
• Issued: 02/09/2016
• Est. Priority Date: 06/28/2013
• Status: Active Grant

• Current Assignee: Emc IP Holding Company LLC (Dell Technologies Inc.)
• Original Assignee: EMC Corporation (Dell Technologies Inc.)
• Inventors: Bono, Jean-Pierre, Davenport, William C., de Forest, Miles A., Forrester, Walter C., Zhang, Ye, Armangau, Philippe, Marko, Michal
• Primary Examiner(s): Stevens, Robert
• Application Number:
  US13/931,775
• Time in Patent Office: 956 Days
• Field of Search: 707/827, 707/756
• US Class Current: 1/1
• CPC Class Codes: G06F 16/10 File systems; File servers
• CPC Class Codes: G06F 16/1824 implemented using Network-attached Storage [NAS] architecture distributed or networked storage systems G06F3/067; protocols for distributed storage of data in a network H04L67/1097

Abstract

A data storage system includes storage devices and a processing subsystem executing software forming a lower-deck file system and an upper-deck file system. The lower-deck file system presents a volume file from which storage is provided to the upper-deck file system, and units of storage of the storage devices are allocable to the upper-deck file system but not reserved to it. The volume file is thinly provisioned, and additional units of storage are added dynamically to increase its allocated size in response to demand from the upper-deck file system. The lower-deck file system operates in a mapped mode in which a block address in the lower-deck file system is obtained by a generally arbitrary mapping (e.g., an Inode/IB tree) of a block address in the volume file.

Claims

1. A data storage system for use by a file system consumer according to a file system interface, comprising:

one or more physical, non-transitory, nonvolatile storage devices; and

a processing subsystem executing software to form a multi-level stack of functional components including a lower-deck file system and an upper-deck file system, the upper-deck file system providing the file system interface to the file system consumer, the lower-deck file system presenting a volume file from which storage is provided to the upper-deck file system, units of storage of the storage devices being allocable to the upper-deck file system as well as to other uses in the data storage system, the volume file being thinly provisioned so that its allocated size at any given time is generally less than its configured size, additional units of storage being added dynamically to the volume file to increase its allocated size in response to demand from the upper-deck file system, the lower-deck file system operating in a mapped mode in which a block address of a data block in the lower-deck file system is obtained by a generally arbitrary mapping of a corresponding block address of a data block in the volume file,

wherein the volume file is a primary volume file and the mapping is a primary file mapping, and one or more snapshots of the primary volume file are stored in respective snapshot volume files of the lower-deck file system, each snapshot being a respective point-in-time copy of the primary volume file, and the mapped mode includes use of generally arbitrary respective second mappings of respective block addresses of data blocks of the snapshot volume files to obtain corresponding block addresses of the data blocks in the lower-deck file system.

2. A data storage system according to claim 1, wherein upon creation of a snapshot it is populated with existing blocks of the primary volume file, and in response to ongoing writes to the primary file, a write split is performed by which current data blocks of the primary volume file are separated from the existing blocks which continue to populate the snapshot, the current data blocks being the subject of the ongoing writes as well as reads directed to the primary volume file.

3. A data storage system according to claim 1, wherein the snapshot volume files are first-level snapshot volume files, and wherein one or more second-level snapshots of the one of the first-level snapshot volume files are stored in respective second-level snapshot volume files of the lower-deck file system, each second-level snapshot being a respective point-in-time copy of a respective first-level snapshot volume file, and the mapped mode includes use of generally arbitrary respective third mappings of respective block addresses of data blocks of the second-level snapshot volume files to obtain corresponding block addresses of the data blocks in the lower-deck file system.

4. A data storage system according to claim 1, wherein the mapping includes traversal of pointers and indirect blocks of an inode structure identifying locations of data blocks in the lower-deck file system based on block addresses of the data blocks in the volume file.

5. A data storage system according to claim 1, wherein the volume file has a maximum size, and wherein, upon the allocated size of the volume file becoming equal to the maximum size, an out-of-space response to the file system consumer.

6. A data storage system according to claim 1, wherein the volume file is a sparse volume for which the allocated size is generally less than one-half its configured size.

7. A data storage system according to claim 1, wherein the mapped mode of operation of the lower-deck file system includes deduplication functionality by which duplicate blocks in a deduplication domain including the upper-deck file system are stored as a single shared block and corresponding block pointers in the lower-deck file system.

8. A data storage system according to claim 1, wherein:

all metadata of the upper-deck file system is stored within the volume file and therefore automatically included as part of the snapshots without need for separately making the metadata stable at a time a snapshot is obtained; and

at a later time of mounting a snap of the upper-deck file system, state is recovered by reading and applying the following from the metadata of the snap;

(1) upper deck slice pool state, (2) metadata state, and (3) file cache state.

9. A data storage system according to claim 1, wherein the units of storage are provided by a pool layer, and the multi-level stack of functional components includes a sparse volume layer of mapping between the lower-deck file system and the pool layer, the sparse volume layer including a sparse volume on which the lower-deck file system is structured and which is accessed by file operations of the lower-deck file system.

10. A data storage system according to claim 9, wherein the upper-deck file system draws slices of storage from the volume file, and the volume file is constituted by slices from the pool via operation of the lower-deck file system which draws slices from the pool.

11. A data storage system according to claim 1, wherein the upper-deck file system operates in a respective mapped mode including use of generally arbitrary mappings of respective block addresses of data blocks of files of the upper-deck file system to obtain corresponding block addresses of the data blocks of the volume file.

12. A method of operating a data storage system to provide file system functionality to a file system consumer according to a file system interface, the data storage system including one or more physical nonvolatile storage devices and a processing subsystem, comprising:

instantiating, in the processing subsystem, software to form a multi-level stack of functional components including a lower-deck file system and an upper-deck file system, the upper-deck file system providing the file system interface to the file system consumer; and

by the lower-deck file system, (i) presenting a volume file from which storage is provided to the upper-deck file system, units of storage of the storage devices being allocable to the upper-deck file system as well as to other uses in the data storage system, the volume file being thinly provisioned so that its allocated size at any given time is generally less than its configured size, (ii) dynamically adding additional units of storage to the volume file to increase its allocated size in response to demand from the upper-deck file system, and (iii) utilizing a mapped operating mode in which a block address of a data block in the lower-deck file system is obtained by a generally arbitrary mapping of a corresponding block address of a data block in the volume file,

wherein the volume file is a primary volume file and the mapping is a primary file mapping, and one or more snapshots of the primary volume file are stored in respective snapshot volume files of the lower-deck file system, each snapshot being a respective point-in-time copy of the primary volume file, and the mapped mode includes use of generally arbitrary respective second mappings of respective block addresses of data blocks of the snapshot volume files to obtain corresponding block addresses of the data blocks in the lower-deck file system.

13. A method according to claim 12, wherein upon creation of a snapshot it is populated with existing blocks of the primary volume file, and in response to ongoing writes to the primary file, a write split is performed by which current data blocks of the primary volume file are separated from the existing blocks which continue to populate the snapshot, the current data blocks being the subject of the ongoing writes as well as reads directed to the primary volume file.

14. A method according to claim 12, wherein the snapshot volume files are first-level snapshot volume files, and wherein one or more second-level snapshots of the one of the first-level snapshot volume files are stored in respective second-level snapshot volume files of the lower-deck file system, each second-level snapshot being a respective point-in-time copy of a respective first-level snapshot volume file, and the mapped mode includes use of generally arbitrary respective third mappings of respective block addresses of data blocks of the second-level snapshot volume files to obtain corresponding block addresses of the data blocks in the lower-deck file system.

15. A method according to claim 14, wherein the mapping includes traversal of pointers and indirect blocks of an inode structure identifying locations of data blocks in the lower-deck file system based on block addresses of the data blocks in the volume file.

16. A method according to claim 14, wherein the volume file has a maximum size, and wherein, upon the allocated size of the volume file becoming equal to the maximum size, an out-of-space response to the file system consumer.

17. A method according to claim 14, wherein the volume file is a sparse volume for which the allocated size is generally less than one-half its configured size.

18. A method according to claim 14, wherein the mapped mode of operation of the lower-deck file system includes deduplication functionality by which duplicate blocks in a deduplication domain including the upper-deck file system are stored as a single shared block and corresponding block pointers in the lower-deck file system.

19. A method according to claim 12, wherein:

all metadata of the upper-deck file system is stored within the volume file and therefore automatically included as part of the snapshots without need for separately making the metadata stable at a time a snapshot is obtained; and

at a later time of mounting a snap of the upper-deck file system, state is recovered by reading and applying the following from the metadata of the snap;

(1) upper deck slice pool state, (2) metadata state, and (3) file cache state.