Unified data services for block and file objects

• US 9,122,697 B1
• Filed: 03/29/2013
• Issued: 09/01/2015
• Est. Priority Date: 03/29/2013
• Status: Active Grant

• Current Assignee: Emc IP Holding Company LLC (Dell Technologies Inc.)
• Original Assignee: EMC Corporation (Dell Technologies Inc.)
• Inventors: Bono, Jean-Pierre, Davenport, William, de Forest, Miles A., Armangau, Philippe, Forrester, Walter C., Owen, Karl
• Primary Examiner(s): Pham, Hung Q
• Application Number:
  US13/853,284
• Time in Patent Office: 886 Days
• Field of Search: None
• US Class Current: 1/1
• CPC Class Codes: G06F 16/13 File access structures, e.g. distributed indices arrangements of input from, or output to, record carriers G06F3/06

Abstract

An improved technique for a data storage apparatus that combines both block-based and file-based functionality in a unified data path architecture. The improved technique brings together IO processing of block-based storage systems and file-based storage systems by expressing both block-based objects (e.g., LUNs) and file-based objects (e.g., host file systems) in the form of files. These files are parts of an underlying set of internal file systems stored on a set of storage units served by a storage pool. Because block and file-based objects are all expressed as files of this set of internal file systems, a common set of services can be applied across block-based and file-based objects. In particular, enhanced data services such as compression, automated storage tiering and deduplication are provided across both types of object using one set of common mechanisms.

Claims

1. A data storage system, comprising:

a back-end interface to physical storage;

a front-end interface to a communications network coupling the data storage system to one or more host computers; and

one or more storage processors configured to execute computer program instructions to cause the data storage system to provide data storage services to the host computers using the physical storage by;

defining a layered operating stack including a pool layer, a file system layer, a map layer and one or more service layers, the pool layer defining and providing slices of the physical storage for storing host objects, the file system layer having one or more internal file systems with respective files stored in corresponding sets of slices of the physical storage, the map layer mapping the host objects to respective files of the one or more internal file systems, the host objects including block-organized host objects and file-oriented host objects, each of the block-organized host objects and the file-oriented host objects is in a corresponding original object form;

by the pool layer, the file system layer and the map layer collectively, providing a basic data storage service for storing a file of one of the one or more internal file systems by distributing blocks of the file across a set of slices of the physical storage, maintaining original object form of the stored file, and providing a basic level of storage efficiency; and

by the service layers, providing one or more automated enhanced data services, each automated enhanced data service applying a respective service transformation to the original object form to generate respective data in a service-specified form and storing the respective data in the slices, the service-specified form providing a service level of storage efficiency generally greater than the basic level of storage efficiency.

2. A data storage system according to claim 1, wherein the basic level of storage efficiency and the service levels level of storage efficiency are respective levels of storage capacity usage efficiency, and wherein the service-specified form is of smaller data size than original object form.

3. A data storage system according to claim 2, wherein the one or more automated enhanced data services include an automatic data compression service operative to apply data compression to each of one or more files of one of the one or more internal file systems to generate respective compressed forms of the one or more files.

4. A data storage system according to claim 3, wherein:

the host objects are of a plurality of respective types selected from text and media;

the respective files for the host objects are accompanied by respective indicators of respective host object types of the host objects; and

the automatic data compression service is operative in response to the respective indicators of the respective host object types to apply respective distinct compression algorithms to the respective files to which the host objects of different host object types are mapped.

5. A data storage system according to claim 2, wherein the one or more automated enhanced data services include an automatic data deduplication service operative to apply deduplication to blocks of one or more files of one of the internal file systems to replace duplicate data blocks with respective pointers to corresponding shared single instances.

6. A data storage system according to claim 5, wherein a pool of the pool layer includes a deduplication container providing a single deduplication domain for the pool.

7. A data storage system according to claim 6, wherein the single deduplication domain hosts a plurality of the files storing host objects of different types, a first file storing a block-organized host object, and a second file storing a file-oriented host object, the deduplication service applying deduplication across the host objects of different types.

8. A data storage system according to claim 6, wherein the one or more automated enhanced data services further include an automatic data compression service to compress files in the deduplication container to further improve storage efficiency.

9. A data storage system according to claim 1, wherein:

the slices of the pool layer are drawn from distinct tiers of the physical storage having respective storage usage efficiency and performance characteristics;

the one or more automated enhanced data services include an automated storage tiering service providing the service level of storage efficiency by automated and dynamic selection among the distinct tiers of the physical storage for the slices to achieve a desired balance of performance and storage usage efficiency.

10. A data storage system according to claim 9, wherein the physical storage include magnetic disk drives and electronic flash drives each forming a respective tier, the magnetic disk drive tier providing a first storage density and first access time, the electronic flash drive tier providing a lower storage density and faster access time.

11. A data storage system according to claim 1, wherein the block-organized host objects include one or more logical storage units (LUNs) and the file-oriented host objects include one or more host file systems, each host file system being stored in a single file of one of the one or more internal file systems.

12. A data storage system according to claim 1, wherein the service layers are provided between the file system layer and the physical storage and operate upon the respective files of the one or more internal file systems for supporting both the block-organized host objects and the file-oriented host objects by operating on respective ones of the respective files that are mapped by the host objects.

13. A data storage system according to claim 12, wherein the service layers include at least one of a first service layer and a second service layer, the first service layer provided between the file system layer and the pool layer and creating the service-specified form as service-specified content and organization of the slices as distinct from a basic content and organization of the slices, the second service layer provided between the pool layer and the physical storage and creating the service-specified form as service-specified mapping of the slices to the physical storage as distinct from a basic mapping of the slices to the physical storage.

14. A data storage system according to claim 1, wherein data of each of the respective files of the one or more internal file systems is distributed across one or more respective sets of multiple fixed-size blocks, each set of multiple fixed-size blocks for a file being mapped to a respective one of the slices of the pool layer.

15. A method of operating a data storage system having a back-end interface to physical storage and a front-end interface to a communications network coupling the data storage system to one or more host computers, comprising:

defining a layered operating stack including a pool layer, a file system layer, a map layer and one or more service layers, the pool layer defining and providing slices of the physical storage for storing host objects, the file system layer having one or more internal file systems with respective files stored in corresponding sets of slices of the physical storage, the map layer mapping the host objects to respective files of the internal file systems, the host objects including block-organized host objects and file-oriented host objects, each of the block-organized host objects and the file-oriented host objects is in a corresponding original object form;

by the pool layer, the file system layer and the map layer collectively, providing a basic data storage service for storing a file of one of the one or more internal file systems by distributing blocks of the file across a set of slices of the physical storage, maintaining original object form of the stored file, and providing a basic level of storage efficiency; and

by the service layers, providing one or more automated enhanced data services, each automated enhanced data service applying a respective service transformation to the original object form to generate respective data in a service-specified form and storing the data in the slices, the service-specified form providing a service level of storage efficiency generally greater than the basic level of storage efficiency.

16. A method according to claim 15, wherein the basic level of storage efficiency and the service level of storage efficiency are respective levels of storage capacity usage efficiency, and wherein the service-specified form is of smaller data size than original object form.

17. A method according to claim 16, wherein the one or more automated enhanced data services include an automatic data compression service operative to apply data compression to each of one or more files of the one or more internal file systems to generate respective compressed forms of the one or more files.

18. A method according to claim 17, wherein:

the host objects are of a plurality of respective types selected from text and media;

the respective files for the host objects are accompanied by respective indicators of respective host object types of the host objects; and

the automatic data compression service is operative in response to the respective indicators of the respective host object types to apply respective distinct compression algorithms to the respective files to which the host objects of different host object types are mapped.

19. A method according to claim 16, wherein the one or more automated enhanced data services include an automatic data deduplication service operative to apply deduplication to blocks of one or more files of an internal file system to replace duplicate data blocks with respective pointers to corresponding shared single instances.

20. A method according to claim 19, wherein a pool of the pool layer includes a deduplication container providing a single deduplication domain for the pool.

21. A method according to claim 20, wherein the single deduplication domain hosts a plurality of the files storing host objects of different types, a first file storing a block-organized host object, and a second file storing a file-oriented host object, the deduplication service applying deduplication across the host objects of different types.

22. A method according to claim 20, wherein the one or more automated enhanced data services further include an automatic data compression service files to compress the files in the deduplication container to further improve storage efficiency.

23. A method according to claim 15, wherein:

the slices of the pool layer are drawn from distinct tiers of the physical storage having respective storage usage efficiency and performance characteristics;

the one or more automated enhanced data services include an automated storage tiering service providing the service level of storage efficiency by automated and dynamic selection among the distinct tiers of the physical storage for the slices to achieve a desired balance of performance and storage usage efficiency.

24. A method according to claim 23, wherein the physical storage include magnetic disk drives and electronic flash drives each forming a respective tier, the magnetic disk drive tier providing a first storage density and first access time, the electronic flash drive tier providing a lower storage density and faster access time.

25. A method according to claim 15, wherein the block-organized host objects include one or more logical storage units (LUNs) and the file-oriented host objects include one or more host file systems, each host file system being stored in a single file of one of the one or more internal file systems.

26. A computer program product having a non-transitory computer readable medium including instructions which, when executed by one or more storage processors of a data storage system cause the data storage system to provide data storage services to the host computers using physical storage, including:

defining a layered operating stack including a pool layer, a file system layer, a map layer and one or more service layers, the pool layer defining and providing slices of the physical storage for storing host objects, the file system layer having one or more internal file systems with respective files stored in corresponding sets of slices of the physical storage, the map layer mapping the host objects to respective files of the one or more internal file systems, the host objects including block-organized host objects and file-oriented host objects, each of the block-organized host objects and the file-oriented host objects is in a corresponding original object form;

by the pool, file system and map layers collectively, providing a basic data storage service for storing a file of one of the one or more internal file systems by distributing blocks of the file across a set of slices of the physical storage, maintaining original object form of the stored file, and providing a basic level of storage efficiency; and

by the service layers, providing one or more automated enhanced data services, each automated enhanced data service applying a respective service transformation to the original object form to generate respective data in a service-specified form and storing the data in the slices, the service-specified form providing a service level of efficiency generally greater than the basic level of storage efficiency.