Storage tiering in replication target based on logical extents

• US 10,037,369 B1
• Filed: 06/26/2015
• Issued: 07/31/2018
• Est. Priority Date: 06/26/2015
• Status: Active Grant

• Current Assignee: Emc IP Holding Company LLC (Dell Technologies Inc.)
• Original Assignee: Emc IP Holding Company LLC (Dell Technologies Inc.)
• Inventors: Bono, Jean-Pierre, Duprey, Dennis, Chen, Xiangping, Owen, Karl M., de Forest, Miles A, Tummala, Himabindu
• Primary Examiner(s): Le, Hung
• Application Number:
  US14/752,616
• Time in Patent Office: 1,131 Days
• CPC Class Codes: G06F 16/164 File meta data generation
• CPC Class Codes: G06F 16/1727 Details of free space management performed by the file system saving storage space on storage systems G06F3/0608; management of blocks in storage devices G06F3/064
• CPC Class Codes: G06F 16/27 Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor

Abstract

A technique for replicating a file from a source data storage system (source) to a destination data storage system (destination) includes receiving, by the destination from the source, (i) file data for multiple logical extents of the file and (ii) respective tiering metadata for those logical extents. The destination selects, based on the tiering metadata, one or more storage tiers available on the destination. The destination updates a local replica of the file by placing the logical extents on the selected tier or tiers.

Claims

1. A method of performing storage tiering in a destination data storage system, the destination data storage system having multiple storage tiers configured to store replicated data received from a source data storage system, the method comprising:

receiving, by the destination data storage system, replication updates from the source data storage system, the replication updates providing file data at each of a set of logical extents of a file on the source data storage system and identifying, for each of the set of logical extents, a respective range of logical offsets of that logical extent into the file on the source data storage system;

receiving, by the destination data storage system, tiering metadata, the tiering metadata (i) generated by the source data storage system for performing storage tiering of the file on the source data storage system and (ii) including a respective portion of tiering metadata for each of the set of logical extents; and

updating a replica of the file on the destination data storage system, including, for each of the set of logical extents, (i) selecting, based on the portion of tiering metadata for that logical extent, a storage tier in the destination data storage system for the file data provided in that logical extent and (ii) placing the file data provided in that logical extent in the selected storage tier,

wherein the portion of tiering metadata for one of the set of logical extents includes data temperature metadata specific to that logical extent, the data temperature metadata for that logical extent indicating a moving average of input/output activity directed to that logical extent on the source data storage system over time,

wherein selecting a storage tier for that logical extent includes performing an auto-tiering operation on that logical extent, the auto-tiering operation generating an auto-tiering output that prescribes, based on the temperature metadata for that logical extent, a storage tier on which to place the file data of that logical extent in the destination data storage system,

wherein receiving the replication updates includes receiving a continuous replication update, the continuous replication update (i) identifying one of the set of logical extents and providing, for the identified logical extent, (a) a logical offset range of the identified logical extent within the file and (b) file data for the logical extent in the first data storage system, and

wherein receiving the continuous replication update includes concurrently receiving the portion of tiering metadata for the identified logical extent provided in the continuous replication update.

2. The method of claim 1,

wherein the portion of tiering metadata for the logical extent further includes QoS metadata for that logical extent, the QoS metadata for that logical extent indicating a storage tier on which a source QoS management operation on the source data storage system prescribes placement of the file data of that logical extent on the source data storage system, and

wherein selecting a storage tier for that logical extent includes performing a destination QoS-management operation on that logical extent, the destination QoS-management operation generating a QoS output that prescribes, based on the QoS metadata for that logical extent, a storage tier on which to place the file data of that logical extent on the destination data storage system.

3. The method of claim 2, wherein the portion of tiering metadata for the logical extent further includes tiering policy metadata for that logical extent, the tiering policy metadata for that logical extent prescribing a procedure for placing the file data of that logical extent when the auto-tiering output prescribes a first storage tier and the QoS output prescribes a second storage tier different from the first storage tier.

4. The method of claim 1, wherein, when placing the file data for each of the set of logical extents on one of the storage tiers of the destination data storage system, the destination data storage system does not temporarily place the file data on a storage tier without regard to the tiering metadata, but rather places the file data directly on a storage tier selected in accordance with the respective portion of tiering metadata received for that logical extent.

5. The method of claim 4, wherein receiving the tiering metadata includes receiving a set of maps that relate multiple logical extents of the file to respective QoS values and to respective data temperature values.

6. A destination data storage system, comprising control circuitry that includes a set of processing units coupled to memory, the control circuitry constructed and arranged to:

receive, by the destination data storage system, a set of replication updates from the source data storage system, the replication updates providing file data at each of a set of logical extents of a file on the source data storage system and identifying, for each of the set of logical extents, a respective range of logical offsets of that logical extent into the file on the source data storage system;

receive, by the destination data storage system, tiering metadata, the tiering metadata (i) generated by the source data storage system for performing storage tiering of the file on the source data storage system and (ii) including a respective portion of tiering metadata for each of the set of logical extents; and

update a replica of the file on the destination data storage system, including, for each of the set of logical extents, (i) selecting, based on the portion of tiering metadata for that logical extent, a storage tier in the destination data storage system for the file data provided in that logical extent and (ii) placing the file data provided in that logical extent in the selected storage tier,

wherein the destination data storage system is configured to receive, concurrently with each replication update, a respective portion of tiering metadata for a logical extent identified in that replication update.

7. The destination data storage system of claim 6,

wherein the portion of tiering metadata for one of the set of logical extents includes data temperature metadata specific to that logical extent, the data temperature metadata for that logical extent indicating a moving average of input/output activity directed to that logical extent on the source data storage system over time, and

wherein, when constructed and arranged to select a storage tier for that logical extent, the control circuitry is further constructed and arranged to perform an auto-tiering operation on that logical extent, the auto-tiering operation generating an auto-tiering output that prescribes, based on the temperature metadata for that logical extent, a storage tier on which to place the file data of that logical extent in the destination data storage system.

8. The destination data storage system of claim 7,

wherein the portion of tiering metadata for the logical extent further includes QoS metadata for that logical extent, the QoS metadata for that logical extent indicating a storage tier on which a source QoS management operation on the source data storage system prescribes placement of the file data of that logical extent on the source data storage system, and

wherein, when constructed and arranged to select a storage tier for that logical extent, the control circuitry is further constructed and arranged to perform a destination QoS-management operation on that logical extent, the destination QoS-management operation generating a QoS output that prescribes, based on the QoS metadata for that logical extent, a storage tier on which to place the file data of that logical extent on the destination data storage system.

9. The destination data storage system of claim 8, wherein the portion of tiering metadata for the logical extent further includes tiering policy metadata for that logical extent, the tiering policy metadata for that logical extent prescribing a procedure for placing the file data of that logical extent when the auto-tiering output prescribes a first storage tier and the QoS output prescribes a second storage tier different from the first storage tier.

10. The destination data storage system of claim 9, wherein, when constructed and arranged to receive the tiering metadata, the control circuitry is further constructed and arranged to receive a set of maps that relate multiple logical extents of the file to respective data QoS values and to respective temperature values.

11. A computer-program product including a set of non-transitory, computer-readable media having instructions which, when executed by control circuitry of a destination data storage system, cause the control circuitry to perform a method for performing storage tiering in the destination data storage system, the method comprising:

receiving, by the destination data storage system, tiering metadata, the tiering metadata generated by a source data storage system for performing storage tiering of a file on the source data storage system;

receiving, by the destination data storage system, replication updates from the source data storage system, the replication updates providing file data at each of a set of logical extents of a file on the source data storage system and identifying, for each of the set of logical extents, a respective range of logical offsets of that logical extent into the file on the source data storage system; and

updating a replica of the file on the destination data storage system, including placing each of the set of logical extents on one of multiple storage tiers of the destination data storage system in accordance with the tiering metadata,

wherein the tiering metadata includes a respective portion of tiering metadata for each of the set of logical extents, and wherein placing each logical extent on a storage tier is performed in accordance with the respective portion of tiering metadata for that logical extent,

wherein, when receiving each replication update, the destination data storage system concurrently receives a respective portion of tiering metadata for a logical extent identified in that replication update.

12. The computer-program product of claim 11,

wherein the portion of tiering metadata for one of the set of logical extents includes data temperature metadata specific to that logical extent, the data temperature metadata for that logical extent indicating a moving average of input/output activity directed to that logical extent on the source data storage system over time, and

wherein selecting a storage tier for that logical extent includes performing an auto-tiering operation on that logical extent, the auto-tiering operation generating an auto-tiering output that prescribes, based on the temperature metadata for that logical extent, a storage tier on which to place the file data of that logical extent in the destination data storage system.

13. The computer-program product of claim 12,

wherein the portion of tiering metadata for the logical extent further includes QoS metadata for that logical extent, the QoS metadata for that logical extent indicating a storage tier on which a source QoS management operation on the source data storage system prescribes placement of the file data of that logical extent on the source data storage system, and

wherein selecting a storage tier for that logical extent includes performing a destination QoS-management operation on that logical extent, the destination QoS-management operation generating a QoS output that prescribes, based on the QoS metadata for that logical extent, a storage tier on which to place the file data of that logical extent on the destination data storage system.

14. The computer-program product of claim 13, wherein the portion of tiering metadata for the logical extent further includes tiering policy metadata for that logical extent, the tiering policy metadata for that logical extent prescribing a procedure for placing the file data of that logical extent when the auto-tiering output prescribes a first storage tier and the QoS output prescribes a second storage tier different from the first storage tier.

15. The computer-program product of claim 14, wherein, when placing the file data for each of the set of logical extents on one of the storage tiers of the destination data storage system, the destination data storage system does not temporarily place the file data on a storage tier without regard to the tiering metadata, but rather places the file data directly on a storage tier selected in accordance with the respective portion of tiering metadata received for that logical extent.

16. The computer program product of claim 12, wherein receiving the replication updates includes receiving a snapshot-shipping update, the snapshot-shipping update providing a list of logical extents of the file, the list of logical extents (i) providing a subset of the set of logical extents, and (ii) identifying, for each listed logical extent, (a) a logical offset range of that listed logical extent within the file and (b) file data for that listed logical extent in the first data storage system.

17. The computer program product of claim 16, wherein receiving the snapshot-shipping update includes concurrently receiving a portion of tiering metadata for each of the logical extents provided in the list of logical extents.

18. The method of claim 11, wherein each replication update is received together with a respective portion of tiering metadata in a single transmission from the source data storage system.