DATA STORAGE OPTIMIZATION USING REPLICATION STATISTICS TO AUTOMATICALLY GENERATE NVMe STREAM IDENTIFIERS

• US 20200133668A1
• Filed: 10/24/2018
• Published: 04/30/2020
• Est. Priority Date: 10/24/2018
• Status: Active Grant

• Current Assignee: Emc IP Holding Company LLC (Dell Technologies Inc.)
• Original Assignee: Emc IP Holding Company LLC (Dell Technologies Inc.)
• Inventors: Chen, Xiangping, Zhao, Junping
• Granted Patent: US 10,664,268 B2
• Time in Patent Office: Days
• CPC Class Codes: G06F 16/137 Hash-based content-based indexing of textual data G06F16/31
• CPC Class Codes: G06F 16/273 Asynchronous replication or reconciliation
• CPC Class Codes: G06F 3/0619 in relation to data integrity, e.g. data losses, bit errors
• CPC Class Codes: G06F 3/0626 Reducing size or complexity of storage systems
• CPC Class Codes: G06F 3/065 Replication mechanisms
• CPC Class Codes: G06F 3/067 Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
• CPC Class Codes: G06F 9/30 Arrangements for executing machine instructions, e.g. instruction decode for executing microinstructions G06F9/22

Abstract

An aspect of optimizing storage of data in a data replication system includes, for a plurality of write requests received from a source site, determining transfer statistics corresponding to each of the write requests and updating a table with the transfer statistics. An aspect also includes grouping pages in the table having common transfer statistics, assigning a unique non-volatile memory express (NVMe) stream identifier (ID) to each of the groups, and identifying grouped pages based on the assigned NVMe stream ID. An aspect further includes selecting a storage optimization technique for each of the groups based on the common transfer statistics and storing data of the write requests for each of the groups according to the selected optimization technique.

Claims

1. A method for optimizing storage of data in a data storage replication system including a source site and a target site, comprising:

for a plurality of write requests received from the source site;

determining transfer statistics corresponding to each of the write requests; and

updating a table with the transfer statistics;

grouping pages in the table having common transfer statistics;

assigning, at the target site, a unique non-volatile memory express (NVMe) stream identifier (ID) to each of the groups;

identifying, at the target site, grouped pages based on the assigned NVMe stream ID;

selecting a storage optimization technique for each of the groups based on the common transfer statistics; and

storing the data of the write requests for each of the groups according to the selected optimization technique;

wherein the transfer statistics include a frequency of updates for the address ranges, the frequency of updates determined, in an asynchronous replication process, by modifications made to a dirty tree of address-to-hash metadata.

2. The method of claim 1, further comprising:

receiving a new write request;

determining transfer statistics for the new write request;

determining whether the transfer statistics of the new write request fall within one of the groups of the table;

upon determining the transfer statistics of the new write request falls within one of the groups of the table;

assigning the corresponding NVMe stream ID for the one of the groups to the new write request;

selecting, at the target site, a storage optimization technique for the write request based on the corresponding NVMe stream ID; and

storing the data of the new write request according to the selected storage optimization technique for the new write request.

3. The method of claim 1, further comprising tracking, via the table, the write requests based on address ranges associated with the write requests.

6. The method of claim 3, wherein the transfer statistics include a size of each of the write requests.

7. The method of claim 1, further comprising generating the NVMe stream ID internally at the target site.

4. (canceled)

5. (canceled)

8. A system for optimizing storage of data in a data storage replication system including a source site and a target site, comprising:

a memory comprising computer-executable instructions; and

a processor executing the computer-executable instructions, the computer-executable instructions when executed by the processor cause the processor to perform operations comprising;

for a plurality of write requests received from the source site;

determining transfer statistics corresponding to each of the write requests; and

updating a table with the transfer statistics;

grouping pages in the table having common transfer statistics;

assigning, at the target site, a unique non-volatile memory express (NVMe) stream identifier (ID) to each of the groups;

identifying, at the target site, grouped pages based on the assigned NVMe stream ID;

selecting a storage optimization technique for each of the groups based on the common transfer statistics; and

storing the data of the write requests for each of the groups according to the selected optimization technique;

wherein the transfer statistics include a frequency of updates for the address ranges, the frequency of updates determined, in an asynchronous replication process, by modifications made to a dirty tree of address-to-hash metadata.

9. The system of claim 8, wherein the operations further comprise:

receiving a new write request;

determining transfer statistics for the new write request;

determining whether the transfer statistics of the new write request fall within one of the groups of the table;

upon determining the transfer statistics of the new write request falls within one of the groups of the table;

assigning the corresponding NVMe stream ID for the one of the groups to the new write request;

selecting, at the target site, a storage optimization technique for the write request based on the corresponding NVMe stream ID; and

storing the data of the new write request according to the selected storage optimization technique for the new write request.

10. The system of claim 8, wherein the operations further comprise tracking, via the table, the write requests based on address ranges associated with the write requests.

13. The system of claim 10, wherein the transfer statistics include a size of each of the write requests.

14. The system of claim 8, wherein the operations further comprise generating the NVMe stream ID internally at the target site.

11. (canceled)

12. (canceled)

15. A computer program product embodied on a non-transitory computer readable medium, the computer program product including instructions that, when executed by a computer causes the computer to perform operations for optimizing storage of data in a data storage replication system including a source site and a target site, the operations comprising:

for a plurality of write requests received at the source site;

determining transfer statistics corresponding to each of the write requests; and

updating a table with the transfer statistics;

grouping pages in the table having common transfer statistics;

assigning, at the target site, a unique non-volatile memory express (NVMe) stream identifier (ID) to each of the groups;

identifying, at the target site, grouped pages based on the assigned NVMe stream ID;

selecting a storage optimization technique for each of the groups based on the common transfer statistics; and

storing the data of the write requests for each of the groups according to the selected optimization technique;

wherein the transfer statistics include a frequency of updates for the address ranges, the frequency of updates determined, in an asynchronous replication process, by modifications made to a dirty tree of address-to-hash metadata.

16. The computer program product of claim 15, wherein the operations further comprise:

receiving a new write request;

determining transfer statistics for the new write request;

determining whether the transfer statistics of the new write request fall within one of the groups of the table;

upon determining the transfer statistics of the new write request falls within one of the groups of the table;

assigning the corresponding NVMe stream ID for the one of the groups to the new rite request;

selecting, at the target site, a storage optimization technique for the write request based on the corresponding NVMe stream ID; and

storing the data of the new write request according to the selected storage optimization technique for the new write request.

17. The computer program product of claim 15, wherein the operations further comprise tracking, via the table, the write requests based on address ranges associated with the write requests.

20. The computer program product of claim 17, wherein the transfer statistics include a size of each of the write requests.

18. (canceled)

19. (canceled)