Method and apparatus for reducing overheads of primary storage by transferring modified data in an out-of-order manner

• US 9,235,535 B1
• Filed: 10/31/2012
• Issued: 01/12/2016
• Est. Priority Date: 10/31/2012
• Status: Active Grant

• Current Assignee: Emc IP Holding Company LLC (Dell Technologies Inc.)
• Original Assignee: EMC Corporation (Dell Technologies Inc.)
• Inventors: Shim, Hyong, Shilane, Philip N., Hsu, Windsor W.
• Primary Examiner(s): Verbrugge, Kevin
• Assistant Examiner(s): Ahmed, Zubair
• Application Number:
  US13/665,885
• Time in Patent Office: 1,168 Days
• Field of Search: 711/162
• US Class Current: 1/1
• CPC Class Codes: G06F 11/1451 by selection of backup contents
• CPC Class Codes: G06F 11/1453 using de-duplication of the data
• CPC Class Codes: G06F 11/1464 for networked environments
• CPC Class Codes: G06F 12/16 Protection against loss of memory contents contains no material, see G06F11/00
• CPC Class Codes: G06F 2201/84 Using snapshots, i.e. a logical point-in-time copy of the data
• CPC Class Codes: G06F 7/08 Sorting, i.e. grouping record carriers in numerical or other ordered sequence according to the classification of at least some of the information they carry by merging two or more sets of carriers in ordered sequence G06F7/16
• CPC Class Codes: G06F 7/24 Sorting, i.e. extracting data from one or more carriers, rearranging the data in numerical or other ordered sequence, and rerecording the sorted data on the original carrier or on a different carrier or set of carriers sorting methods in generalG06F7/36 takes precedence

Abstract

Techniques for reducing overheads of primary storage transferring during a backup by transferring in an out-of-order manner are described herein. According to one embodiment, in response to a request at a primary storage for a backup of a plurality of data blocks, a transfer order of dirty data blocks is determined based on an access assessment of the dirty data blocks, wherein the dirty data blocks are data blocks of the plurality of data blocks that have been changed from a previous backup, and wherein the transfer order is different from a sequential logical order of the dirty data blocks provided by the primary storage. Then the dirty data blocks are transferred to a secondary storage in the determined order.

Claims

1. A computer-implemented method, comprising:

receiving a request at a primary storage for a backup of a plurality of data blocks to a secondary storage, the plurality of data blocks comprising a plurality of dirty data blocks, wherein a dirty data block comprises a data block that has changed since a previous backup of the data block;

backing up the plurality of dirty data blocks to the secondary storage, wherein the backing up comprises;

determining a transfer order of the dirty data blocks wherein a dirty data block is given a higher priority for transfer based on a proximity of the dirty data block to a hot region on the primary storage, and wherein the transfer order is different from a sequential logical order of the dirty data blocks provided by the primary storage; and

transferring the dirty data blocks to the secondary storage in the determined transfer order, wherein the secondary storage is a deduplication storage system.

2. The method of claim 1, wherein a dirty data block is in proximity to the hot region of the primary storage based on at least one of:

the dirty data block is physically close to a frequently accessed region of the primary storage;

the dirty data block is physically close to another dirty data block on the primary storage;

the dirty data block is physically close to a data block being written to primary storage;

the dirty data block is physically close to a region of the primary storage that frequently changes state, or

the dirty data block is physically close a number of dirty data blocks on the primary storage.

3. The method of claim 1, wherein an access assessment of a dirty data block is determined based on readiness of a stable read resource for the dirty data block.

4. The method of claim 1, wherein a dirty data block is given a higher priority based on at least one of:

the dirty data block is physically close to a region of the primary storage containing frequently updated metadata;

contents of the dirty data block containing metadata about a file having one or more dirty data blocks.

5. The method of claim 1, further comprising capturing a snapshot of the plurality of data blocks, wherein the snapshot represents a consistent state of the plurality of data blocks.

6. The method of claim 1, further comprising:

identifying the dirty data blocks by marking each data block in a transfer state if the data block is dirty and marking each data block in a normal state if the data block is not dirty, wherein only data blocks in the transfer state are transferred to the secondary storage; and

marking the dirty data blocks in the normal state after the dirty data blocks have been transferred to the secondary storage.

7. The method of claim 6, wherein markings of the data blocks are stored in a bit vector having a plurality of bits, each corresponding to one of the data blocks, wherein a bit having a first logical value indicates that the corresponding data block is dirty, and wherein a bit having a second logical value indicates that the corresponding data block is not dirty.

8. The method of claim 6, wherein markings of the data blocks are stored in a bit vector having a plurality of bits, each corresponding to one of the data blocks, wherein a bit having a first logical value indicates that the corresponding data block is the transfer state, and wherein a bit having a second logical value indicates that the corresponding data block is in the normal state.

9. The method of claim 6, further comprising:

in response to receiving data to be written to a first data block at the primary storage during the transfer,

preserving current contents of the first data block if the first data block is in the identified set of data blocks; and

committing the received data to the first data block.

10. The method of claim 6, wherein marking of dirty data blocks from the transfer state to the normal state is performed after all dirty data blocks have been transferred from the primary storage to the secondary storage.

11. The method of claim 6, wherein when a particular dirty data block has been transferred from the primary storage to the secondary storage, that particular dirty data block is marked from the transfer state to the normal state, while remaining dirty data blocks are still being transferred.

12. The method of claim 1, wherein the secondary storage is one of:

a backup storage system, a replica of the primary storage, an archival storage system, and a snapshot repository of the primary storage.

13. A non-transitory computer-readable storage medium having instructions stored therein, which when executed by a processor, cause the processor to perform a method, the method comprising:

receiving a request at a primary storage for a backup of a plurality of data blocks to a secondary storage, the plurality of data blocks comprising a plurality of dirty data blocks, wherein a dirty data block comprises a data block that has changed since a previous backup of the data block;

backing up only the plurality of dirty data blocks to the secondary storage, wherein the backing up comprises;

determining a transfer order of the dirty data blocks wherein a dirty data block is given a higher priority for transfer based on a proximity of the dirty data block to a hot region on the primary storage, and wherein the transfer order is different from a sequential logical order of the dirty data blocks provided by the primary storage; and

transferring the dirty data blocks to the secondary storage in the determined transfer order, wherein the secondary storage is a deduplication storage system.

14. The non-transitory computer-readable storage medium of claim 13, wherein a dirty data block is in proximity to the hot region of the primary storage based on at least one of:

the dirty data block is physically close to a frequently accessed region of the primary storage;

the dirty data block is physically close to another dirty data block;

the dirty data block is physically close to a data block being written primary storage;

the dirty data block is physically close to a region of the primary storage containing frequently updated metadata;

the dirty data block is physically close to a region of the primary storage that frequently changes state, or

the dirty data block is physical closeness to a number of dirty data blocks.

15. The non-transitory computer-readable storage medium of claim 13, wherein an access assessment of a dirty data block is determined based on readiness of a stable read resource for the dirty data block.

16. The non-transitory computer-readable storage medium of claim 13, wherein a dirty data block is given a higher priority based on at least one of:

the dirty data block is physically close to a region of the primary storage containing frequently updated metadata; and

contents of the dirty data block containing metadata about a file having one or more dirty data blocks.

17. The non-transitory computer-readable storage medium of claim 13, further comprising capturing a snapshot of the plurality of data blocks, wherein the snapshot represents a consistent state of the plurality of data blocks.

18. The non-transitory computer-readable storage medium of claim 13, further comprising:

identifying the dirty data blocks by marking each data block in a transfer state if the data block is dirty and marking each data block in a normal state if the data block is not dirty, wherein only data blocks in the transfer state are transferred to the secondary storage; and

marking the dirty data blocks in the normal state after the dirty data blocks have been transferred to the secondary storage.

19. A primary storage, comprising:

a data protection logic configured to receive a request at the primary storage for a backup of a plurality of data blocks to a secondary storage, the plurality of data blocks comprising a plurality of dirty data blocks, wherein a dirty data block comprises a data block that has changed since a previous backup of the data block;

backup the plurality of dirty data blocks to the secondary storage, wherein the backup comprises;

a transfer order determinator configured to determine a transfer order of the dirty data blocks wherein a dirty data block is given a higher priority for transfer based on a proximity of the dirty data block to a hot region on the primary storage, and wherein the transfer order is different from a sequential logical order of the dirty data blocks provided by the primary storage; and

the data protection logic further configured to transfer the dirty data blocks to the secondary storage in the determined transfer order, wherein the secondary storage is a deduplication storage system.

20. The primary storage of claim 19, wherein a dirty data block is in proximity to the hot region of the primary storage based on at least one of:

the dirty data block is physically close to a frequently accessed region of the primary storage;

the dirty data block is physically close to another dirty data block on the primary storage;

the dirty data block is physically close to a data block being written to the primary storage;

the dirty data block is physically close to a region of the primary storage that frequently changes state, or

the dirty data block is physically close to a number of dirty data blocks on the primary storage.

21. The primary storage of claim 19, wherein an access assessment of a dirty data block is determined based on readiness of a stable read resource for the dirty data block.

22. The primary storage of claim 19, wherein a dirty data block is in proximity to the hot region of the primary storage based on at least one of:

the dirty data block is physically close to a region of the primary storage containing frequently updated metadata;

contents of the dirty data block containing metadata about a file having one or more dirty data blocks.

23. The primary storage of claim 19, wherein the data protection logic further configured to capture a snapshot of the plurality of data blocks, wherein the snapshot represents a consistent state of the plurality of data blocks.

24. The primary storage of claim 19, further comprising:

a change tracker configured to identify the dirty data blocks by marking each data block in a transfer state if the data block is dirty and marking each data block in a normal state if the data block is not dirty, wherein only data blocks in the transfer state are transferred to the secondary storage; and

mark the dirty data blocks in the normal state after the dirty data blocks have been transferred to the secondary storage.