Method, system, and program for improved throughput in remote mirroring systems

• US 20040181640A1
• Filed: 03/11/2003
• Published: 09/16/2004
• Est. Priority Date: 03/11/2003
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: International Business Machines Corporation
• Inventors: Zlotnick, Aviad, Micka, William Frank, Jarvis, Thomas Charles, Hyde, Joseph Smith II, Fienblit, Shachar, Spear, Gail Andrea, Factor, Michael E., Gluck, Olympia, Tross, Martin Jacob
• Granted Patent: US 6,996,688 B2
• Time in Patent Office: Days
• US Class Current: 711/162
• CPC Class Codes: G06F 11/1451 by selection of backup contents
• CPC Class Codes: G06F 11/1658 Data re-synchronization of a redundant component, or initial sync of replacement, additional or spare unit
• CPC Class Codes: G06F 11/1666 where the redundant component is memory or memory area
• CPC Class Codes: G06F 11/20 using active fault-masking, e.g. by switching out faulty elements or by switching in spare elements
• CPC Class Codes: G06F 11/2066 Optimisation of the communication load
• CPC Class Codes: G06F 11/2074 Asynchronous techniques

Abstract

Disclosed is a method, system, and program for reducing an mount of data transferred. Indicators of sub-blocks modified for a block of data since the block of data was last sent to remote storage are maintained until the block of data is demoted. The maintained indicators are used to identify sub-blocks on the block of data for transfer to the remote storage.

Disclosed is a method, system, and program for managing metadata in cache. A first policy is used to determine when to remove data from a primary cache, and a second policy is used to determine when to remove data from a metadata cache.

Claims

1. A method for reducing an amount of data transferred, comprising:

maintaining indicators of sub-blocks modified for a block of data since the block of data was last sent to remote storage until the block of data is demoted; and

using the maintained indicators to identify sub-blocks on the block of data for transfer to the remote storage.

2. The method of claim 1, further comprising:

transferring sub-blocks of the block of data whose indicators indicate that the sub-blocks have been modified since the block of data was last sent to the remote storage.

3. The method of claim 1, further comprising:

receiving a host request to access the block of data.

4. The method of claim 3, further comprising:

determining that the block of data does not exist in a primary cache; and

promoting the block of data to the primary cache.

5. The method of claim 4, wherein promoting further comprises:

determining whether an out of sync indicator indicates that the block of data has been modified; and

if the out of sync indicator indicates that the block of data has been modified, setting indicators in a copy of a modified sub-blocks structure to indicate that all sub-blocks of the block of data have been modified since the block of data was last transferred to the remote storage.

6. The method of claim 3, wherein the host request comprises a write request, and further comprising:

writing the block of data;

updating indicators for sub-blocks written by the host in a modified sub-blocks structure; and

updating the indicators for sub-blocks written by the host in a copy of the modified sub-blocks structure.

7. The method of claim 6, wherein updating the indicators further comprises:

determining whether an out of sync indicator indicates that the block of data has been modified and whether all indicators in a copy of a modified sub-blocks structure are set to indicate that no sub-blocks in the block of data have been modified since the block of data was last transferred to the remote storage; and

if the out of sync indicator indicates that the block of data has been modified and the indicators in the copy of the modified sub-blocks structure are set to indicate that no sub-blocks in the block of data have been modified, setting indicators in the copy of the modified sub-blocks structure to indicate that all sub-blocks of the block of data have been modified since the block of data was last transferred to the remote storage.

8. The method of claim 6, wherein updating the indicators for sub-blocks written by the host in the copy of the modified sub-blocks structure occurs at or before updating.

9. The method of claim 6, wherein updating the indicators for sub-blocks written by the host in the copy of the modified sub-blocks structure occurs after updating.

10. The method of claim 3, wherein the host request comprises a read request, further comprising:

determining whether a portion of the block of data to be accessed by the read request is in primary cache;

if the portion of the block of data to be accessed is in the primary cache, reading the block of data; and

if the portion of the block of data to be accessed is not in primary cache, staging the portion of the block of data to the primary cache.

11. The method of claim 1, further comprising:

determining that an indicator is set in an out of sync structure for the block of data that indicates that the block of data has been modified since the block of data was last transferred to the remote storage.

12. The method of claim 11, further comprising:

determining that the block of data does not exist in a primary cache; and

promoting the block of data to the primary cache.

13. The method of claim 12, wherein promoting further comprises:

if the out of sync indicator indicates that the block of data has been modified, setting indicators in a copy of a modified sub-blocks structure to indicate that all sub-blocks of the block of data have been modified since the block of data was last transferred to the remote storage.

14. The method of claim 13, further comprising:

determining whether all sub-blocks of the block of data that have been modified since the block of data was last sent to the remote storage are in primary cache; and

if not all of the sub-blocks are in the primary cache, staging the sub-blocks that are not already in the primary cache into the primary cache.

15. The method of claim 14, further comprising:

transferring sub-blocks of the block of data whose indicators are set in a remote mirroring system modified sub-blocks structure to indicate that the sub-blocks have been modified since the block of data was last sent to the remote storage; and

clearing the indicators to indicate that the sub-blocks have not been modified since block of data was last transferred to the remote storage.

16. The method of claim 12, further comprising:

determining whether all indicators in a copy of a modified sub-blocks structure are set to indicate that no sub-blocks in the block of data have been modified since the block of data was last transferred to the remote storage; and

if the out of sync indicator indicates that the block of data has been modified and the indicators in the copy of the modified sub-blocks structure are set to indicate that no sub-blocks in the block of data have been modified, setting indicators in the copy of the modified sub-blocks structure to indicate that all sub-blocks of the block of data have been modified since the block of data was last transferred to the remote storage.

17. The method of claim 16, further comprising:

determining whether all sub-blocks of the block of data that have been modified since the block of data was last sent to the remote storage are in primary cache; and

if not all of the sub-blocks are in the primary cache, staging the sub-blocks that are not already in the primary cache into the primary cache.

18. The method of claim 17, further comprising:

transferring sub-blocks of the block of data whose indicators are set in a remote mirroring system modified sub-blocks structure to indicate that the sub-blocks have been modified since the block of data was last sent to the remote storage; and

clearing the indicators to indicate that the sub-blocks have not been modified since block of data was last transferred to the remote storage.

19. The method of claim 1, wherein the block of data comprises a first block of storage and further comprising:

determining that a second block of data is to be promoted into primary cache; and

setting all indicators to indicate that sub-blocks for the second block of data have not been modified since the second block of data was last transferred to remote storage.

20. The method of claim 19, further comprising:

determining whether an indicator in an out of sync structure indicates that the block of data has been modified; and

if the indicator indicates that the block of data has been modified, determining whether a metadata cache modified sub-blocks structure is available for the block of data.

21. The method of claim 20, further comprising:

if the metadata cache modified sub-blocks structure is not available, setting all indicators in a remote mirroring system modified sub-blocks structure for the block of data to indicate that all sub-blocks have been modified since the block of data was last transferred to the remote storage.

22. The method of claim 20, further comprising:

if the metadata cache modified sub-blocks structure is available, copying the metadata cache modified sub-blocks structure to a remote mirroring system modified sub-blocks structure.

23. The method of claim 22, further comprising:

removing the metadata cache modified sub-blocks structure from a metadata cache.

24. The method of claim 20, further comprising:

if the metadata cache modified sub-blocks structure is available, linking the metadata cache modified sub-blocks structure to a remote mirroring system modified sub-blocks structure.

25. The method of claim 1, further comprising:

demoting a block of data from a primary cache; and

adding a remote mirroring system modified sub-blocks structure to a metadata cache.

26. A method for managing metadata in cache, comprising:

using a first policy to determine when to remove data from primary cache; and

using a second policy to determine when to remove data from metadata cache.

27. The method of claim 26, wherein use of different policies for the metadata cache and the primary cache reduces an amount of data transferred from the primary cache to remote storage.

28. The method of claim 26, wherein the data removed from the primary cache comprises one or more blocks of data.

29. The method of claim 26, wherein the first policy comprises a least recently used policy.

30. The method of claim 26, wherein the data removed from the metadata cache comprises a metadata cache modified sub-blocks structure.

31. The method of claim 26, wherein the second policy comprises a least recently used policy.

32. The method of claim 26, wherein the second policy comprises a most recently used policy.

33. The method of claim 26, wherein the second policy comprises a penalty based policy.

34. The method of claim 33, wherein the second policy comprises removing a metadata cache modified sub-blocks structure with a largest number of indicators indicating that sub-blocks have been modified since a block of data was last transferred to remote storage.

35. A system for reducing an amount of data transferred, comprising:

means for maintaining indicators of sub-blocks modified for a block of data since the block of data was last sent to remote storage until the block of data is demoted; and

means for using the maintained indicators to identify sub-blocks on the block of data for transfer to the remote storage.

36. A system for managing metadata in cache, comprising:

means for using a first policy to determine when to remove data from primary cache; and

means for using a second policy to determine when to remove data from metadata cache.

37. An article of manufacture for reducing an amount of data transferred, wherein the article of manufacture causes operations, the operations comprising:

maintaining indicators of sub-blocks modified for a block of data since the block of data was last sent to remote storage until the block of data is demoted; and

using the maintained indicators to identify sub-blocks on the block of data for transfer to the remote storage.

38. An article of manufacture for managing metadata in cache, wherein the article of manufacture causes operations, the operations comprising:

using a first policy to determine when to remove data from primary cache; and

using a second policy to determine when to remove data from metadata cache.