Adaptive cache engine for storage area network including systems and methods related thereto

• US 20050243609A1
• Filed: 02/09/2005
• Published: 11/03/2005
• Est. Priority Date: 05/03/2004
• Status: Active Grant

• Current Assignee: Gemini Storage, Slater Technology Fund, Inc.
• Original Assignee: Gemini Storage
• Inventors: Zhang, Ming, Yang, Ken Q., Yinan, Liu
• Granted Patent: US 7,370,163 B2
• Time in Patent Office: Days
• US Class Current: 365/189.50
• CPC Class Codes: G06F 12/0866 for peripheral storage systems, e.g. disk cache
• CPC Class Codes: G06F 3/0617 in relation to availability
• CPC Class Codes: G06F 3/0643 Management of files
• CPC Class Codes: G06F 3/065 Replication mechanisms
• CPC Class Codes: G06F 3/067 Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
• CPC Class Codes: H04L 67/1095 Replication or mirroring of data, e.g. scheduling or transport for data synchronisation between network nodes
• CPC Class Codes: H04L 67/1097 for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]

Abstract

Featured is a data storage back-up system for replication, mirroring and/or backing-up data including one or more first and second data storage devices that embody iSCSI, FC or alike principals and that are operably coupled to each other preferably via a WAN. The first data storage device is configured and arranged so there are two writes of data, one write to a persistent storage device from which reads are done and another write to a SAPS device wherein the data is saved using log-structured file system (LSF) techniques. After saving data to the First storage device, the data logs in the SAPS device are communicated to the second data storage device whereat a de-staging process is conducted so as to de-stage the data logs and write the de-staged data to a persistent storage device in the second data storage device.

Claims

1. A data storage system that is adaptable for one of replicating, mirroring or back-up data, said data storage system comprising;

a first data storage device that includes a persistent storage and a LSF storage where data is saved using log-structured file system (LSF) techniques;

wherein the first data storage device is configured and arranged so as to be responsive to I/O requests being transmitted thereto;

wherein the first data storage device is configured and arranged so as to two perform writes responsive to each write I/O request, one write to the persistent storage and another write to the LSF storage.

2. The data storage system of claim 1, further comprising a second data storage device that is operably coupled to the first data storage device;

wherein the second data storage device includes at least a persistent storage;

wherein the second data storage device is configured and arranged so as to receive data from the LSF device of the first data storage device and for replicating at least some data being stored in the persistent storage of the first data storage device in the persistent storage of the second data storage device.

3. The data storage system of claim 2, wherein:

the second data storage device is configured so as to include a LSF storage;

the second data storage device is configured and arranged so that the data from the LSF storage of the first data storage device is written to the LSF storage of the second data storage device; and

the second data storage device is configured and arranged so as to de-stage the data stored in the LSF storage of the second data storage device and to write the de-staged data to the persistent storage of the second data storage device.

4. The data storage system of claim 3, wherein the LSF storage of the first data storage device includes a cache and a sequentially accessed persistent storage (SAPS) device being operably coupled to the cache, said SAPS device being characterized in that data is stored in a sequential data layout.

5. The data storage system of claim 4, wherein the SAPS device and the persistent storage of the first data storage device is one of one or more magnetic hark disk drives or one or more magneto-optical disk drives.

6. The data storage system of claim 3, wherein the LSF storage of the second data storage device includes a cache and a sequentially accessed persistent storage (SAPS) device that is operably coupled thereto, said SAPS device being characterized in that data is stored in a sequential data layout.

7. The data storage system of claim 6, wherein the SAPS device and the persistent storage of the second data storage device is one of one or more magnetic hark disk drives or one or more magneto-optical disk drives.

8. The data storage system of claim 1, wherein the first data storage device is operably coupled via a network to one or more computers/workstations and wherein the first data storage device is configured and arranged so as to be responsive to received I/O requests transmitted via the network from the one or more computer/workstations.

9. The data storage system of claim 1, wherein:

the first data storage device is operably coupled via a network to one or more computers/workstations and wherein the first data storage device is configured and arranged so as to be responsive to received I/O requests transmitted via the network from the one or more computer/workstations; and

wherein the first data storage device is operably coupled to the second data storage device via a network, wherein data being transmitted from the first data storage device to the second data storage device is sent via the network.

10. The data storage system of claim 1, wherein the persistent storage of the first data storage device includes a cache and a persistent storage device being operably coupled to the cache.

11. The data storage system of claim 2, wherein the system comprises N first data storage devices and M second storage devices, wherein N≧

1 and M≧

1.

12. The data storage system of claim 2, wherein the system comprises N first data storage devices and M second storage devices and wherein one of N=1 and M≧

1 or N≧

1 and M=1.

13. The data storage system of claim 2, wherein the system comprises N first data storage devices and M second storage devices, wherein N≧

1 and M≧

1 and wherein one of N=M or N≠

M.

14. The data storage device of claim 11, wherein N≧

2 and M≧

2.

15. The data storage device of claim 1, wherein the first data storage device further includes a microprocessor that is operably coupled to the persistent storage and the LSF storage and an application program for execution on the microprocessor, and the application program including instructions, criteria and code segments for:

controlling writing to the persistent storage and the LSF storage of the first data storage device, and

controlling communication between a source of the I/O request and the first data storage device.

16. The data storage system of claim 13, wherein the application program further includes instructions, criteria and code segments for:

controlling reads from the persistent storage, and

communicating the read data from the first data storage device to a requestor.

17. The data storage system of claim 13, wherein the LSF storage of the first data storage device includes a cache and a sequentially accessed persistent storage (SAPS) device that is operably coupled thereto, said SAPS device being characterized in that data is stored in a sequential data layout, and wherein the application program further includes instructions, criteria and code segments for:

creating data logs for data being received and written to the LSF storage, and

buffering the created data logs in the cache.

18. The data storage device of claim 15, wherein the application program further includes instructions, criteria and code segments for:

evaluating content of the cache to determine if a condition is satisfied, and

when it is determined that the condition is satisfied, causing the data logs in the cache to be sequentially written to the SAPS device.

19. The data storage device of claim 15, wherein the application program further includes instructions, criteria and code segments for:

evaluating content of the cache to determine if a condition is satisfied,

when it is determined that the condition is satisfied, causing the data logs in the cache to be sequentially written to the SAPS device, and

when it is determined that the condition is not satisfied, causing data logs to be continued to be acquired in the cache.

20. The data storage device of claim 15, wherein the application program further includes instructions, criteria and code segments for:

transmitting data stored in the SAPS device to a second data storage device.

21. The data storage device of claim 18, wherein the second data storage device includes a persistent storage and wherein the second data storage device is configured and arranged so as to process and to write the transmitted data to the persistent storage of the second data storage device.

22. The data storage device of claim 15, wherein the application program further includes instructions, criteria and code segments for:

determining if a condition is satisfied for causing data in the SAPS device to be written to a second data storage device, and

when it is determined that the condition is satisfied, causing the data in the SAPS device to be transmitted to the second data storage device.

23. The data storage device of claim 20, wherein the second data storage device includes a persistent storage is configured and arranged so as to process and to write the transmitted data to the persistent storage of the second data storage device.

24. The data storage device of claim 2:

wherein the first data storage device further includes a microprocessor that is operably coupled to the persistent storage and the LSF storage and an application program for execution on the microprocessor, and the application program including instructions, criteria and code segments for;

controlling writing to the persistent storage and the LSF storage of the first data storage device,

controlling communication between a source of the I/O request and the first data storage device, and

controlling transmission of data from the first data storage device to the second data storage device; and

wherein the second data storage device further includes an LSF storage, a microprocessor that is operably coupled to the persistent storage and the LSF storage and an application program for execution on the microprocessor, and the application program including instructions, criteria and code segments for;

controlling writing to the LSF storage of the second data storage device, and

controlling de-staging of data stored in the LSF storage to the persistent storage.

25. The data storage system of claim 22, wherein the LSF storage of the first data storage device includes a cache and a sequentially accessed persistent storage (SAPS) device that is operably coupled thereto, said SAPS device being characterized in that data is stored in a sequential data layout, and wherein the application program including instructions, criteria and code segments for:

creating data logs for data being received and written to the LSF storage, and

buffering the created data logs in the cache.

26. The data storage device of claim 23, wherein the application program for the first data storage device includes instructions, criteria and code segments for:

evaluating content of the cache to determine if a condition is satisfied, and

when it is determined that the condition is satisfied, causing the data logs in the cache to be sequentially written to the SAPS device.

27. The data storage device of claim 23, wherein the application program for the first data storage device includes instructions, criteria and code segments for:

evaluating content of the cache to determine if a condition is satisfied,

when it is determined that the condition is satisfied, causing the data logs in the cache to be sequentially written to the SAPS device, and

when it is determined that the condition is not satisfied, causing data logs to be continued to be acquired in the cache.

28. The data storage device of claim 22, wherein the application program including instructions, criteria and code segments for:

determining if a condition is satisfied for causing data in the SAPS device to be written to a second data storage device, and

when it is determined that the condition is satisfied, causing the data in the SAPS device to be transmitted to the second data storage device.

29. The data storage system of claim 22, wherein the LSF storage of the second data storage device includes a cache and a sequentially accessed persistent storage (SAPS) device that is operably coupled thereto, said SAPS device being characterized in that data is stored in a sequential data layout, and wherein the application program including instructions, criteria and code segments for:

processing the transmitted data so as to re-create the data logs that were written to the LSF storage of the first data storage device, and

buffering the re-created data logs in the cache of the second data storage device.

30. The data storage device of claim 27, wherein the application program for the second data storage device includes instructions, criteria and code segments for:

evaluating content of the cache to determine if a condition is satisfied, and

when it is determined that the condition is satisfied, causing the data logs buffered in the cache to be sequentially written to the SAPS device of the second data storage device.

31. The data storage device of claim 28, wherein the application program for the second data storage device includes instructions, criteria and code segments for:

evaluating the second data storage device to determine if a condition is satisfied, and

when it is determined that the condition is satisfied, reading the data logs from the SAPS device of the second data storage device, processing the data contained in the data logs and writing the processed data to the persistent storage of the second data storage device.

32. The data storage device of claim 29, wherein said processing of the data includes reformatting the data so as to be in the appropriate format for writing to the persistent storage and ordering the data so as to be in the proper order.

34. The data storage system of claim 31, wherein the SAPS device and the persistent storage of the first and second data storage device is one of one or more magnetic hark disk drives or one or more magneto-optical disk drives.

35. The data storage system of claim 31, wherein:

wherein the first data storage device is operably coupled to the second data storage device via a network, wherein data being transmitted from the first data storage device to the second data storage device is sent via the network.

36. The data storage system of claim 31, wherein the system comprises N first data storage devices and M second storage devices, wherein N≧

1 and M≧

1.

37. The data storage system of claim 31, wherein the system comprises N first data storage devices and M second storage devices and wherein one of N=1 and M≧

1 or N≧

1 and M=1.

38. The data storage system of claim 31, wherein the system comprises N first data storage devices and M second storage devices, wherein N≧

1 and M≧

1 and wherein one of N=M or N≠

M.

39. The data storage device of claim 36, wherein N≧

2 and M≧

2.

40. The data storage device of claim 1, wherein the application program for the first data storage device further includes instructions, criteria and code segments for:

controlling writing to the persistent storage and the LSF storage of the first data storage device, and

controlling communication between a source of the I/O request and the first data storage device.

41. The data storage system of claim 38, wherein the application program for the first data storage device further includes instructions, criteria and code segments for:

controlling reads from the persistent storage, and

communicating the read data from the first data storage device to a requester.

42. The data storage system of claim 38, wherein the application program for the first data storage device further includes instructions, criteria and code segments for:

creating data logs for data being received and written to the LSF storage, and

buffering the created data logs in the cache.

43. The data storage device of claim 40, wherein the application program for the first data storage device further includes instructions, criteria and code segments for:

evaluating content of the cache to determine if a condition is satisfied, and

when it is determined that the condition is satisfied, causing the data logs in the cache to be sequentially written to the SAPS device.

44. The data storage device of claim 40, wherein the application program for the first data storage device further includes instructions, criteria and code segments for:

evaluating content of the cache to determine if a condition is satisfied,

when it is determined that the condition is satisfied, causing the data logs in the cache to be sequentially written to the SAPS device, and

when it is determined that the condition is not satisfied, causing data logs to be continued to be acquired in the cache.

45. The data storage device of claim 31, wherein the application program for the second data storage device further includes instructions, criteria and code segments for:

controlling writing to the LSF storage of the second data storage device, and

controlling de-staging of data stored in the LSF storage to the persistent storage.

46. The data storage system of claim 31, wherein the application program for the second data storage device further includes instructions, criteria and code segments for:

processing the transmitted data so as to re-create the data logs that were written to the LSF storage of the first data storage device, and

buffering the re-created data logs in the cache of the second data storage device.

47. The data storage device of claim 44, wherein the application program for the second data storage device further includes instructions, criteria and code segments for:

evaluating content of the cache to determine if a condition is satisfied, and

when it is determined that the condition is satisfied, causing the data logs buffered in the cache to be sequentially written to the SAPS device of the second data storage device.

48. The data storage device of claim 45, wherein the application program for the second data storage device further includes instructions, criteria and code segments for:

evaluating the second data storage device to determine if a condition is satisfied, and

when it is determined that the condition is satisfied, reading the data logs from the SAPS device of the second data storage device, processing the data contained in the data logs and writing the processed data to the persistent storage of the second data storage device.

49. The data storage device of claim 46, wherein said processing of the data includes reformatting the data so as to be in the appropriate format for writing to the persistent storage and ordering the data so as to be in the proper order.

33. A data storage system that is adaptable for one of replicating, mirroring or back-up data, said data storage system comprising a first data storage device that is operably coupled via a network to one or more computers/workstations and a second data storage device that is operably coupled to the first data storage device;

wherein the first data storage device includes a persistent storage device, a LSF storage where data is saved using log-structured file system (LSF) techniques, and a microprocessor that is operably coupled to the persistent storage device and the LSF storage;

wherein the LSF storage includes a cache and a sequentially accessed persistent storage (SAPS) device being operably coupled to the cache, said SAPS device being characterized in that data is stored in a sequential data layout;

wherein the second data storage device includes a persistent storage device, a LSF storage where data is saved using log-structured file system (LSF) techniques, and a microprocessor that is operably coupled to the persistent storage device and the LSF storage;

wherein the LSF storage of the second data storage device includes a cache and a sequentially accessed persistent storage (SAPS) device being operably coupled to the cache, said SAPS device being characterized in that data is stored in a sequential data layout;

wherein the first data storage device further includes an application program for execution on the microprocessor, the application program including instructions, criteria and code segments for;

controlling operation of the first data storage device responsive to received I/O requests transmitted via the network from the one or more computer/workstations,

performing writes responsive to each write I/O request, one write to the persistent storage and another write to the LSF storage,

determining if a condition is satisfied for causing data in the SAPS device to be written to the second data storage device, and

when it is determined that the condition is satisfied, causing the data in the SAPS device to be transmitted to the second data storage device;

wherein the second data storage device is configured and arranged so as to receive data from the LSF device of the first data storage device and for replicating at least some data being stored in the persistent storage of the first data storage device in the persistent storage of the second data storage device; and

wherein the second data storage device further includes an application program for execution on the microprocessor, the application program including instructions, criteria and code segments for;

controlling operation of the second data storage, and

controlling writes responsive to each of the persistent storage and the LSF storage.