Adaptive cache engine for storage area network including systems and methods related thereto

• US 7,370,163 B2
• Filed: 02/09/2005
• Issued: 05/06/2008
• Est. Priority Date: 05/03/2004
• Status: Expired due to Fees

• Current Assignee: Gemini Storage, Slater Technology Fund, Inc.
• Original Assignee: Gemini Storage
• Inventors: Zhang, Ming, Yang, Ken Q., Yinan, Liu
• Primary Examiner(s): Thai; Tuan V.
• Application Number:
  US11/054,933
• Publication Number:
  US 20050243609A1
• Time in Patent Office: 1,182 Days
• Field of Search: 711/100, 711/118, 711/154, 711/162, 365/189.05
• US Class Current: 711/162
• CPC Class Codes: G06F 12/0866 for peripheral storage systems, e.g. disk cache
• CPC Class Codes: G06F 3/0617 in relation to availability
• CPC Class Codes: G06F 3/0643 Management of files
• CPC Class Codes: G06F 3/065 Replication mechanisms
• CPC Class Codes: G06F 3/067 Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
• CPC Class Codes: H04L 67/1095 Replication or mirroring of data, e.g. scheduling or transport for data synchronisation between network nodes
• CPC Class Codes: H04L 67/1097 for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]

Abstract

Featured is a data storage back-up system for replication, mirroring and/or backing-up data including one or more first and second data storage devices that embody iSCSI, FC or alike principals and that are operably coupled to each other preferably via a WAN. The first data storage device is configured and arranged so there are two writes of data, one write to a persistent storage device from which reads are done and another write to a SAPS device wherein the data is saved using log-structured file system (LSF) techniques. After saving data to the First storage device, the data logs in the SAPS device are communicated to the second data storage device whereat a de-staging process is conducted so as to de-stage the data logs and write the de-staged data to a persistent storage device in the second data storage device.

Claims

1. A data storage system that is adaptable for one of replicating, mirroring or back-up data, said data storage system comprising;

a first data storage device that includes a memory, a persistent storage disk array and a log-structured file system LSF storage disk where data is saved using LSF techniques;

wherein the memory is functionally partitioned so as to form discrete memory partitions, a first partition being for buffering data to be stored in the persistent storage disk array and a second partition being for buffering logs to be sequentially written to the LSF storage disk;

wherein the first data storage device is configured and arranged so as to parse I/O requests being transmitted thereto such that the data is in the first partition and the logs are in the second partition; and

wherein the first data storage device is configured and arranged so as to perform two writes responsive to each write I/O request, one write of the data to the persistent storage disk array and another subsequent write of the logs to the LSF storage disk such that the memory is relatively small.

2. The data storage system of claim 1, further comprising a second data storage device that is operably coupled to the first data storage device;

wherein the second data storage device includes at least a persistent storage disk array;

wherein the second data storage device is configured and arranged so as to receive data from the LSF device of the first data storage device and for replicating at least some data being stored in the persistent storage disk array of the first data storage device in the persistent storage disk array of the second data storage device.

3. The data storage system of claim 2, wherein:

the second data storage device is configured so as to include a LSF storage disk;

the second data storage device is configured and arranged so that the data from the LSF storage disk of the first data storage device is written to the LSF storage disk of the second data storage device; and

the second data storage device is configured and arranged so as to de-stage the data stored in the LSF storage disk of the second data storage device and to write the de-staged data to the persistent storage disk array of the second data storage device.

4. The data storage system of claim 3, wherein the LSF storage disk of the first data storage device includes a cache and a sequentially accessed persistent storage (SAPS) device being operably coupled to the cache, said SAPS device being characterized in that data is stored in a sequential data layout.

5. The data storage system of claim 4, wherein the SAPS device and the persistent storage disk array of the first data storage device is one of one or more magnetic hark disk drives or one or more magneto-optical disk drives.

6. The data storage system of claim 3, wherein the LSF storage disk of the second data storage device includes a cache and a sequentially accessed persistent storage (SAPS) device that is operably coupled thereto, said SAPS device being characterized in that data is stored in a sequential data layout.

7. The data storage system of claim 6, wherein the SAPS device and the persistent storage disk array of the second data storage device is one of one or more magnetic hark disk drives or one or more magneto-optical disk drives.

8. The data storage system of claim 2, wherein the system comprises N first data storage devices and M second storage devices, wherein N≧

1 and M≧

1.

9. The data storage device of claim 8, wherein N≧

2 and M≧

2.

10. The data storage system of claim 2, wherein the system comprises N first data storage devices and M second storage devices and wherein one of N=1 and M ≧

1 or N≧

1 and M=1.

11. The data storage system of claim 2, wherein the system comprises N first data storage devices and M second storage devices, wherein N≧

1 and M≧

1 and wherein one of N=M or N≠

M.

12. The data storage system of claim 11, wherein an application program in the memory further includes instructions, criteria and code segments for:

controlling reads from the persistent storage disk array, and

communicating the read data from the first data storage device to a requestor.

13. The data storage system of claim 11, wherein the LSF storage disk of the first data storage device includes a cache and a sequentially accessed persistent storage (SAPS) device that is operably coupled thereto, said SAPS device being characterized in that data is stored in a sequential data layout, and wherein an application program in the memory further includes instructions, criteria and code segments for:

creating data logs for data being received and written to the LSF storage disk, and

buffering the created data logs in the cache.

14. The data storage device of claim 2:

wherein the first data storage device further includes a microprocessor that is operably coupled to the persistent storage disk array and the LSF storage disk and an application program for execution on the microprocessor, and the application program including instructions, criteria and code segments for;

controlling writing to the persistent storage disk array and the LSF storage disk of the first data storage device,

controlling communication between a source of the I/O request and the first data storage device, and

controlling transmission of data from the first data storage device to the second data storage device; and

wherein the second data storage device further includes an LSF storage disk, a microprocessor that is operably coupled to the persistent storage disk array and the LSF storage disk and an application program for execution on the microprocessor, and the application program including instructions, criteria and code segments for;

controlling writing to the LSF storage disk of the second data storage device, and

controlling de-staging of data stored in the LSF storage disk to the persistent storage disk array.

15. The data storage system of claim 1, wherein the first data storage device is operably coupled via a network to one or more computers/workstations and wherein the first data storage device is configured and arranged so as to be responsive to received I/O requests transmitted via the network from the one or more computer/workstations.

16. The data storage system of claim 1, wherein:

the first data storage device is operably coupled via a network to one or more computers/workstations and wherein the first data storage device is configured and arranged so as to be responsive to received I/O requests transmitted via the network from the one or more computer/workstations; and

wherein the first data storage device is operably coupled to the second data storage device via a network, wherein data being transmitted from the first data storage device to the second data storage device is sent via the network.

17. The data storage system of claim 1, wherein the persistent storage disk array of the first data storage device includes a cache and a persistent storage device being operably coupled to the cache.

18. The data storage device of claim 1, wherein the first data storage device further includes a microprocessor that is operably coupled to the persistent storage disk array and the LSF storage disk and an application program for execution on the microprocessor, and an application program in the memory including instructions, criteria and code segments for:

controlling writing to the persistent storage disk array and the LSF storage disk of the first data storage device, and

controlling communication between a source of the I/O request and the first data storage device.

19. The data storage device of claim 18,

wherein the LSF storage disk of the first data storage device includes a cache and a sequentially accessed persistent storage (SAPS) device being operably coupled to the cache, said SAPS device being characterized in that data is stored in a sequential data layout;

and

wherein the application program further includes instructions, criteria and code segments for;

evaluating content of a cache of the LSF storage disk to determine if a condition is satisfied, and

when it is determined that the condition is satisfied, causing the logs in the cache to be sequentially written to the SAPS device.

20. The data storage device of claim 19, wherein the second data storage device includes a persistent storage and wherein the second data storage device is configured and arranged so as to process and to write the transmitted data to the persistent storage disk array of the second data storage device.

21. The data storage device of claim 18,

wherein the LSF storage disk of the first data storage device includes a cache and a sequentially accessed persistent storage (SAPS) device being operably coupled to the cache, said SAPS device being characterized in that data is stored in a sequential data layout;

and

wherein the application program further includes instructions, criteria and code segments for;

evaluating content of a cache of the LSF storage disk to determine if a condition is satisfied,

when it is determined that the condition is satisfied, causing the data logs in the cache to be sequentially written to the SAPS device, and

when it is determined that the condition is not satisfied, causing data logs to be continued to be acquired in the cache.

22. The data storage device of claim 18,

wherein the LSF storage disk of the first data storage device includes a cache and a sequentially accessed persistent storage (SAPS) device being operably coupled to the cache, said SAPS device being characterized in that data is stored in a sequential data layout;

and

wherein the application program further includes instructions, criteria and code segments for;

transmitting data stored in the SAPS device to a second data storage device.

23. The data storage device of claim 22, wherein the second data storage device includes a persistent storage is configured and arranged so as to process and to write the transmitted data to the persistent storage of the second data storage device.

24. The data storage device of claim 18,

wherein the LSF storage disk of the first data storage device includes a cache and a sequentially accessed persistent storage (SAPS) device being operably coupled to the cache, said SAPS device being characterized in that data is stored in a sequential data layout;

and

wherein the application program further includes instructions, criteria and code segments for;

determining if a condition is satisfied for causing data in the SAPS device to be written to a second data storage device, and

when it is determined that the condition is satisfied, causing the data in the SAPS device to be transmitted to the second data storage device.

25. The data storage system of claim 24, wherein the LSF storage disk of the first data storage device includes a cache and a sequentially accessed persistent storage (SAPS) device that is operably coupled thereto, said SAPS device being characterized in that data is stored in a sequential data layout, and wherein the application program including instructions, criteria and code segments for:

creating data logs for data being received and written to the LSF storage disk, and

buffering the created data logs in a cache in the LSF storage disk.

26. The data storage device of claim 24,

wherein the LSF storage disk of the first data storage device includes a cache and a sequentially accessed persistent storage (SAPS) device being operably coupled to the cache, said SAPS device being characterized in that data is stored in a sequential data layout;

and

wherein the application program including instructions, criteria and code segments for;

when it is determined that the condition is satisfied, causing the data in the SAPS device to be transmitted to the second data storage device.

27. The data storage device of claim 26, wherein the application program for the second data storage device includes instructions, criteria and code segments for:

evaluating the second data storage device to determine if a condition is satisfied, and

when it is determined that the condition is satisfied, reading the data logs from the SAPS device of the second data storage device, processing the data contained in the data logs and writing the processed data to the persistent storage disk array of the second data storage device.

28. The data storage system of claim 24, wherein the LSF storage disk of the second data storage device includes a cache and a sequentially accessed persistent storage (SAPS) device that is operably coupled thereto, said SAPS device being characterized in that data is stored in a sequential data layout, and wherein the application program including instructions, criteria and code segments for:

processing the transmitted data so as to re-create the logs that were written to the LSF storage disk of the first data storage device, and

buffering the re-created data logs in the cache of the second data storage device.

29. The data storage device of claim 28, wherein said processing of the data includes reformatting the data so as to be in the appropriate format for writing to the persistent storage disk array of the second storage device and ordering the data so as to be in the proper order.