Ordering data writes from a local storage device to a remote storage device

• US 6,898,685 B2
• Filed: 03/25/2003
• Issued: 05/24/2005
• Est. Priority Date: 03/25/2003
• Status: Expired due to Term

• Current Assignee: Emc IP Holding Company LLC (Dell Technologies Inc.)
• Original Assignee: EMC Corporation (Dell Technologies Inc.)
• Inventors: Ofer, Adi, Meiri, David, Kopylovitz, Haim, Moreshet, Hana, Yoder, Benjamin W., Halstead, Mark J.
• Primary Examiner(s): ELMORE, STEPHEN C
• Application Number:
  US10/396,786
• Publication Number:
  US 20040193820A1
• Time in Patent Office: 791 Days
• Field of Search: 711/167, 711/162, 711/112, 711/111, 700/82, 707/204, 714/6
• US Class Current: 711/167
• CPC Class Codes: G06F 11/2064 while ensuring consistency
• CPC Class Codes: G06F 11/2074 Asynchronous techniques
• CPC Class Codes: G06F 12/0868 Data transfer between cache memory and other subsystems, e.g. storage devices or host systems
• CPC Class Codes: G06F 12/0871 Allocation or management of cache space
• CPC Class Codes: G06F 2003/0697 device management, e.g. handlers, drivers, I/O schedulers
• CPC Class Codes: G06F 2201/82 Solving problems relating to consistency ensuring consistency in mirrored systems G06F11/2064
• CPC Class Codes: G06F 2212/284 being distributed
• CPC Class Codes: G06F 3/0601 Interfaces specially adapted for storage systems
• CPC Class Codes: G06F 3/061 Improving I/O performance
• CPC Class Codes: G06F 3/0614 Improving the reliability of storage systems
• CPC Class Codes: G06F 3/0635 by changing the path, e.g. traffic rerouting, path reconfiguration
• CPC Class Codes: G06F 3/0656 Data buffering arrangements
• CPC Class Codes: G06F 3/0659 Command handling arrangements, e.g. command buffers, queues, command scheduling
• CPC Class Codes: G06F 3/067 Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
• CPC Class Codes: Y10S 707/99953 Recoverability
• CPC Class Codes: Y10S 707/99955 Archiving or backup

Abstract

Ordering data writes include a host computer providing a plurality of data writes to a primary storage device, the primary storage device assigning a first sequence number to data writes begun after a first time and before a second time, the primary storage device assigning a second sequence number, different from the first sequence number, to data writes begun after the second time, in response to completion of all writes assigned the first sequence number, the primary storage device transferring writes assigned the first sequence number to a secondary storage device, and, in response to the primary storage device transferring all writes assigned the first sequence number to the secondary storage device, the primary storage device sending a commit message to the secondary storage device. Ordering data writes may also include, in response to receiving a commit message from the primary storage device, the secondary storage device storing the data writes having the first sequence number.

Claims

1. A method of ordering data writes, comprising:

a primary storage device receiving a plurality of data writes;

the primary storage device associating data writes begun after a first time and before a second time with a first chunk of data;

the primary storage device associating data writes begun after the second time with a second chunk of data different from the first chunk of data;

after completion of all writes associated with the first chunk of data, the primary storage device initiating transfer of writes associated with the first chunk of data to a secondary storage device;

following the primary storage device transferring all writes associated with the first chunk of data to the secondary storage device, the primary storage device sending a message to the secondary storage device; and

in response to receiving the message from the primary storage device, the secondary storage device storing the data writes associated with the first chunk of data.

2. A method, according to claim 1, further comprising:

after storing all of the data writes associated with the first chunk of data, the secondary storage device sending an acknowledge to the primary storage device.

3. A method, according to claim 2, further comprising:

following sending a message to the secondary storage device, the primary storage device suspending transferring data to the secondary storage device.

4. A method, according to claim 3, further comprising:

following suspending transferring data, the primary storage device associating subsequent data writes with a third chunk of data, different from the first and second chunks of data.

5. A method, according to claim 4, further comprising:

in response to the secondary storage device sending an acknowledge to the primary storage device, the primary storage device resuming transferring data to the secondary storage device.

6. A method of ordering data writes, comprising:

a primary storage device receiving a plurality of data writes;

the primary storage device associating data writes begun after a first time and before a second time with a first chunk of data;

the primary storage device associating data writes begun after the second time with a second chunk of data different from the first chunk of data;

after completion of all writes associated with the first chunk of data, the primary storage device initiating transfer of writes associated with the first chunk of data to a secondary storage device;

providing data writes to cache slots of the primary storage device; and

in response to a data write being associated with the second chunk of data corresponding to a cache slot already associated with the first chunk of data, copying data to a new cache slot.

7. A method of ordering data writes, comprising:

a primary storage device receiving a plurality of data writes;

the primary storage device associating data writes begun after a first time and before a second time with a first chunk of data;

the primary storage device associating data writes begun after the second time with a second chunk of data different from the first chunk of data;

after completion of all writes associated with the first chunk of data, the primary storage device initiating transfer of writes associated with the first chunk of data to a secondary storage device;

providing data writes to cache slots of the primary storage device;

the primary storage device using a first list of pointers to the cache slots for data writes associated with the first chunk of data; and

the primary storage device using a second list of pointers to the cache slots for data writes associated with the second chunk of data.

8. A method, according to claim 7, further comprising:

providing a cache stamp field in a header portion of each of the cache slots, the cache stamp field including a sequence number associated with each of the cache slots, wherein the sequence number corresponds to a particular one of the chunks of data.

9. A method, according to claim 8, wherein the cache stamp field also includes a password field that is written when each of the cache slots is first used.

10. A method, according to claim 9, further comprising:

in response to a slot no longer being used, clearing the password field.

11. A method of ordering data writes, comprising:

a primary storage device receiving a plurality of data writes;

the primary storage device associating data writes begun after a first time and before a second time with a first chunk of data;

the primary storage device associating data writes begun after the second time with a second chunk of data different from the first chunk of data;

after completion of all writes associated with the first chunk of data, the primary storage device initiating transfer of writes associated with the first chunk of data to a secondary storage device;

providing data writes to cache slots of the primary storage device;

providing a first cache only virtual device;

providing a second cache only virtual device;

the primary storage device using the first cache only virtual device for the cache slots corresponding to data writes associated with the first chunk of data; and

the primary storage device using the second cache only virtual device for the cache slots corresponding to data writes associated with the second chunk of data.

12. A method of storing ordered writes to a remote storage device, comprising:

receiving a message for data associated with a first chunk of data;

in response to data associated with a second chunk of data different from said first chunk of data not yet being stored, waiting for the data associated with the second chunk of data to be stored; and

initiating storage of the data associated with the first chunk of data.

13. A method, according to claim 12, further comprising:

acknowledging the message.

14. A method, according to claim 12, further comprising:

constructing a list of modified slots in connection with receiving data associated with the first chunk of data.

15. A method, according to claim 14, wherein initiating storage of data associated with the first chunk of data includes traversing the list of modified slots.

16. Computer executable code, embodied in a computer readable medium, that orders data writes, comprising:

executable code that, when executed, associates data writes begun after a first time and before a second time with a first chunk of data;

executable code that, when executed, associates data writes begun after the second time with a second chunk of data different from the first chunk of data; and

executable code that, when executed, initiates transfer of writes associated with the first chunk of data to a storage device after completion of all writes associated with the first chunk of data;

executable code that, when executed, sends a message to the storage device after transferring all writes associated with the first chunk of data; and

executable code that, when executed, suspends transferring data to the storage device after sending the message thereto.

17. Computer executable code, according to claim 16, further comprising:

executable code that, when executed, associates data writes with a third chunk of data, different from the first and second chunks of data, to subsequent data writes following suspending transferring data.

18. Computer executable code, according to claim 17, further comprising:

executable code that, when executed, resumes transferring data to the storage device in response to the storage device acknowledging handling the message.

19. Computer executable code, according to claim 16, further comprising:

executable code that, when executed, provides data writes to cache slots.

20. Computer executable code, according to claim 19, further comprising:

executable code that, when executed, copies data to a new cache slot in response to a data write being associated with the second chunk of data corresponding to a cache slot already associated with the first chunk of data.

21. Computer executable code, according to claim 19, further comprising:

executable code that, when executed, uses a first list of pointers to the cache slots for data writes-associated with the first chunk of data; and

executable code that, when executed, uses a second list of pointers to the cache slots for data writes associated with the second chunk of data.

22. Computer executable code, according to claim 21, further comprising:

executable code that, when executed, provides a cache stamp field in a header portion of each of the cache slots, the cache stamp field including a sequence number associated with each of the cache slots, wherein the sequence number corresponds to a particular one of the chunks of data.

23. Computer executable code, according to claim 22, wherein the cache stamp field also includes a password field that is written when each of the cache slots is first used.

24. Computer executable code, according to claim 23, further comprising:

executable code that, when executed, clears the password field in response to a slot no longer being used.

25. Computer executable code, according to claim 19, further comprising:

executable code that, when executed, uses a first cache only virtual device for the cache slots corresponding to data writes associated with the first chunk of data; and

executable code that, when executed, uses a second cache only virtual device for the cache slots corresponding to data writes associated with the second chunk of data.

26. Computer executable code, in a computer readable medium, that stores ordered writes to a remote storage device, comprising:

executable code that, when executed, receives a message for data associated with a first chunk of data;

executable code that, when executed, waits for the data associated with a second chunk of data, different from the first chunk of data, to be stored; and

executable code that, when executed, initiates storage of the data associated with the first chunk of data after the second chunk of data has been stored.

27. Computer executable code, according to claim 26, further comprising:

executable code that, when executed, acknowledges the message.

28. Computer executable code, according to claim 26, further comprising:

executable code that, when executed, constructs a list of modified slots in connection with receiving data associated with the first chunk of data.

29. Computer executable code, according to claim 28, wherein executable code that initiates storage of data associated with the first chunk of data includes executable code that traverses the list of modified slots.

30. Computer executable code, according to claim 26, further comprising:

executable code that, when executed, sends a message to the storage device after transferring all writes associated with the first chunk of data.

31. A method of ordering data writes, comprising:

a primary storage device receiving a plurality of data writes;

the primary storage device associating data writes begun after a first time and before a second time with a first chunk of data;

the primary storage device associating data writes begun after the second time with a second chunk of data different from the first chunk of data;

after completion of all writes associated with the first chunk of data, the primary storage device initiating transfer of writes associated with the first chunk of data to a secondary storage device; and

following the primary storage device transferring all writes associated with the first chunk of data to the secondary storage device, the primary storage device sending a message to the secondary storage device.

32. A method, according to claim 31, further comprising:

providing data writes to cache slots of the primary storage device.

33. A method, according to claim 31, wherein receiving a plurality of data writes includes receiving a plurality of data writes from a host.

34. A data storage device, comprising:

at least one disk drive that contains data;

at least one host adaptor, coupled to the at least one disk drive, that receives data to be stored on the at least one disk drive; and

at least one remote adaptor, coupled to the at least one disk drive and the at least one host adaptor, that transmits data to a remote storage device, wherein, in response to receipt of data by the at least one host adaptor, data writes begun after a first time and before a second time are associated with a first chunk of data, data writes begun after the second time are associated with a second chunk of data different from the first chunk of data, after completion of all writes associated with the first chunk of data, a transfer of writes associated with the first chunk of data to the remote storage device is initiated and, following completing transfer of writes associated with the first chunk of data to the remote storage device, sending a message to the remote storage device.