Method and system for implementing a distributed operations log

• US 9,336,132 B1
• Filed: 02/06/2012
• Issued: 05/10/2016
• Est. Priority Date: 02/06/2012
• Status: Active Grant

• Current Assignee: Nutanix Incorporated
• Original Assignee: Nutanix Incorporated
• Inventors: Aron, Mohit, Byrne, Brian
• Primary Examiner(s): Ruiz, Aracelis
• Application Number:
  US13/367,073
• Time in Patent Office: 1,555 Days
• US Class Current: 1/1
• CPC Class Codes: G06F 11/2056 by mirroring
• CPC Class Codes: G06F 11/2058 using more than 2 mirrored copies
• CPC Class Codes: G06F 11/2074 Asynchronous techniques
• CPC Class Codes: G06F 11/2076 Synchronous techniques
• CPC Class Codes: G06F 12/0238 Memory management in non-volatile memory, e.g. resistive RAM or ferroelectric memory
• CPC Class Codes: G06F 12/0246 in block erasable memory, e.g. flash memory
• CPC Class Codes: G06F 12/08 in hierarchically structured memory systems, e.g. virtual memory systems
• CPC Class Codes: G06F 12/0804 with main memory updating G06F12/0806 takes precedence
• CPC Class Codes: G06F 12/0868 Data transfer between cache memory and other subsystems, e.g. storage devices or host systems
• CPC Class Codes: G06F 2201/855 Details of asynchronous mirroring using a journal to transfer not-yet-mirrored changes
• CPC Class Codes: G06F 2212/1016 Performance improvement
• CPC Class Codes: G06F 2212/222 Non-volatile memory
• CPC Class Codes: G06F 2212/286 Mirrored cache memory
• CPC Class Codes: G06F 3/06 Digital input from, or digital output to, record carriers , e.g. RAID, emulated record carriers or networked record carriers
• CPC Class Codes: G06F 3/0619 in relation to data integrity, e.g. data losses, bit errors
• CPC Class Codes: G06F 3/065 Replication mechanisms
• CPC Class Codes: G06F 3/067 Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]

Abstract

Disclosed is an improved approach for implementing a storage system having an operation log where data writes are initially performed to a fast operation log, rather than to its permanent storage location. A distributed operation log can be employed to replicate the data to multiple locations in the storage system.

Claims

1. A method for managing storage devices, comprising:

receiving, at a first node, a write request to write data to a relatively slow back-end storage tier in a storage system, the storage system further comprising a relatively fast storage tier comprising SSDs;

initially writing the write request to write data to the relatively slow back-end storage tier as temporary data to a first operation log on the first node at the relatively fast storage tier, the temporary data being distributed across multiple nodes by replicating the first operation log on the first node to one or more other operation logs on other nodes based on a replication factor;

sending a confirmation of the write request to an originator of the write request at a point in time where data to be committed in the write request has not been committed to the relatively slow back-end storage tier; and

transferring contents of the operation log to the relatively slow back-end storage tier.

2. The method of claim 1, in which the data is written to the relatively slow back-end storage tier after the confirmation of the write request is sent to the originator of the write request.

3. The method of claim 2, in which the data is written to the relatively slow back-end storage tier over time.

4. The method of claim 2, in which the data in the operation log is deleted after the data has been written to the relatively slow back-end storage tier.

5. The method of claim 1, in which the data in the operation log are created as a log having a contiguous time range of the data.

6. The method of claim 5, in which a succeeding write to the log is appended to existing data in the log.

7. The method of claim 1, in which an in-memory index is maintained to track about all outstanding record writes.

8. The method of claim 1, in which the first operation log comprises a primary operation log on a first node and the one or more other operation logs comprise a slave operation log on a second node, the slave operation log being a replicated form of the primary operation log so that the primary operation log and the slave operation log contain the same write requests.

9. The method of claim 8, in which the primary operation log is responsible for committing writes to persistent storage and the slave operation log comprises a replication of the data on the primary log, and wherein the slave operation log comprising the replicated temporary data is configured to replace the primary operation log on the first node upon failure of the primary operation log at the first node.

10. The method of claim 9, wherein replacing the primary operation log comprises using the slave operation log to commit the write request.

11. The method of claim 8, in which the primary operation log on the first node waits for confirmation from the slave operation log before confirming to the originator of the write request.

12. The method of claim 8, in which the primary operation log on the first node does not wait for confirmation from the slave operation log before confirming to the originator of the write request.

13. The method of claim 8, in which the data is striped across multiple slave operation logs.

14. The method of claim 1, in which the storage system comprises a virtualization storage system.

15. A system for managing storage devices, comprising:

a processor;

a memory for holding programmable code; and

wherein the programmable code includes instructions for receiving, at a first node, a write request to write data to a relatively slow back-end storage tier in a storage system, the storage system further comprising a relatively fast storage tier comprising SSDs;

initially writing the write request to write data to the relatively slow back-end storage tier as temporary data to a first operation log on the first node at the relatively fast storage tier, the temporary data being distributed across multiple nodes by replicating the first operation log on the first node to one or more other operation logs on other nodes based on a replication factor;

sending a confirmation of the write request to an originator of the write request at a point in time where data to be committed in the write request has not been committed to the relatively slow back-end storage tier; and

transferring contents of the operation log to the relatively slow back-end storage tier.

16. The system of claim 15, in which the data is written to the relatively slow back-end storage tier after the confirmation of the write request is sent to the originator of the write request.

17. The system of claim 16, in which the data is written to the relatively slow back-end storage tier over time.

18. The system of claim 16, in which the data in the operation log is deleted after the data has been written to the relatively slow back-end storage tier.

19. The system of claim 15, in which the data in the operation log are created as a log having a contiguous time range of the data.

20. The system of claim 19, in which a succeeding write to the log is appended to existing data in the log.

21. The system of claim 15, in which an in-memory index is maintained to track about all outstanding record writes.

22. The system of claim 15, in which the first operation log comprises a primary operation log on a first node and the one or more other operation logs comprise a slave operation log on a second node, the slave operation log being a replicated form of the primary operation log so that the primary operation log and the slave operation log contain the same write requests.

23. The system of claim 22, in which the primary operation log is responsible for committing writes to persistent storage and the slave operation log comprises a replication of the data on the primary log, and wherein the slave operation log comprising the replicated temporary data is configured to replace the primary operation log on the first node upon failure of the primary operation log at the first node.

24. The system of claim 23, wherein replacing the primary operation log comprises using the slave operation log to commit the write request.

25. The system of claim 22, in which the primary operation log on the first node waits for confirmation from the slave operation log before confirming to the originator of the write request.

26. The system of claim 22, in which the primary operation log on the first node does not wait for confirmation from the slave operation log before confirming to the originator of the write request.

27. The system of claim 22, in which the data is striped across multiple slave operation logs.

28. The system of claim 15, in which the storage system comprises a virtualization storage system.

29. A computer program product embodied on a non-transitory computer readable medium, the computer readable medium having stored thereon a sequence of instructions which, when executed by a processor causes the processor to execute a method for managing storage devices, the method comprising:

receiving, at a first node, a write request to write data to a relatively slow back-end storage tier in a storage system, the storage system further comprising a relatively fast storage tier comprising SSDs;

initially writing the write request to write data to the relatively slow back-end storage tier as temporary data to a first operation log on the first node at the relatively fast storage tier, the temporary data being distributed across multiple nodes by replicating the first operation log on the first node to one or more other operation logs on other nodes based on a replication factor;

sending a confirmation of the write request to an originator of the write request at a point in time where data to be committed in the write request has not been committed to the relatively slow back-end storage tier; and

transferring contents of the operation log to the relatively slow back-end storage tier.

30. The computer program product of claim 29, in which the data is written to the relatively slow back-end storage tier after the confirmation of the write request is sent to the originator of the write request.

31. The computer program product of claim 30, in which the data is written to the relatively slow back-end storage tier over time.

32. The computer program product of claim 30, in which the data in the operation log is deleted after the data has been written to the relatively slow back-end storage tier.

33. The computer program product of claim 29, in which the data in the operation log are created as a log having a contiguous time range of the data.

34. The computer program product of claim 33, in which a succeeding write to the log is appended to existing data in the log.

35. The computer program product of claim 29, in which an in-memory index is maintained to track about all outstanding record writes.

36. The computer program product of claim 29, in which the first operation log comprises a primary operation log on a first node and the one or more other operation logs comprise a slave operation log on a second node, the slave operation log being a replicated form of the primary operation log so that the primary operation log and the slave operation log contain the same write requests.

37. The computer program product of claim 36, in which the primary operation log is responsible for committing writes to persistent storage and the slave operation log comprises a replication of the data on the primary log, and wherein the slave operation log comprising the replicated temporary data is configured to replace the primary operation log on the first node upon failure of the primary operation log at the first node.

38. The computer program product of claim 37, wherein replacing the primary operation log comprises using the slave operation log to commit the write request.

39. The computer program product of claim 36, in which the primary operation log on the first node waits for confirmation from the slave operation log before confirming to the originator of the write request.

40. The computer program product of claim 36, in which the primary operation log on the first node does not wait for confirmation from the slave operation log before confirming to the originator of the write request.

41. The computer program product of claim 36, in which the data is striped across multiple slave operation logs.

42. The computer program product of claim 29, in which the storage system comprises a virtualization storage system.