PULSED LEADER CONSENSUS MANAGEMENT

• US 20200034048A1
• Filed: 01/30/2015
• Published: 01/30/2020
• Est. Priority Date: 01/30/2015
• Status: Active Grant

• Current Assignee: Nutanix Incorporated
• Original Assignee: Nutanix Incorporated
• Inventors: GUPTA, Karan, BHARDWAJ, Rishi, KAMAT, Chinmay Dinesh, KONKA, Pavan Kumar
• Granted Patent: US 10,642,507 B2
• Time in Patent Office: Days
• CPC Class Codes: G06F 16/11 File system administration, e.g. details of archiving or snapshots file system backup G06F11/14
• CPC Class Codes: G06F 16/27 Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
• CPC Class Codes: G06F 3/061 Improving I/O performance
• CPC Class Codes: G06F 3/0619 in relation to data integrity, e.g. data losses, bit errors
• CPC Class Codes: G06F 3/065 Replication mechanisms
• CPC Class Codes: G06F 3/0658 Controller construction arrangements
• CPC Class Codes: G06F 3/067 Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]

Abstract

A system using pulsed leader consensus management that can manage storage by electing a leader node from a group of nodes and fulfilling read requests for the group of nodes if a sync state is not expired. The sync state can correspond to two conditions: first, that the nodes in the group direct the write requests for the group to the leader node; second, that a health pulse is detected, wherein the health pulse is generated if the leader node is alive (e.g. has not failed, terminated, or become non-responsive). If either of the two conditions fail, the sync state can be considered expired and the group of nodes may fall back to consensus-based read mechanisms, such as Paxos, until a new leader node is elected.

Claims

1. A method for managing storage, the method comprising:

receiving a read request for data managed by a group of two or more nodes at a node of the group of two or more nodes, wherein a leader node is elected from the group of two or more nodes;

determining that a sync state of the leader node is expire, the sync state corresponding to the read request and to the group of two or more nodes, wherein the sync state is expired at least because the leader node has failed or is unresponsive; and

reading from two or more storage devices using a plurality of nodes in response to the read request while the leader node has failed or is unresponsive, the node implementing a consensus scheme before a new leader node is elected to identify data to transmit to one or more users associated with the read request.

2. The method of claim 1, further comprising:

receiving a second read request at a second node of the group of two or more nodes for data managed by the group of two or more nodes;

determining that a second sync state of the new leader node is not expired, the second sync state corresponding to the second read request and to the group of two of more nodes; and

reading from one or more storage devices using only the new leader node in response to the second read request while the second sync state is not expired.

3. The method of claim 1, wherein consensus scheme-reads are Paxos-based read operations.

4. The method of claim 1, wherein the sync state is expired if a health pulse that corresponds to the group of two or more nodes is not detected.

5. The method of claim 4, wherein the sync state is expired if write requests managed by the group of two or more nodes are not routed to the leader node.

6. The method of claim 4, wherein the health pulse corresponds to the leader node such that if the leader node fails the health pulse will not be detected.

7. The method of claim 4, wherein the health pulse is generated on a first time interval.

8. The method of claim 7, wherein the sync state is expired when the health pulse is not detected within a second time interval.

9. The method of claim 1, wherein the group of two or more nodes are informed of the leader node and agree to send all write requests to the leader node.

10. The method of claim 2, further comprising electing a new leader node when the sync state is expired.

11. The method of claim 10, wherein electing a new leader node comprises:

electing a node of the group of two or more nodes as the new leader node based on at least one or more parameters;

entering a transitional state at the new leader node;

scanning the group of two or more nodes to identify data being written to nodes of the group of nodes;

aggregating the data being written at the nodes of the group of two or more nodes at the leader node; and

notifying the nodes of the group of two or more nodes of a status and identity of the new leader node to exit the transitional state.

12. A computer program product embodied on a non-transitory computer readable medium, the computer readable medium having stored thereon a sequence of instructions which, when executed by a processor causes the processor to execute a process for implementing a method for storage management, the process comprising:

receiving a read request for data managed by a group of two or more nodes at a node of the group of two or more nodes, wherein a leader node is elected from the group of two or more nodes;

determining that a sync state of the leader node is expire, the sync state corresponding to the read request and to the group of two or more nodes, wherein the sync state is expired at least because the leader node has failed or is unresponsive; and

reading from two or more storage devices using a plurality of nodes in response to the read request while the leader node has failed or is unresponsive, the node implementing a consensus scheme before a new leader node is elected to identify data to transmit to one or more users associated with the read request.

13. The computer program product of claim 12, the process further comprising:

receiving a second read request at a second node of the group of two or more nodes for data managed by the group of two or more nodes;

determining that a second sync state of the new leader node is not expired, the second sync state corresponding to the second read request and to the group of two of more nodes; and

reading from one or more storage devices using only the new leader node in response to the second read request while the second sync state is not expired.

14. The computer program product of claim 12, wherein the sync state is expired if a health pulse that corresponds to the group of two or more nodes is not detected.

15. The computer program product of claim 14, wherein the sync state is expired if write requests managed by the group of two or more nodes are not routed to the leader node.

16. A system for managing storage with a processor, the processor performing actions when executing a instructions of a set of instructions stored in a memory, the actions comprising:

receiving a read request for data managed by a group of two or more nodes at a node of the group of two or more nodes, wherein a leader node is elected from the group of two or more nodes;

determining that a sync state of the leader node is expire, the sync state corresponding to the read request and to the group of two or more nodes, wherein the sync state is expired at least because the leader node has failed or is unresponsive; and

reading from two or more storage devices using a plurality of nodes in response to the read request while the leader node has failed or is unresponsive, the node implementing a consensus scheme before a new leader node is elected to identify data to transmit to one or more users associated with the read request.

17. The system of claim 16, the actions further comprising:

receiving a second read request at a second node of the group of two or more nodes for data managed by the group of two or more nodes;

determining that a second sync state of the new leader node is not expired, the second sync state corresponding to the second read request and to the group of two of more nodes; and

reading from one or more storage devices using only the new leader node in response to the second read request while the second sync state is not expired.

18. The system of claim 16, wherein the sync state is expired if a health pulse that corresponds to the group of two or more nodes is not detected.

19. The system of claim 18, wherein the sync state is expired if write requests managed by the group of two or more nodes are not routed to the leader node.

20. The system of claim 18, wherein the health pulse corresponds to the leader node such that if the leader node fails the health pulse will not be detected.