Techniques for error handling in parallel splitting of storage commands

• US 9,489,275 B2
• Filed: 10/02/2014
• Issued: 11/08/2016
• Est. Priority Date: 10/02/2014
• Status: Active Grant

• Current Assignee: NetApp, Inc.
• Original Assignee: NetApp, Inc.
• Inventors: Mu, Paul Yuedong, Ngan, Paul, Sundararajan, Manoj
• Primary Examiner(s): LE, DIEU MINH T
• Application Number:
  US14/504,618
• Publication Number:
  US 20160098330A1
• Time in Patent Office: 768 Days
• Field of Search: 714/6.1, 714/6.11, 714/6.2, 714/6.12, 714/6.13, 714/6.3
• US Class Current: 1/1
• CPC Class Codes: G06F 11/07 Responding to the occurrence of a fault, e.g. fault tolerance
• CPC Class Codes: G06F 11/2069 Management of state, configuration or failover
• CPC Class Codes: G06F 11/2094 Redundant storage or storage space G06F11/2056 takes precedence
• CPC Class Codes: G06F 2201/805 Real-time
• CPC Class Codes: G06F 2201/82 Solving problems relating to consistency ensuring consistency in mirrored systems G06F11/2064
• CPC Class Codes: G06F 2201/85 Active fault masking without idle spares active fault masking without idle spare hardware where processing functionality is redundant G06F11/2035
• CPC Class Codes: G06F 3/0619 in relation to data integrity, e.g. data losses, bit errors
• CPC Class Codes: G06F 3/065 Replication mechanisms
• CPC Class Codes: G06F 3/067 Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
• CPC Class Codes: G06F 3/0689 Disk arrays, e.g. RAID, JBOD
• CPC Class Codes: H04L 1/08 by repeating transmission, e.g. Verdan system H04L1/1858 and H04L1/189 take precedence

Abstract

Various embodiments are generally directed to techniques for handling errors affecting the at least partially parallel performance of data access commands between nodes of a storage cluster system. An apparatus may include a processor component of a first node, an access component to perform a command received from a client device via a network to alter client device data stored in a first storage device coupled to the first node, a replication component to transmit a replica of the command to a second node via the network to enable performance of the replica by the second node at least partially in parallel, an error component to retry transmission of the replica based on a failure indicated by the second node and a status component to select a status indication to transmit to the client device based on the indication of failure and results of retrial of transmission of the replica.

Claims

1. A method comprising:

performing at a first node of a storage cluster system a command received from a client device via a network to alter client device data stored in a first storage device coupled to the first node;

transmitting a replica of the command to a second node of the storage cluster system via the network to enable performance of the replica by the second node at least partially in parallel with the command;

retrying transmission of the replica based on a failure indication by the second node; and

selecting a status indication of performance of the command to transmit to the client device based on the failure indication and results of retrial of transmission of the replica, wherein an indication of successful performance of the command is selected as the status indication based upon a determination that a failure is a short-term failure and successful commencement of performance of the replica by the second node following retrial of transmission of the replica to the second node.

2. The method of claim 1, comprising:

responsive to determining that the failure is the short-term failure that the second node is able to correct, retrying transmission of the replica to the second node.

3. The method of claim 1, comprising:

selecting the indication of successful performance of the command as the status indication based on the determination that the failure is the short-term failure and based on successful acceptance to retry performance of the replica by the second node following retrial of transmission of the replica to the second node.

4. The method of claim 1, comprising:

determining whether a predetermined status time period has elapsed since receipt of the command; and

selecting an indication of retrial of the command as currently underway as the status indication based on the determination that the failure is the short-term failure and on an elapsing of the status time period as retrial of the transmission of the replica to the second node is underway.

5. The method of claim 1, comprising:

responsive to determining that the failure is a long-term failure that the second node is unable to correct based on a limit of multiple retrials of the replica with the second node being met, retrying transmission of the replica to a third node comprising a partner node of the second node in a high availability group that comprises the second node and the third node.

6. The method of claim 1, comprising:

responsive to determining that the failure is a long-term failure that the second node is unable to correct, retrying transmission of the replica to a third node comprising a partner node of the second node in a high availability group that comprises the second node and the third node.

7. The method of claim 6, comprising:

selecting the indication of successful performance of the command as the status indication based on the determination that the failure is the long-term failure and based on at least one of successful acceptance, successful commencement of performance, and successful performance of the replica by the third node following the retrial of transmission of the replica to the third node.

8. The method of claim 6, comprising:

determining whether a predetermined status time period has elapsed since receipt of the command; and

selecting an indication of retrial of the command as currently underway as the status indication based on the determination that the failure is the long-term failure and on an elapsing of the status time period as retrial of the transmission of the replica to the third node is underway.

9. The method of claim 1, comprising:

transmitting the status indication to the client device at least partially in parallel with performing of the command at the first node.

10. A non-transitory machine-readable storage medium having stored thereon instructions for performing a method comprising computing code which when executed by at least one machine, causes the machine to:

perform at a first node of a storage cluster system a command received from a client device via a network to alter client device data stored in a first storage device coupled to the first node;

transmit a replica of the command to a second node of the storage cluster system via the network to enable performance of the replica by the second node at least partially in parallel with the command;

retry transmission of the replica based on a failure indication by the second node; and

select a status indication of performance of the command to transmit to the client device based on the failure indication and results of retrial of transmission of the replica, wherein an indication of successful performance of the command is selected as the status indication based upon a determination that a failure is a short-term failure and successful acceptance to retry performance of the replica by the second node following retrial of transmission of the replica to the second node.

11. The non-transitory machine-readable storage medium of claim 10, wherein the computing code causes the machine to:

responsive to determining that the failure is the short-term failure that the second node is able to correct, retry transmission of the replica to the second node.

12. The non-transitory machine-readable storage medium of claim 10, wherein the computing code causes the machine to:

select the indication of successful performance of the command as the status indication based on the determination that the failure is the short-term failure and based on successful commencement of performance of the replica by the second node following retrial of transmission of the replica to the second node.

13. The non-transitory machine-readable storage medium of claim 10, wherein the computing code causes the machine to:

determine whether a predetermined status time period has elapsed since receipt of the command; and

select an indication of retrial of the command as currently underway as the status indication based on the determination that the failure is the short-term failure and on an elapsing of the status time period as retrial of the transmission of the replica to the second node is underway.

14. The non-transitory machine-readable storage medium of claim 10, wherein the computing code causes the machine to:

responsive to determining that the failure is a long-term failure that the second node is unable to correct based on a limit of multiple retrials of the replica with the second node being met, retry transmission of the replica to a third node comprising a partner node of the second node in a high availability group that comprises the second node and the third node.

15. The non-transitory machine-readable storage medium of claim 10, wherein the computing code causes the machine to:

responsive to determining that the failure is a long-term failure that the second node is unable to correct, retry transmission of the replica to a third node comprising a partner node of the second node in a high availability group that comprises the second node and the third node.

16. The non-transitory machine-readable storage medium of claim 15, wherein the computing code causes the machine to:

select the indication of successful performance of the command as the status indication based on the determination that the failure is the long-term failure and based on at least one of successful acceptance, successful commencement of performance, and successful performance of the replica by the third node following the retrial of transmission of the replica to the third node.

17. The non-transitory machine-readable storage medium of claim 15, wherein the computing code causes the machine to:

determine whether a predetermined status time period has elapsed since receipt of the command; and

select an indication of retrial of the command as currently underway as the status indication based on the determination that the failure is the long-term failure and on an elapsing of the status time period as retrial of the transmission of the replica to the third node is underway.

18. The non-transitory machine-readable storage medium of claim 10, wherein the computing code causes the machine to:

transmit the status indication to the client device at least partially in parallel with performing of the command at the first node.

19. The non-transitory machine-readable storage medium of claim 10, wherein the computing code causes the machine to:

condition transmission of the replica to the second node on at least one of successful acceptance and successful commencement of performance of the command.

20. A computing device comprising:

a memory containing machine-readable storage medium comprising computing code having stored thereon instructions for performing a method; and

a processor component coupled to the memory, the processor component configured to execute the computing code to cause the processor component to;

perform a command, received by a first node of a storage cluster system from a client device via a network, to alter client device data stored in a first storage device coupled to the first node;

transmit a replica of the command to a second node of the storage cluster system via the network to enable performance of the replica by the second node at least partially in parallel with the command;

retry transmission of the replica based on a failure indication by the second node; and

select a status indication of performance of the command to transmit to the client device based on the failure indication and results of retrial of transmission of the replica, wherein an indication of successful performance of the command is selected as the status indication based upon a determination that a failure is a short-term failure and successful acceptance to retry performance of the replica by the second node following retrial of transmission of the replica to the second node.

21. The computing device of claim 20, wherein the computing code causes the processor component to:

responsive to determining that the failure is the short-term failure that the second node is able to correct, retry transmission of the replica to the second node.

22. The computing device of claim 20, wherein the computing code causes the processor component to:

select the indication of successful performance of the command as the status indication based on the determination that the failure is the short-term failure and based on successful commencement of performance of the replica by the second node following retrial of transmission of the replica to the second node.

23. The computing device of claim 20, wherein the computing code causes the processor component to:

determine whether a predetermined status time period has elapsed since receipt of the command; and

select an indication of retrial of the command as currently underway as the status indication based on the determination that the failure is the short-term failure and on an elapsing of the status time period as retrial of the transmission of the replica to the second node is underway.

24. The computing device of claim 20, wherein the computing code causes the processor component to:

responsive to determining that the failure is a long term failure that the second node is not able to correct based on a limit of multiple retrials of the replica with the second node being met, retry the replica with a third node comprising a partner node of the second node in a high availability group that comprises the second node and the third node.

25. The computing device of claim 20, wherein the computing code causes the processor component to:

responsive to determining that the failure is a long-term failure that the second node is unable to correct, retry transmission of the replica to a third node comprising a partner node of the second node in a high availability group that comprises the second node and the third node.

26. The computing device of claim 25, wherein the computing code causes the processor component to:

select the indication of successful performance of the command as the status indication based on the determination that the failure is the long-term failure and based on at least one of successful acceptance, successful commencement of performance, and successful performance of the replica by the third node following the retrial of transmission of the replica to the third node.

27. The computing device of claim 25, wherein the computing code causes the processor component to:

determine whether a predetermined status time period has elapsed since receipt of the command; and

select an indication of retrial of the command as currently underway as the status indication based on the determination that the failure is the long-term failure and on an elapsing of the status time period as retrial of the transmission of the replica to the third node is underway.

28. The computing device of claim 25, wherein the computing code causes the processor component to:

responsive to detecting an absence of an active communications session coupling the first node and the second node through the network, determine that the failure is the long term failure.

29. The computing device of claim 20, wherein the computing code causes the processor component to:

translate the command from a storage service request as received from the client device to a data access command; and

translate the status indication from a command response associated with the data access command to a service request response associated with the storage service request as transmitted to the client device.