RESOLVING A PERFORMANCE ISSUE WITHIN A DISPERSED STORAGE NETWORK

• US 20130282865A1
• Filed: 06/18/2013
• Published: 10/24/2013
• Est. Priority Date: 04/26/2010
• Status: Active Grant

• Current Assignee: Pure Storage, Inc.
• Original Assignee: Cleversafe Incorporated (International Business Machines Corporation)
• Inventors: Baptist, Andrew, Leggette, Wesley, Resch, Jason K.
• Granted Patent: US 10,447,767 B2
• Time in Patent Office: Days
• US Class Current: 709/217
• CPC Class Codes: H04L 1/24 Testing correct operation
• CPC Class Codes: H04L 63/08 for authentication of entities cryptographic mechanisms or cryptographic arrangements for entity authentication H04L9/32
• CPC Class Codes: H04L 63/123 received data contents, e.g. message integrity
• CPC Class Codes: H04L 67/10 in which an application is distributed across nodes in the network software deployment G06F8/60; multiprogramming arrangements G06F9/46
• CPC Class Codes: H04L 67/51 Discovery or management thereof, e.g. service location protocol [SLP] or web services

Abstract

A method begins by a processing module identifying one or more devices of a dispersed storage network (DSN) potentially contributing to a DSN performance issue. For a device of the identified one or more devices, the method continues where the processing module determines a potential performance issue of the device and determines a performance test based on the potential performance issue. The method continues where the processing module issues a message to the device that includes test information specific for the device to execute the performance test and receives a response message that includes a specific test result data generated based on the test information. The method continues where the processing module determines, based on the specific test result data, whether the device has the potential performance issue and is contributing to the DSN performance issue.

Claims

1. A method for execution by one or more processing modules of one or more requesting devices of a dispersed storage network (DSN), the method comprises:

when a DSN performance issue is detected, identifying one or more devices of the DSN potentially contributing to the DSN performance issue; and

for a device of the identified one or more devices;

determining a potential performance issue of the device based on how the device is potentially contributing to the DSN performance issue;

determining a performance test based on the potential performance issue;

generating a message that includes a protocol header and a payload, wherein the protocol header includes an indication to identify the message as a test request message and the payload includes test information specific for the device to execute the performance test;

sending the message to the device;

receiving, from the device, a response message that includes a response header and a response payload, wherein the response header includes an indication to identify the response message as a test response message and the response payload includes a specific test result data generated based on the test information; and

determining, based on the specific test result data, whether the device has the potential performance issue and is contributing to the DSN performance issue.

2. The method of claim 1 further comprises:

for a second device of the identified one or more devices;

determining a second potential performance issue of the second device based on how the second device is potentially contributing to the DSN performance issue;

determining a second performance test based on the second potential performance issue;

generating a second message that includes a second protocol header and a second payload, wherein the second protocol header includes an indication to identify the second message as another test request message and the second payload includes second test information specific for the second device to execute the second performance test;

sending the second message to the second device;

receiving, from the second device, a second response message that includes a second response header and a second response payload, wherein the second response header includes an indication to identify the second response message as another test response message and the second response payload includes a second specific test result data generated based on the second test information; and

determining, based on the second specific test result data, whether the second device has the second potential performance issue and is contributing to the DSN performance issue.

3. The method of claim 2 further comprises:

when the potential performance issue and the second potential performance issue are not individually contributing to the DSN performance issue, determining whether a combination of the potential performance issue and the second potential performance issues are collectively contributing to the DSN performance issue.

4. The method of claim 1 further comprises:

detecting the DSN performance issue based on one or more of;

detecting a communication error to, from, or between the one or more devices;

detecting a storage error indication regarding the one or more devices;

detecting a reading error indication regarding the one or more devices;

in response to performing a general test in accordance with a DSN test schedule;

detecting a reliability issue regarding the one or more devices; and

receiving a potential failing performance indicator regarding the one or more devices.

5. The method of claim 1, wherein the identifying the one or more devices comprises one or more of:

when the DSN performance issue is based on a communication error, identifying the one or more devices based on a failure to send a communication or to acknowledge receipt of the communication;

when the DSN performance issue is based on a storage error, identifying the one or more devices based on a failure to provide confirmation of successfully completing a storage request;

when the DSN performance issue is based on a read error, identifying the one or more devices based on a failure to provide retrieved data in accordance with a read request;

receiving a message identifying the one or more devices as having a potential reliability issue; and

receiving a message identifying the one or more devices as having the potential performance issue.

6. The method of claim 1, wherein the device comprises one of:

a dispersed storage (DS) unit;

a DS processing unit;

a DS processing module;

a DS managing module;

a DS rebuilding module; and

a user device.

7. The method of claim 1, wherein the generating the message further comprises:

generating the protocol header to include;

an operation code field to identify the message as a test message;

a request/response field to identify the message as a request message;

a request number field to identify the message as a particular test request message; and

a payload length field.

8. The method of claim 1, wherein the generating the message further comprises at least one of:

generating the payload to test data transfer speed, wherein the payload includes;

a response length field to indicate a desired length of the response message or portion thereof; and

the test information specific for the device that includes an indication of the data transfer speed test and minimal data for transferring between the device and the one or more requesting devices to test speed;

generating the payload to test data transfer bandwidth, wherein the payload includes;

the response length field to indicate the desired length of the response message or the portion thereof; and

the test information specific for the device that includes an indication of the data transfer bandwidth test and a large amount of data for transferring between the device and the one or more requesting devices to test bandwidth;

generating the payload to test data transfer latency, wherein the payload includes;

the response length field to indicate the desired length of the response message or the portion thereof; and

the test information specific for the device that includes an indication of the data transfer latency test and data for transferring between the device and the one or more requesting devices to test latency; and

generating the payload to test error rate of data transfer, wherein the payload includes;

the response length field to indicate the desired length of the response message or the portion thereof; and

the test information specific for the device that includes an indication of the data transfer error rate test and control data for transferring between the device and the one or more requesting devices to test error rate.

9. A computer of a dispersed storage network (DSN) comprises:

an interface;

a memory; and

a processing module operably coupled to the interface and to the memory, where the processing module is operable to;

when a DSN performance issue is detected, identify one or more devices of the DSN potentially contributing to the DSN performance issue; and

for a device of the identified one or more devices;

determine a potential performance issue of the device based on how the device is potentially contributing to the DSN performance issue;

determine a performance test based on the potential performance issue;

generate a message that includes a protocol header and a payload, wherein the protocol header includes an indication to identify the message as a test request message and the payload includes test information specific for the device to execute the performance test;

send, via the interface, the message to the device;

receive, from the device via the interface, a response message that includes a response header and a response payload, wherein the response header includes an indication to identify the response message as a test response message and the response payload includes a specific test result data generated based on the test information; and

determine, based on the specific test result data, whether the device has the potential performance issue and is contributing to the DSN performance issue.

10. The computer of claim 9 further comprises:

the processing module is further operable to, for a second device of the identified one or more devices;

determine a second potential performance issue of the second device based on how the second device is potentially contributing to the DSN performance issue;

determine a second performance test based on the second potential performance issue;

generate a second message that includes a second protocol header and a second payload, wherein the second protocol header includes an indication to identify the second message as another test request message and the second payload includes second test information specific for the second device to execute the second performance test;

send, via the interface, the second message to the second device;

receive, from the second device via the interface, a second response message that includes a second response header and a second response payload, wherein the second response header includes an indication to identify the second response message as another test response message and the second response payload includes a second specific test result data generated based on the second test information; and

determine, based on the second specific test result data, whether the second device has the second potential performance issue and is contributing to the DSN performance issue.

11. The computer of claim 10 further comprises:

the processing module is further operable to, when the potential performance issue and the second potential performance issue are not individually contributing to the DSN performance issue, determine whether a combination of the potential performance issue and the second potential performance issue are collectively contributing to the DSN performance issue.

12. The computer of claim 9 further comprises:

the processing module is further operable to detect the DSN performance issue based on one or more of;

detecting a communication error to, from, or between the one or more devices;

detecting a storage error indication regarding the one or more devices;

detecting a reading error indication regarding the one or more devices;

in response to performing a general test in accordance with a DSN test schedule;

detecting a reliability issue regarding the one or more devices; and

receiving, via the interface, a potential failing performance indicator regarding the one or more devices.

13. The computer of claim 9, wherein the processing module identifies the one or more devices by one or more of:

when the DSN performance issue is based on a communication error, identifying the one or more devices based on a failure to send a communication or to acknowledge receipt of the communication;

when the DSN performance issue is based on a storage error, identifying the one or more devices based on a failure to provide confirmation of successfully completing a storage request;

when the DSN performance issue is based on a read error, identifying the one or more devices based on a failure to provide retrieved data in accordance with a read request;

receiving, via the interface, a message identifying the one or more devices as having a potential reliability issue; and

receiving, via the interface, a message identifying the one or more devices as having the potential performance issue.

14. The computer of claim 9, wherein the device comprises one of:

a dispersed storage (DS) unit;

a DS processing unit;

a DS processing module;

a DS managing module;

a DS rebuilding module; and

a user device.

15. The computer of claim 9, wherein the processing module further functions to generate the message by:

generating the protocol header to include;

an operation code field to identify the message as a test message;

a request/response field to identify the message as a request message;

a request number field to identify the message as a particular test request message; and

a payload length field.

16. The computer of claim 9, wherein the processing module further functions to generate the message by at least one of:

generating the payload to test data transfer speed, wherein the payload includes;

a response length field to indicate a desired length of the response message or portion thereof; and

the test information specific for the device that includes an indication of the data transfer speed test and minimal data for transferring between the device and one or more requesting devices to test speed;

generating the payload to test data transfer bandwidth, wherein the payload includes;

the response length field to indicate the desired length of the response message or the portion thereof; and

the test information specific for the device that includes an indication of the data transfer bandwidth test and a large amount of data for transferring between the device and the one or more requesting devices to test bandwidth;

generating the payload to test data transfer latency, wherein the payload includes;

the response length field to indicate the desired length of the response message or the portion thereof; and

the test information specific for the device that includes an indication of the data transfer latency test and data for transferring between the device and the one or more requesting devices to test latency; and

generating the payload to test error rate of data transfer, wherein the payload includes;

the response length field to indicate the desired length of the response message or the portion thereof; and

the test information specific for the device that includes an indication of the data transfer error rate test and control data for transferring between the device and the one or more requesting devices to test error rate.