Using reinforcement learning to select a DS processing unit

• US 10,268,545 B2
• Filed: 12/13/2017
• Issued: 04/23/2019
• Est. Priority Date: 09/08/2014
• Status: Active Grant

• Current Assignee: Pure Storage, Inc.
• Original Assignee: International Business Machines Corporation
• Inventors: Khadiwala, Ravi V., Resch, Jason K.
• Primary Examiner(s): Abraham, Esaw T
• Application Number:
  US15/841,070
• Publication Number:
  US 20180121288A1
• Time in Patent Office: 496 Days
• CPC Class Codes: G06F 11/1076 Parity data used in redundant arrays of independent storages, e.g. in RAID systems
• CPC Class Codes: G06F 11/1092 Rebuilding, e.g. when physically replacing a failing disk
• CPC Class Codes: G06F 2211/1028 Distributed, i.e. distributed RAID systems with parity
• CPC Class Codes: G06F 3/061 Improving I/O performance
• CPC Class Codes: G06F 3/0619 in relation to data integrity, e.g. data losses, bit errors
• CPC Class Codes: G06F 3/0635 by changing the path, e.g. traffic rerouting, path reconfiguration
• CPC Class Codes: G06F 3/064 Management of blocks
• CPC Class Codes: G06F 3/0653 Monitoring storage devices or systems
• CPC Class Codes: G06F 3/067 Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
• CPC Class Codes: H04L 67/1097 for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]

Abstract

A method begins by, for a data access request, a user computing device accessing a plurality of estimated efficiency models of a plurality of dispersed storage (DS) processing units of a dispersed storage network. The method continues by selecting one of the DS processing units from the plurality of DS processing units based on the plurality of estimated efficiency models, a type of request of the data access request, and a randomizing factor to produce a selected DS processing unit. The method continues by sending the data access request to the selected DS processing unit for execution. The method continues by determining an actual processing efficiency of the processing of the data access request by the selected DS processing unit. The method continues by updating the estimated efficiency model of the selected DS processing module based on the actual processing efficiency.

Claims

1. A method comprises:

for a data access request, accessing, by a user computing device of a dispersed storage network (DSN), a plurality of estimated efficiency models of a plurality of dispersed storage (DS) processing units of the DSN, wherein an estimated efficiency model of the plurality of estimated efficiency models includes a list of estimated efficiency probabilities, wherein the list of estimated efficiency probabilities corresponds to a list of data access request types for a DS processing unit of the plurality of DS processing units;

selecting, by the user computing device, one of the DS processing units from the plurality of DS processing units based on the plurality of estimated efficiency models, a type of request of the data access request, and a randomizing factor to produce a selected DS processing unit;

sending, by the user computing device, the data access request to the selected DS processing unit for execution;

determining, by the user computing device, an actual processing efficiency of a processing of the data access request by the selected DS processing unit; and

updating, by the user computing device, the estimated efficiency model of the selected DS processing unit based on the actual processing efficiency.

2. The method of claim 1, wherein an estimated efficiency probability of the list of estimated efficiency probabilities comprises one or more of:

an estimated time to complete the data access request by the DS processing unit;

an estimated probability that the estimated time will be met;

an estimated probability of an error occurring when executing the data access request; and

an estimated probability of a connection failure prior to fulfillment of the data access request.

3. The method of claim 1, wherein the selecting the one of the DS processing units comprises:

when the randomizing factor indicates selecting an estimated most efficient processing of the data access request, selecting the one of the DS processing units having a highest estimated efficiency probability for the type of data access request.

4. The method of claim 1, wherein the selecting the one of the DS processing units comprises:

when the randomizing factor indicates selecting an estimated second-most efficient processing of the data access request, selecting the one of the DS processing units having a second highest estimated efficiency probability for the type of data access request; and

when the randomizing factor indicates selecting an estimated third-most efficient processing of the data access request, selecting the one of the DS processing units having a third highest estimated efficiency probability for the type of data access request.

5. The method of claim 1, wherein the selecting the one of the DS processing units comprises:

when the randomizing factor indicates randomly selecting, when the estimated efficiency processing of the data access request is above an efficiency threshold, selecting the one of the DS processing units at random.

6. The method of claim 1 further comprises:

creating, by the user computing device, the plurality of estimated efficiency models based on one or more of historical performance data, estimated performance data, network bandwidth, network reliability, processing resources of the plurality of DS processing units, and data access request volumes.

7. The method of claim 1 further comprises:

for a second data access request, accessing, by a second user computing device of the DSN, a second plurality of estimated efficiency models of the plurality of DS processing units;

selecting, by the second user computing device, a second one of the DS processing units from the plurality of DS processing units based on the second plurality of estimated efficiency models, a type of request of the second data access request, and the randomizing factor to produce a second selected DS processing unit;

sending, by the second user computing device, the second data access request to the second selected DS processing unit for execution;

determining, by the second user computing device, a second actual processing efficiency of a second processing of the second data access request by the second selected DS processing unit; and

updating, by the second user computing device, the estimated efficiency model of the second selected DS processing unit based on the second actual processing efficiency.

8. The method of claim 1 further comprises:

for a second data access request, accessing, by the user computing device, the plurality of estimated efficiency models;

selecting, by the user computing device, a second one of the DS processing units from the plurality of DS processing units based on the plurality of estimated efficiency models, a type of request of the second data access request, and the randomizing factor to produce a second selected DS processing unit;

sending, by the user computing device, the second data access request to the second selected DS processing unit for execution;

determining, by the user computing device, a second actual processing efficiency of a second processing of the second data access request by the second selected DS processing unit; and

updating, by the user computing device, the estimated efficiency model of the second selected DS processing unit based on the second actual processing efficiency.

9. A user computing device of a dispersed storage network (DSN) comprises:

memory;

an interface; and

a processing module operably coupled to the memory and the interface, wherein the processing module is operable to;

for a data access request, access a plurality of estimated efficiency models of a plurality of dispersed storage (DS) processing units of the DSN, wherein an estimated efficiency model of the plurality of estimated efficiency models includes a list of estimated efficiency probabilities, wherein the list of estimated efficiency probabilities corresponds to a list of data access request types for a DS processing unit of the plurality of DS processing units;

select one of the DS processing units from the plurality of DS processing units based on the plurality of estimated efficiency models, a type of request of the data access request, and a randomizing factor to produce a selected DS processing unit;

send, via the interface, the data access request to the selected DS processing unit for execution;

determine an actual processing efficiency of a processing of the data access request by the selected DS processing unit; and

update the estimated efficiency model of the selected DS processing unit based on the actual processing efficiency.

10. The user computing device of claim 9, wherein an estimated efficiency probability of the list of estimated efficiency probabilities comprises one or more of:

an estimated time to complete the data access request by the DS processing unit;

an estimated probability that the estimated time will be met;

an estimated probability of an error occurring when executing the data access request; and

an estimated probability of a connection failure prior to fulfillment of the data access request.

11. The user computing device of claim 9, wherein the processing module is operable to select the one of the DS processing units by:

when the randomizing factor indicates selecting an estimated most efficient processing of the data access request, selecting the one of the DS processing units having a highest estimated efficiency probability for the type of data access request.

12. The user computing device of claim 9, wherein the processing module is operable to select the one of the DS processing units by:

when the randomizing factor indicates selecting an estimated second-most efficient processing of the data access request, selecting the one of the DS processing units having a second highest estimated efficiency probability for the type of data access request; and

when the randomizing factor indicates selecting an estimated third-most efficient processing of the data access request, selecting the one of the DS processing units having a third highest estimated efficiency probability for the type of data access request.

13. The user computing device of claim 9, wherein the processing module is operable to select the one of the DS processing units by:

when the randomizing factor indicates randomly selecting, and when the estimated efficiency processing of the data access request is above an efficiency threshold, selecting the one of the DS processing units at random.

14. The user computing device of claim 9, wherein the processing module is further operable to:

create the plurality of estimated efficiency models based on one or more of historical performance data, estimated performance data, network bandwidth, network reliability, processing resources of the plurality of DS processing units, and data access request volumes.

15. The user computing device of claim 9, wherein the processing module is further operable to:

for a second data access request, access the plurality of estimated efficiency models;

select a second one of the DS processing units from the plurality of DS processing units based on the plurality of estimated efficiency models, a type of request of the second data access request, and the randomizing factor to produce a second selected DS processing unit;

sending, via the interface, the second data access request to the second selected DS processing unit for execution;

determine a second actual processing efficiency of a second processing of the second data access request by the second selected DS processing unit; and

update the estimated efficiency model of the second selected DS processing unit based on the second actual processing efficiency.