Data transfer priority levels

• US 10,229,004 B2
• Filed: 05/02/2016
• Issued: 03/12/2019
• Est. Priority Date: 07/31/2015
• Status: Expired due to Fees

• Current Assignee: Pure Storage, Inc.
• Original Assignee: International Business Machines Corporation
• Inventors: Kazi, Asimuddin, Niedbala, Michael J., Resch, Jason K.
• Primary Examiner(s): Shingles, Kristie D
• Application Number:
  US15/143,885
• Publication Number:
  US 20170034272A1
• Time in Patent Office: 1,044 Days
• Field of Search: 709213
• CPC Class Codes: G06F 11/1076 Parity data used in redundant arrays of independent storages, e.g. in RAID systems
• CPC Class Codes: G06F 11/108 Parity data distribution in semiconductor storages, e.g. in SSD
• CPC Class Codes: G06F 11/1662 the resynchronized component or unit being a persistent storage device re-synchronization of failed mirror storage G06F11/2082; rebuild or reconstruction of parity RAID storage G06F11/1008
• CPC Class Codes: G06F 11/3034 where the computing system component is a storage system, e.g. DASD based or network based digital input from or digital output to record carriers G06F3/06; digital recording or reproducing G11B20/18; for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS], H04L67/1097
• CPC Class Codes: G06F 13/4282 on a serial bus, e.g. I2C bus, SPI bus on daisy chain buses G06F13/4247
• CPC Class Codes: G06F 16/122 using management policies backup systems G06F11/1446; file migration policies for HSM systems G06F16/185
• CPC Class Codes: G06F 16/1824 implemented using Network-attached Storage [NAS] architecture distributed or networked storage systems G06F3/067; protocols for distributed storage of data in a network H04L67/1097
• CPC Class Codes: G06F 16/24578 using ranking
• CPC Class Codes: G06F 21/6218 to a system of files or objects, e.g. local or distributed file system or database
• CPC Class Codes: G06F 21/645 using a third party
• CPC Class Codes: G06F 2201/805 Real-time
• CPC Class Codes: G06F 3/0604 Improving or facilitating administration, e.g. storage management
• CPC Class Codes: G06F 3/0605 by facilitating the interaction with a user or administrator
• CPC Class Codes: G06F 3/061 Improving I/O performance
• CPC Class Codes: G06F 3/0619 in relation to data integrity, e.g. data losses, bit errors
• CPC Class Codes: G06F 3/0622 in relation to access
• CPC Class Codes: G06F 3/064 Management of blocks
• CPC Class Codes: G06F 3/0643 Management of files
• CPC Class Codes: G06F 3/0644 Management of space entities, e.g. partitions, extents, pools
• CPC Class Codes: G06F 3/0647 Migration mechanisms
• CPC Class Codes: G06F 3/065 : Replication mechanisms
• CPC Class Codes: G06F 3/0653 : Monitoring storage devices or systems
• CPC Class Codes: G06F 3/0668 : adopting a particular infrastructure
• CPC Class Codes: G06F 3/067 : Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
• CPC Class Codes: G06F 3/0689 : Disk arrays, e.g. RAID, JBOD
• CPC Class Codes: H03M 13/1515 : Reed-Solomon codes
• CPC Class Codes: H03M 13/2906 : using block codes H03M13/2957 takes precedence
• CPC Class Codes: H03M 13/3761 : using code combining, i.e. using combining of codeword portions which may have been transmitted separately, e.g. Digital Fountain codes, Raptor codes or Luby Transform [LT] codes
• CPC Class Codes: H04L 63/0428 : wherein the data content is protected, e.g. by encrypting or encapsulating the payload
• CPC Class Codes: H04L 63/061 : for key exchange, e.g. in peer-to-peer networks cryptographic mechanisms or cryptographic arrangements for key agreement H04L9/0838
• CPC Class Codes: H04L 63/0853 : using an additional device, e.g. smartcard, SIM or a different communication terminal cryptographic mechanisms or cryptographic arrangements for entity authentication involving additional secure or trusted devices H04L9/3234
• CPC Class Codes: H04L 63/108 : when the policy decisions are valid for a limited amount of time
• CPC Class Codes: H04L 67/1097 : for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]
• CPC Class Codes: H04L 67/63 : Routing a service request depending on the request content or context
• CPC Class Codes: H04L 9/0861 : Generation of secret information including derivation or calculation of cryptographic keys or passwords

Abstract

A method begins by a processing module receiving updated weighting factors for each of a plurality of storage sets. The method continues with the processing module identifying locally stored encoded data slices of a plurality of locally stored encoded data slices for transfer to one or more storage sets of the plurality of storage sets based on the updated weighting factors. The method continues with the processing module, for each identified locally stored encoded data slice, assigning a transfer priority level. The method continues with the processing module, for each identified locally stored encoded data slice, facilitating transfer of the identified locally stored encoded data slice to the one or more storage sets in accordance with the corresponding assigned transfer priority level.

Claims

1. A method for execution by one or more processing modules of one or more computing devices of a dispersed storage network (DSN), the method comprises:

receiving, from a processing unit, updated weighting factors for each of a plurality of storage sets;

identifying locally stored encoded data slices of a plurality of locally stored encoded data slices for transfer to one or more storage sets of the plurality of storage sets based on the updated weighting factors and further by, for each of the plurality of encoded data slices, performing a distributed agreement protocol function on a corresponding slice name to produce a score of a plurality of scores, and identifying an encoded data slice for transfer when a score associated with the identified encoded data slice corresponds to another storage set of the one or more storage sets;

for each identified locally stored encoded data slice, assigning a transfer priority level; and

for each identified locally stored encoded data slice, facilitating transfer of the identified locally stored encoded data slice to the one or more storage sets in accordance with a corresponding assigned transfer priority level.

2. The method of claim 1, wherein the locally stored encoded data slices are identified based on the updated weighting factors and slice names of the plurality of locally stored encoded data slices.

3. The method of claim 1, wherein the assigned transfer priority level is based on one or more of:

a historical frequency of access level, an expected frequency of access level, a prioritization request, a schedule, a data type indicator, a data size indicator, or a DSN activity level.

4. The method of claim 1, wherein the transfer of the identified locally stored encoded data slices is based on a highest transfer priority level first.

5. The method of claim 1, wherein the assigning of transfer priority levels is performed periodically or based on an event.

6. The method of claim 1, wherein the transfer priority level of frequently accessed encoded data slices is assigned as a higher than average priority.

7. A non-transitory computer readable storage medium comprises:

at least one memory section that stores operational instructions that, when executed by one or more processing modules of one or more computing devices of a dispersed storage network (DSN), causes the one or more computing devices to;

receive updated weighting factors for each of a plurality of storage sets;

identify locally stored encoded data slices of a plurality of locally stored encoded data slices for transfer to one or more storage sets of the plurality of storage sets based on the updated weighting factors and further by, for each of the plurality of encoded data slices, performing a distributed agreement protocol function on a corresponding slice name to produce a score of a plurality of scores, and identifying an encoded data slice for transfer when a score associated with the identified encoded data slice corresponds to another storage set of the one or more storage sets;

for each identified locally stored encoded data slice, assign a transfer priority level; and

for each identified locally stored encoded data slice, facilitate transfer of the identified locally stored encoded data slice to the one or more storage sets in accordance with a corresponding assigned transfer priority level.

8. The non-transitory computer readable storage medium of claim 7 further comprises:

the at least one memory section stores further operational instructions that, when executed by the one or more processing modules, causes the one or more computing devices of the DSN to;

identify the locally stored encoded data slices based on the updated weighting factors and slice names of the plurality of locally stored encoded data slices.

9. The non-transitory computer readable storage medium of claim 7 further comprises:

the at least one memory section stores further operational instructions that, when executed by the one or more processing modules, causes the one or more computing devices of the DSN to;

assign the transfer priority level based on one or more of;

a historical frequency of access level, an expected frequency of access level, a prioritization request, a schedule, a data type indicator, a data size indicator, or a DSN activity level.

10. The non-transitory computer readable storage medium of claim 7 further comprises:

the at least one memory section stores further operational instructions that, when executed by the one or more processing modules, causes the one or more computing devices of the DSN to;

identify the transfer of the locally stored encoded data slices based on a highest transfer priority level first.

11. The non-transitory computer readable storage medium of claim 7 further comprises:

the at least one memory section stores further operational instructions that, when executed by the one or more processing modules, causes the one or more computing devices of the DSN to;

assign the priority level of frequently accessed encoded data slices as a higher than average priority.

12. A computing device of a group of computing devices of a dispersed storage network (DSN), the computing device comprises:

an interface;

a local memory; and

a processing module operably coupled to the interface and the local memory, wherein the processing module functions to;

receive updated weighting factors for each of a plurality of storage sets;

identify locally stored encoded data slices of a plurality of locally stored encoded data slices for transfer to one or more storage sets of the plurality of storage sets based on the updated weighting factors and further by, for each of the plurality of encoded data slices, performing a distributed agreement protocol function on a corresponding slice name to produce a score of a plurality of scores, and identifying an encoded data slice for transfer when a score associated with the identified encoded data slice corresponds to another storage set of the one or more storage sets;

for each identified locally stored encoded data slice, assign a transfer priority level; and

for each identified locally stored encoded data slice, facilitate transfer of the identified locally stored encoded data slice to the one or more storage sets in accordance with a corresponding assigned transfer priority level.

13. The computing device of claim 12, wherein the processing module further functions to:

identify the locally stored encoded data slices based on the updated weighting factors and slice names of the plurality of locally stored encoded data slices.

14. The computing device of claim 12, wherein the processing module further functions to:

assign the transfer priority level based on one or more of;

a historical frequency of access level, an expected frequency of access level, a prioritization request, a schedule, a data type indicator, a data size indicator, or a DSN activity level.

15. The computing device of claim 12, wherein the processing module further functions to:

transfer the identified locally stored encoded data slices based on a highest transfer priority level first.

16. The computing device of claim 12, wherein the processing module further functions to:

perform the assigning of transfer priority levels periodically or based on an event.

17. The computing device of claim 12, wherein the processing module further functions to:

assign the transfer priority level of frequently accessed encoded data slices as a higher than average priority.