Estimating relative data importance in a dispersed storage network

• US 10,394,630 B2
• Filed: 10/26/2016
• Issued: 08/27/2019
• Est. Priority Date: 10/26/2016
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Kalaf, Dennis A., Kazi, Asimuddin
• Primary Examiner(s): Nguyen, Tu T
• Application Number:
  US15/334,899
• Publication Number:
  US 20180115617A1
• Time in Patent Office: 1,035 Days
• Field of Search: 709223
• CPC Class Codes: G06F 11/00 Error detection; Error correction; Monitoring error detection, correction or monitoring in information storage based on relative movement between record carrier and transducer G11B20/18; monitoring, i.e. supervising the progress of recording or reproducing G11B27/36; in static stores G11C29/00
• CPC Class Codes: G06F 11/1076 Parity data used in redundant arrays of independent storages, e.g. in RAID systems
• CPC Class Codes: G06F 16/182 Distributed file systems
• CPC Class Codes: H04L 67/1097 for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]

Abstract

Methods for use in a dispersed storage network (DSN) to determine relative importance of data objects stored in the DSN for use in making prioritization decisions relating to necessary rebuilding of data. In various examples, relative importance rankings are generated based on heuristic measures and explicit ranking factors. The heuristics measures, which may be updated periodically, may include the frequency of access requests relating to a data object, the number of unique requestors of a data object, the age of a data object, the size of a data object, the amount of other data referenced by a data object, and a predicted amount of other data that would be rendered useless or inaccessible should a data object be lost. The explicit ranking factors may include a storage vault identifier and an importance marker associated with a data object. Higher priority data is preferably rebuilt prior to lower priority data.

Claims

1. A method for execution by one or more processing modules of one or more computing devices of a dispersed storage network (DSN) having a plurality of storage units, the plurality of storage units storing a plurality of data objects in the form of encoded data slices, the method comprises:

generating a first importance ranking for a first data object of the plurality of data objects;

generating a second importance ranking for a second data object of the plurality of data objects, the first importance ranking and the second importance ranking based on one or more ranking factor;

detecting a plurality of the encoded data slices that require rebuilding, wherein each encoded data slice of the plurality of the encoded data slices is a dispersed storage error encoded portion of a respective one of the plurality of data objects, and wherein the plurality of the encoded data slices that require rebuilding include at least one encoded data slice of the first data object and at least one encoded data slice of the second data object;

performing a comparison of the first importance ranking and the second importance ranking; and

based on the comparison, assigning respective rebuilding priority levels to the at least one encoded data slice of the first data object and the at least one encoded data slice of the second data object.

2. The method of claim 1, wherein the first importance ranking and the second importance ranking vary over time, and wherein the one or more ranking factor includes a heuristic measure.

3. The method of claim 2, wherein the heuristic measure includes at least one of:

the frequency of access requests relating to a particular data object, the number of unique requestors of a particular data object, the age of a particular data object, or the size of a particular data object.

4. The method of claim 2, wherein the heuristic measure includes the amount of other data referenced by a particular data object.

5. The method of claim 2, wherein the heuristic measure predicts the amount of other data that would be rendered useless or inaccessible should a particular data object be lost.

6. The method of claim 1, wherein the one or more ranking factor includes an explicit ranking factor.

7. The method of claim 6, wherein the explicit ranking factor includes at least one of a storage vault identifier or an importance marker associated with a particular data object.

8. The method of claim 1, wherein the at least one encoded data slice of the first data object and the at least one encoded data slice of the second data object are stored in a common storage unit.

9. The method of claim 1 further comprises:

facilitating rebuilding of the at least one encoded data slice of the first data object and the at least one encoded data slice of the second data object based on the respective rebuilding priority levels.

10. The method of claim 1 further comprises:

storing the first importance ranking in a metadata object associated with the first data object; and

storing the second importance ranking in a metadata object associated with the second data object.

11. The method of claim 10 further comprises:

updating the first importance ranking and the second importance ranking on a periodic basis.

12. A method for execution by one or more processing modules of one or more computing devices of a dispersed storage network (DSN) having a plurality of storage units, the plurality of storage units storing a plurality of data objects in the form of encoded data slices, the method comprises:

generating a first importance ranking for a first data object of the plurality of data objects;

generating a second importance ranking for a second data object of the plurality of data objects, the first importance ranking and the second importance ranking based on at least one heuristic measure and at least one explicit ranking factor;

detecting a plurality of the encoded data slices that require rebuilding, wherein each encoded data slice of the plurality of the encoded data slices is a dispersed storage error encoded portion of a respective one of the plurality of data objects, and wherein the plurality of the encoded data slices that require rebuilding include at least one encoded data slice of the first data object and at least one encoded data slice of the second data object;

performing a comparison of the first importance ranking and the second importance ranking;

based on the comparison, assigning a higher rebuilding priority level to the at least one encoded data slice of the first data object or the at least one encoded data slice of the second data object; and

facilitating rebuilding of the at least one encoded data slice of the first data object and the at least one encoded data slice of the second data object based on the higher rebuilding priority level.

13. The method of claim 12, wherein the heuristic measure includes at least one of:

the frequency of access requests relating to a particular data object, the number of unique requestors of a particular data object, the age of a particular data object, or the size of a particular data object.

14. The method of claim 12, wherein the at least one encoded data slice of the first data object and the at least one encoded data slice of the second data object are stored in a common storage unit.

15. A computing device of a group of computing devices of a dispersed storage network (DSN) having a plurality of storage units, the computing device comprises:

a network interface;

a local memory; and

a processing module operably coupled to the network interface and the local memory, wherein the processing module operates to;

generate a first importance ranking for a first data object of the plurality of data objects;

generate a second importance ranking for a second data object of the plurality of data objects, the first importance ranking and the second importance ranking based on one or more ranking factor;

detect, via the network interface, a plurality of the encoded data slices that require rebuilding, wherein each encoded data slice of the plurality of the encoded data slices is a dispersed storage error encoded portion of a respective one of the plurality of data objects, and wherein the plurality of the encoded data slices that require rebuilding include at least one encoded data slice of the first data object and at least one encoded data slice of the second data object;

perform a comparison of the first importance ranking and the second importance ranking; and

based on the comparison, assign respective rebuilding priority levels to the at least one encoded data slice of the first data object and the at least one encoded data slice of the second data object.

16. The computing device of claim 15, wherein the first importance ranking and the second importance ranking vary over time, and wherein the one or more ranking factor includes a heuristic measure.

17. The computing device of claim 16, wherein the heuristic measure includes at least one of:

the frequency of access requests relating to a particular data object, the number of unique requestors of a particular data object, the age of a particular data object, or the size of a particular data object.

18. The computing device of claim 16, wherein the heuristic measure includes the amount of other data referenced by a particular data object.

19. The computing device of claim 16, wherein the one or more ranking factor further includes an explicit ranking factor.

20. The computing device of claim 15, wherein the processing module further operates to:

facilitate, via the network interface, rebuilding of the at least one encoded data slice of the first data object and the at least one encoded data slice of the second data object based on the respective rebuilding priority levels.