Assigning slices to storage locations based on a predicted lifespan

• US 10,031,805 B2
• Filed: 08/09/2016
• Issued: 07/24/2018
• Est. Priority Date: 08/09/2016
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Resch, Jason K., Volvovski, Ilya
• Primary Examiner(s): Chase, Shelly A
• Application Number:
  US15/232,288
• Publication Number:
  US 20180046544A1
• Time in Patent Office: 714 Days
• CPC Class Codes: G06F 11/1076 Parity data used in redundant arrays of independent storages, e.g. in RAID systems
• CPC Class Codes: G06F 3/0616 in relation to life time, e.g. increasing Mean Time Between Failures [MTBF]
• CPC Class Codes: G06F 3/0619 in relation to data integrity, e.g. data losses, bit errors
• CPC Class Codes: G06F 3/0631 by allocating resources to storage systems
• CPC Class Codes: G06F 3/064 Management of blocks
• CPC Class Codes: G06F 3/0644 Management of space entities, e.g. partitions, extents, pools
• CPC Class Codes: G06F 3/0649 Lifecycle management
• CPC Class Codes: G06F 3/067 Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]

Abstract

A computing device includes an interface configured to interface and communicate with a dispersed storage network (DSN), a memory that stores operational instructions, and a processing module operably coupled to the interface and to the memory such that the processing module, when operable within the computing device based on the operational instructions, is configured to perform various operations. The computing device determines characteristic(s) of encoded data slices (EDSs) of a data object and determines a first estimated class for the EDSs based on the characteristic(s) of the EDSs of the data object. The computing device then selects storage locations within storage units (SUs) of the DSN for the EDSs to be distributedly stored based on the first estimated class and collects write and/or delete request analytics associated with the EDSs. Based on updated information regarding the analytics, the computing device may then update/move/change the storage locations of the EDSs.

Claims

1. A computing device comprising:

an interface configured to interface and communicate with a dispersed storage network (DSN), wherein the DSN includes at least one of a wireless communication system, a wire lined communication system, a non-public intranet system, a public internet system, a local area network (LAN), or a wide area network (WAN);

memory that stores operational instructions; and

a processing module operably coupled to the interface and to the memory, wherein the processing module, when operable within the computing device based on the operational instructions, is configured to;

determine at least one characteristic of a plurality of encoded data slices (EDSs) of a data object, wherein the data object is segmented into a plurality of data segments, wherein a data segment of the plurality of data segments is dispersed error encoded in accordance with dispersed error encoding parameters to produce the plurality of EDSs, wherein a read threshold number of EDSs provides for reconstruction of the data segment, and wherein a write threshold number of EDSs provides for a successful transfer of the plurality of EDSs from a first at least one storage location in the DSN to a second at least one storage location in the DSN;

determine a first estimated class from a plurality of classes for the plurality of EDSs based on the at least one characteristic of a plurality of EDSs of the data object;

select a first plurality of storage locations within a plurality of storage units (SUs) of the DSN for the plurality of EDSs to be distributedly stored based on the first estimated class;

collect at least one of write request analytics or delete request analytics associated with the plurality of EDSs distributedly stored within the plurality of SUs of the DSN during a first period of time;

process the at least one of write request analytics or delete request analytics to determine an updated at least one characteristic of the plurality of EDSs of the data object;

when the at least one characteristic of the plurality of EDSs of the data object compares unfavorably with the updated at least one characteristic of the plurality of EDSs of the data object;

determine a second estimated class from the plurality of classes for the plurality of EDSs based on the updated at least one characteristic of the plurality of EDSs of the data object;

select a second plurality of storage locations within the plurality of SUs of the DSN for the plurality of EDSs to be distributedly stored based on the second estimated class; and

direct the plurality of SUs to move at least the write threshold number of EDSs from the first plurality of storage locations within the plurality of SUs of the DSN to the second plurality of storage locations within the plurality of SUs of the DSN at or during a second period of time.

2. The computing device of claim 1, wherein the processing module, when operable within the computing device based on the operational instructions, is further configured to:

when the at least the write threshold number of EDSs have been moved from the first plurality of storage locations within the plurality of SUs of the DSN to the second plurality of storage locations within the plurality of SUs of the DSN, direct the plurality of SUs to move any remaining EDSs of the plurality of EDSs from the first plurality of storage locations within the plurality of SUs of the DSN to the second plurality of storage locations within the plurality of SUs of the DSN at or during a third period of time, wherein fewer than all of the plurality of EDSs are moved from the first plurality of storage locations within the plurality of SUs of the DSN to the second plurality of storage locations within the plurality of SUs of the DSN at or during the second period of time.

3. The computing device of claim 1, wherein the processing module, when operable within the computing device based on the operational instructions, is further configured, when the at least one characteristic of the plurality of EDSs of the data object compares favorably with the updated at least one characteristic of the plurality of EDSs of the data object, to:

collect additional at least one of write request analytics or delete request analytics associated with the plurality of EDSs distributedly stored within the plurality of SUs of the DSN during a third period of time;

process the additional at least one of write request analytics or delete request analytics to determine additional updated at least one characteristic of the plurality of EDSs of the data object;

when the at least one characteristic of the plurality of EDSs of the data object compares unfavorably with the additional updated at least one characteristic of the plurality of EDSs of the data object;

determine the second estimated class or a third estimated class from the plurality of classes for the plurality of EDSs based on the additional updated at least one characteristic of the plurality of EDSs of the data object;

select the second plurality of storage locations or a third plurality of storage locations within the plurality of SUs of the DSN for the plurality of EDSs to be distributedly stored based on the second estimated class or the third estimated class; and

direct the plurality of SUs to move at least the write threshold number of EDSs from the first plurality of storage locations within the plurality of SUs of the DSN to the second plurality of storage locations or the third plurality of storage locations within the plurality of SUs of the DSN at or during a fourth period of time.

4. The computing device of claim 1, wherein the at least one characteristic of the plurality of EDSs of the data object includes at least one of:

a type of the data object, wherein the type of the data object includes at least one of a data type, a video type, an image or photographic type, an audio type, a text type, a spreadsheet type, a graphics type, or an animation type;

a user or owner associated with the data object;

a user or owner provided hint that is associated with the data object;

a size of the data object;

metadata associated with the data object;

a creation date associated with the data object;

a modification date associated with the data object;

a modification frequency associated with the data object;

an estimated expected lifetime (EL) of at least one EDS of the plurality of EDSs of the data object, wherein the estimated EL of the at least one EDS of the plurality of EDSs of the data object is based on at least one of an update counter, a count of a number instances of the at least one EDS of the plurality of EDSs of the data object in a log file associated with the plurality of EDSs of the data object, or at least one other characteristic of the plurality of EDSs of the data object;

or

a comparison of the EL of the at least one EDS of the plurality of EDSs of the data object to an average log compaction interval (LCI) of the log file associated with the plurality of EDSs of the data object.

5. The computing device of claim 4, wherein the log file associated with the plurality of EDSs of the data object includes at least one of a file, an offset within a memory device of the DSN, a zone in a shingled magnetic recording (SMR) drive, a chip within a solid state drive (SSD), a position in a tape reel, or a logical or physical section that can independently be accessed within the memory device or another memory device of the DSN.

6. The computing device of claim 1, wherein the plurality of classes for the plurality of EDSs includes:

a short lifetime expected lifetime (EL) class that is characterized by at least one of write operation or delete operation performed on the plurality of EDSs of the data object with frequency that compares favorably to a threshold number; and

a long lifetime EL class that is characterized by the at least one of write operation or delete operation performed on the plurality of EDSs of the data object with frequency that compares unfavorably to the threshold number.

7. The computing device of claim 1, wherein the computing device is located at a first premises that is remotely located from at least one SU of the plurality of SUs within the DSN.

8. The computing device of claim 1 further comprising:

a SU of the plurality of SUs within the DSN.

9. The computing device of claim 1 further comprising:

a wireless smart phone, a laptop, a tablet, a personal computers (PC), a work station, or a video game device.

10. A method for execution by a computing device, the method comprising:

determining at least one characteristic of a plurality of encoded data slices (EDSs) of a data object, wherein the data object is segmented into a plurality of data segments, wherein a data segment of the plurality of data segments is dispersed error encoded in accordance with dispersed error encoding parameters to produce the plurality of EDSs, wherein a read threshold number of EDSs provides for reconstruction of the data segment, and wherein a write threshold number of EDSs provides for a successful transfer of the plurality of EDSs from a first at least one storage location in a dispersed storage network (DSN) to a second at least one storage location in the DSN, wherein the DSN includes at least one of a wireless communication system, a wire lined communication systems, a non-public intranet system, a public internet system, a local area network (LAN), or a wide area network (WAN);

determining a first estimated class from a plurality of classes for the plurality of EDSs based on the at least one characteristic of a plurality of EDSs of the data object;

selecting a first plurality of storage locations within a plurality of storage units (SUs) of the DSN for the plurality of EDSs to be distributedly stored based on the first estimated class;

collecting at least one of write request analytics or delete request analytics associated with the plurality of EDSs distributedly stored within the plurality of SUs of the DSN during a first period of time;

processing the at least one of write request analytics or delete request analytics to determine an updated at least one characteristic of the plurality of EDSs of the data object;

when the at least one characteristic of the plurality of EDSs of the data object compares unfavorably with the updated at least one characteristic of the plurality of EDSs of the data object;

determining a second estimated class from the plurality of classes for the plurality of EDSs based on the updated at least one characteristic of the plurality of EDSs of the data object;

selecting a second plurality of storage locations within the plurality of SUs of the DSN for the plurality of EDSs to be distributedly stored based on the second estimated class; and

directing the plurality of SUs to move at least the write threshold number of EDSs from the first plurality of storage locations within the plurality of SUs of the DSN to the second plurality of storage locations within the plurality of SUs of the DSN at or during a second period of time.

11. The method of claim 10 further comprising:

directing, when the at least the write threshold number of EDSs have been moved from the first plurality of storage locations within the plurality of SUs of the DSN to the second plurality of storage locations within the plurality of SUs of the DSN, the plurality of SUs to move any remaining EDSs of the plurality of EDSs from the first plurality of storage locations within the plurality of SUs of the DSN to the second plurality of storage locations within the plurality of SUs of the DSN at or during a third period of time, wherein fewer than all of the plurality of EDSs are moved from the first plurality of storage locations within the plurality of SUs of the DSN to the second plurality of storage locations within the plurality of SUs of the DSN at or during the second period of time.

12. The method of claim 10 further comprising, when the at least one characteristic of the plurality of EDSs of the data object compares favorably with the updated at least one characteristic of the plurality of EDSs of the data object:

collecting additional at least one of write request analytics or delete request analytics associated with the plurality of EDSs distributedly stored within the plurality of SUs of the DSN during a third period of time;

processing the additional at least one of write request analytics or delete request analytics to determine additional updated at least one characteristic of the plurality of EDSs of the data object;

when the at least one characteristic of the plurality of EDSs of the data object compares unfavorably with the additional updated at least one characteristic of the plurality of EDSs of the data object;

determining the second estimated class or a third estimated class from the plurality of classes for the plurality of EDSs based on the additional updated at least one characteristic of the plurality of EDSs of the data object;

selecting the second plurality of storage locations or a third plurality of storage locations within the plurality of SUs of the DSN for the plurality of EDSs to be distributedly stored based on the second estimated class or the third estimated class; and

directing the plurality of SUs to move at least the write threshold number of EDSs from the first plurality of storage locations within the plurality of SUs of the DSN to the second plurality of storage locations or the third plurality of storage locations within the plurality of SUs of the DSN at or during a fourth period of time.

13. The method of claim 10, wherein the at least one characteristic of the plurality of EDSs of the data object includes at least one of:

a type of the data object, wherein the type of the data object includes at least one of a data type, a video type, an image or photographic type, an audio type, a text type, a spreadsheet type, a graphics type, or an animation type;

a user or owner associated with the data object;

a user or owner provided hint that is associated with the data object;

a size of the data object;

metadata associated with the data object;

a creation date associated with the data object;

a modification date associated with the data object;

a modification frequency associated with the data object;

an estimated expected lifetime (EL) of at least one EDS of the plurality of EDSs of the data object, wherein the estimated EL of the at least one EDS of the plurality of EDSs of the data object is based on at least one of an update counter, a count of a number instances of the at least one EDS of the plurality of EDSs of the data object in a log file associated with the plurality of EDSs of the data object, or at least one other characteristic of the plurality of EDSs of the data object;

or

a comparison of the EL of the at least one EDS of the plurality of EDSs of the data object to an average log compaction interval (LCI) of the log file associated with the plurality of EDSs of the data object.

14. The method of claim 13, wherein the log file associated with the plurality of EDSs of the data object includes at least one of a file, an offset within a memory device of the DSN, a zone in a shingled magnetic recording (SMR) drive, a chip within a solid state drive (SSD), a position in a tape reel, or a logical or physical section that can independently be accessed within the memory device or another memory device of the DSN.

15. The method of claim 10, wherein the plurality of classes for the plurality of EDSs includes:

a short lifetime expected lifetime (EL) class that is characterized by at least one of write operation or delete operation performed on the plurality of EDSs of the data object with frequency that compares favorably to a threshold number; and

a long lifetime EL class that is characterized by the at least one of write operation or delete operation performed on the plurality of EDSs of the data object with frequency that compares unfavorably to the threshold number.

16. The method of claim 10, wherein the computing device is located at a first premises that is remotely located from at least one SU of the plurality of SUs within the DSN.

17. The method of claim 10, wherein the computing device includes a SU of the plurality of SUs within the DSN.

18. The method of claim 10, wherein the computing device includes a wireless smart phone, a laptop, a tablet, a personal computers (PC), a work station, or a video game device.

19. A method for execution by a computing device, the method comprising:

determining at least one characteristic of a plurality of encoded data slices (EDSs) of a data object, wherein the data object is segmented into a plurality of data segments, wherein a data segment of the plurality of data segments is dispersed error encoded in accordance with dispersed error encoding parameters to produce the plurality of EDSs, wherein a read threshold number of EDSs provides for reconstruction of the data segment, and wherein a write threshold number of EDSs provides for a successful transfer of the plurality of EDSs from a first at least one storage location in a dispersed storage network (DSN) to a second at least one storage location in the DSN, wherein the computing device includes a wireless smart phone, a laptop, a tablet, a personal computers (PC), a work station, or a video game device;

determining a first estimated class from a plurality of classes for the plurality of EDSs based on the at least one characteristic of a plurality of EDSs of the data object;

selecting a first plurality of storage locations within a plurality of storage units (SUs) of the DSN for the plurality of EDSs to be distributedly stored based on the first estimated class;

collecting at least one of write request analytics or delete request analytics associated with the plurality of EDSs distributedly stored within the plurality of SUs of the DSN during a first period of time;

processing the at least one of write request analytics or delete request analytics to determine an updated at least one characteristic of the plurality of EDSs of the data object;

when the at least one characteristic of the plurality of EDSs of the data object compares unfavorably with the updated at least one characteristic of the plurality of EDSs of the data object;

determining a second estimated class from the plurality of classes for the plurality of EDSs based on the updated at least one characteristic of the plurality of EDSs of the data object;

selecting a second plurality of storage locations within the plurality of SUs of the DSN for the plurality of EDSs to be distributedly stored based on the second estimated class; and

directing the plurality of SUs to move at least the write threshold number of EDSs from the first plurality of storage locations within the plurality of SUs of the DSN to the second plurality of storage locations within the plurality of SUs of the DSN at or during a second period of time.