Transferring data in response to detection of a memory system imbalance

• US 8,694,752 B2
• Filed: 01/04/2012
• Issued: 04/08/2014
• Est. Priority Date: 02/01/2011
• Status: Expired due to Fees

• Current Assignee: Pure Storage, Inc.
• Original Assignee: Cleversafe Incorporated (International Business Machines Corporation)
• Inventors: Gladwin, S. Christopher, Resch, Jason K.
• Primary Examiner(s): Rojas, Midys
• Application Number:
  US13/343,353
• Publication Number:
  US 20120198197A1
• Time in Patent Office: 825 Days
• Field of Search: 711/100, 711/154, 711/170, 711/202, 711/205, 711/207, 711/221, 711/103, 711/E12.001, 711/E12.002
• US Class Current: 711/170
• CPC Class Codes: G06F 16/13 File access structures, e.g. distributed indices arrangements of input from, or output to, record carriers G06F3/06
• CPC Class Codes: G06F 16/1727 Details of free space management performed by the file system saving storage space on storage systems G06F3/0608; management of blocks in storage devices G06F3/064

Abstract

A method begins by a processing module determining an imbalance between inode utilization and data storage utilization. When the imbalance compares unfavorably to an imbalance threshold, the method continues with the processing module determining whether utilization of another inode memory and utilization of another corresponding data storage memory are not imbalanced. When the utilization of the other inode memory and the utilization of the other corresponding data storage memory are not imbalanced, determining whether the inode utilization is out of balance with respect to the data storage utilization. When the inode utilization is out of balance, the method continues with the processing module transferring data objects from a data storage memory to the other corresponding data storage memory and transferring mapping information of data objects from a inode memory to the other inode memory.

Claims

1. A method for execution by a dispersed storage (DS) unit, the method comprises:

determining that an imbalance between a first inode memory utilization of a first inode memory of a plurality of inode memories and first data storage memory utilization of a corresponding first data storage memory of a plurality of data storage memories compares unfavorably to an imbalance threshold, wherein the corresponding first data storage memory stores data as at least one of data objects and data blocks, and wherein the first inode memory stores mapping information that associates directory information of the data with storage location information of the data, wherein a data object of the data objects includes an encoded data slice and a data block of the data blocks includes a plurality of encoded data slices;

determining that a second inode memory utilization of a second inode memory of the plurality of inode memories and a second data storage memory utilization of a corresponding second data storage memory of the plurality of data storage memories have an imbalance that compares favorably to the imbalance threshold;

determining that the corresponding second data storage memory has available storage space;

determining whether the imbalance between the first inode memory utilization and the first data storage memory utilization is due to the first inode memory utilization being greater than the first data storage memory utilization or due to the first data storage memory utilization being greater than first inode memory utilization; and

when the imbalance is due to the first inode memory utilization being greater than the first data storage memory utilization;

transferring at least one data object from the corresponding first data storage memory to the corresponding second data storage memory; and

transferring mapping information of the at least one data object from the first inode memory to the second inode memory.

2. The method of claim 1, wherein the transferring the at least one data object comprises:

determining a desired number of the data objects to include in the at least one data object based on a difference between a desire imbalance and the imbalance between the first inode memory utilization and the first data storage memory utilization;

determining whether transferring the desired number of the data objects will cause the imbalance between the second inode memory utilization and the second data storage memory utilization to compare unfavorably to the imbalance threshold; and

when the transferring the desired number of the data objects will not cause the imbalance between the second inode memory utilization and the second data storage memory utilization to compare unfavorably to the imbalance threshold, transferring the desired amount of the data objects.

3. The method of claim 2 further comprises:

when the transferring the desired number of the data objects will cause the imbalance between the second inode memory utilization and the second data storage memory utilization to compare unfavorably to the imbalance threshold, scaling back on the desired number of data objects until the transferring the desired number of data objects will not cause the imbalance between the second inode memory utilization and the second data storage memory utilization to compare unfavorably to the imbalance threshold;

transferring the scaled-back number of data objects to the corresponding second data storage memory;

for a difference between the desired number of data objects and the scaled-back number of data objects;

determining that a third inode memory utilization of a third inode memory of the plurality of inode memories and a third data storage memory utilization of a third corresponding data storage memory of the plurality of data storage memories have an imbalance that compares favorably to the imbalance threshold;

determining that the third corresponding data storage memory has available storage space;

transferring a difference amount of data objects to the third corresponding data storage memory, wherein the difference amount of data objects equals the desired number of data objects less the scaled-back number of data objects; and

transferring mapping information of the difference amount of the data objects from the first inode memory to the third inode memory.

4. The method of claim 1 further comprises:

when the imbalance between the first inode memory utilization and the first data storage memory utilization is due to the first data storage memory utilization being greater than first inode memory utilization;

transferring one or more data blocks from the corresponding first data storage memory to the corresponding second data storage memory; and

transferring mapping information of the one or more data blocks from the first inode memory to the second inode memory.

5. The method of claim 4, wherein the transferring the one or more data blocks comprises:

determining a desired number of the data blocks to include in the one or more data blocks based on a difference between a desire imbalance and the imbalance between the first inode memory utilization and the first data storage memory utilization;

determining whether transferring the desired number of the data blocks will cause the imbalance between the second inode memory utilization and the second data storage memory utilization to compare unfavorably to the imbalance threshold; and

when the transferring the desired number of the data blocks will not cause the imbalance between the second inode memory utilization and the second data storage memory utilization to compare unfavorably to the imbalance threshold, transferring the desired number of the data blocks.

6. The method of claim 5 further comprises:

when the transferring the desired number of the data blocks will cause the imbalance between the second inode memory utilization and the second data storage memory utilization to compare unfavorably to the imbalance threshold, scaling back on the desired number of data blocks until the transferring the desired number data blocks will not cause the imbalance between the second inode memory utilization and the second data storage memory utilization to compare unfavorably to the imbalance threshold;

transferring the scaled-back number of data blocks to the corresponding second data storage memory;

for a difference between the desired number of data blocks and the scaled-back number of data blocks;

determining that a third inode memory utilization of a third inode memory of the plurality of inode memories and a third data storage memory utilization of a third corresponding data storage memory of the plurality of data storage memories have an imbalance that compares favorably to the imbalance threshold;

determining that the third corresponding data storage memory has available storage space;

transferring a difference amount of data blocks to the third corresponding data storage memory, wherein the difference amount of data blocks equals the desired number of data blocks less the scaled-back number of data blocks; and

transferring mapping information of the different amount of the data blocks from the first inode memory to the third inode memory.

7. A dispersed storage (DS) module comprises:

a first module operable to determine that an imbalance between a first inode memory utilization of a first inode memory of a plurality of inode memories and first data storage memory utilization of a corresponding first data storage memory of a plurality of data storage memories compares unfavorably to an imbalance threshold, wherein the corresponding first data storage memory stores data as at least one of data objects and data blocks, and wherein the first inode memory stores mapping information that associates directory information of the data with storage location information of the data, wherein a data object of the data objects includes an encoded data slice and a data block of the data blocks includes a plurality of encoded data slices;

a second module operable to;

determine that a second inode memory utilization of a second inode memory of the plurality of inode memories and a second data storage memory utilization of a corresponding second data storage memory of the plurality of data storage memories have an imbalance that compares favorably to the imbalance threshold; and

determine that the corresponding second data storage memory has available storage space;

a third module operable to determine whether the imbalance between the first inode memory utilization and the first data storage memory utilization is due to the first inode memory utilization being greater than the first data storage memory utilization or due to the first data storage memory utilization being greater than first inode memory utilization; and

when the imbalance is due to the first inode memory utilization being greater than the first data storage memory utilization;

a fourth module operable to transfer at least one data object from the corresponding first data storage memory to the corresponding second data storage memory; and

a fifth module operable to transfer mapping information of the at least one data object from the first inode memory to the second inode memory.

8. The DS module of claim 7, wherein the fourth module is further operable to transfer the at least one data object by:

determining a desired number of the data objects to include in the at least one data object based on a difference between a desire imbalance and the imbalance between the first inode memory utilization and the first data storage memory utilization;

determining whether transferring the desired number of the data objects will cause the imbalance between the second inode memory utilization and the second data storage memory utilization to compare unfavorably to the imbalance threshold; and

when the transferring the desired number of the data objects will not cause the imbalance between the second inode memory utilization and the second data storage memory utilization to compare unfavorably to the imbalance threshold, transferring the desired amount of the data objects.

9. The DS module of claim 8 further comprises:

when the transferring the desired number of the data objects will cause the imbalance between the second inode memory utilization and the second data storage memory utilization to compare unfavorably to the imbalance threshold, the fourth module is further operable to scale back on the desired number of data objects until the transferring the desired number of data objects will not cause the imbalance between the second inode memory utilization and the second data storage memory utilization to compare unfavorably to the imbalance threshold;

the fourth module is further operable to transfer the scaled-back number of data objects to the corresponding second data storage memory;

for a difference between the desired number of data objects and the scaled-back number of data objects;

the second module is further operable to determine that a third inode memory utilization of a third inode memory of the plurality of inode memories and a third data storage memory utilization of a third corresponding data storage memory of the plurality of data storage memories have an imbalance that compares favorably to the imbalance threshold;

determine that the third corresponding data storage memory has available storage space; and

the fourth module is further operable to transfer a difference amount of data objects to the third corresponding data storage memory, wherein the difference amount of data objects equals the desired number of data objects less the scaled-back number of data objects; and

the fifth module is further operable to transfer mapping information of the difference amount of the data objects from the first inode memory to the third inode memory.

10. The DS module of claim 7 further comprises:

when the imbalance between the first inode memory utilization and the first data storage memory utilization is due to the first data storage memory utilization being greater than first inode memory utilization;

the fourth module is further operable to transfer one or more data blocks from the corresponding first data storage memory to the corresponding second data storage memory; and

the fifth module is further operable to transfer mapping information of the one or more data blocks from the first inode memory to the second inode memory.

11. The DS module of claim 10, wherein the fourth module is further operable to transfer data blocks by:

determining a desired number of the data blocks to include in the one or more data blocks based on a difference between a desire imbalance and the imbalance between the first inode memory utilization and the first data storage memory utilization;

determining whether transferring the desired number of the data blocks will cause the imbalance between the second inode memory utilization and the second data storage memory utilization to compare unfavorably to the imbalance threshold; and

when the transferring the desired number of the data blocks will not cause the imbalance between the second inode memory utilization and the second data storage memory utilization to compare unfavorably to the imbalance threshold, transferring the desired number of the data blocks.

12. The DS module of claim 11 further comprises:

number of the data blocks will cause the imbalance between the second inode memory utilization and the second data storage memory utilization to compare unfavorably to the imbalance threshold, the fourth module is further operable to scale back on the desired number of data blocks until the transferring the desired number data blocks will not cause the imbalance between the second inode memory utilization and the second data storage memory utilization to compare unfavorably to the imbalance threshold;

the fourth module is further operable to transfer the scaled-back number of data blocks to the corresponding second data storage memory;

for a difference between the desired number of data blocks and the scaled-back number of data blocks, the second module is further operable to;

determine that a third inode memory utilization of a third inode memory of the plurality of inode memories and a third data storage memory utilization of a third corresponding data storage memory of the plurality of data storage memories have an imbalance that compares favorably to the imbalance threshold;

determine that the third corresponding data storage memory has available storage space; and

the fourth module is further operable to transfer a difference amount of data blocks to the third corresponding data storage memory, wherein the difference amount of data blocks equals the desired number of data blocks less the scaled-back number of data blocks; and

the fifth module is further operable to transfer mapping information of the different amount of the data blocks from the first inode memory to the third inode memory.