Modifying data storage in response to detection of a memory system imbalance

• US 8,914,609 B2
• Filed: 03/31/2014
• Issued: 12/16/2014
• Est. Priority Date: 02/01/2011
• Status: Expired due to Fees

• Current Assignee: Pure Storage, Inc.
• Original Assignee: Cleversafe Incorporated (International Business Machines Corporation)
• Inventors: Resch, Jason K., Grube, Gary W., Gladwin, S. Christopher
• Primary Examiner(s): Rojas, Midys
• Application Number:
  US14/230,446
• Publication Number:
  US 20140214909A1
• Time in Patent Office: 260 Days
• Field of Search: None
• US Class Current: 711/202
• CPC Class Codes: G06F 16/13 File access structures, e.g. distributed indices arrangements of input from, or output to, record carriers G06F3/06
• CPC Class Codes: G06F 16/1727 Details of free space management performed by the file system saving storage space on storage systems G06F3/0608; management of blocks in storage devices G06F3/064

Abstract

A computing device includes an interface, memory, and a processing module. The memory stores a directory and inode tables. The directory stores a file identifier and a corresponding inumber for each file that is stored in storage units. An inode table stores an inumber, metadata, and a DSN address for each file stored in a corresponding storage unit. The processing module is operable to monitor, for each of the inode tables, utilization of the memory. The processing module is further operable to monitor, for each of the storage units, utilization of memory of the storage units. The processing module is further operable to process, for the inode table and/or the corresponding storage unit, per inode table memory utilization data and per storage unit memory utilization data to adjust memory utilization of the inode table and/or memory utilization of the corresponding storage unit.

Claims

1. A computing device of a dispersed storage network (DSN) comprises:

an interface;

memory that stores;

a directory that includes a file identifying field and an inumber field, wherein the directory stores a file identifier and a corresponding inumber for each of a plurality of files, wherein the plurality of files is stored in storage units of the DSN;

inode tables corresponding to the storage units, wherein an inode table of the inode tables includes an inumber field, a meta data field, and a DSN address field and wherein the inode table stores an inumber, metadata, and a DSN address for each file stored in a corresponding storage unit of the storage units;

a processing module operably coupled to the interface and the memory, wherein the processing module is operable to;

monitor, for each of the inode tables, utilization of the memory to produce per inode table memory utilization data;

monitor, for each of the storage units, utilization of memory of the storage units to produce per storage unit memory utilization data; and

process, for at least one of the inode table and the corresponding storage unit, the per inode table memory utilization data and the per storage unit memory utilization data to adjust at least one of memory utilization of the inode table and memory utilization of the corresponding storage unit.

2. The computing device of claim 1, wherein the processing module is further operable to process the per inode table memory utilization data and the per storage unit memory utilization data by:

detecting an imbalance between the per inode table memory utilization data and the per storage unit memory utilization data, wherein the imbalance indicates that the memory utilization for the inode table is being consumed at a greater rate than the memory utilization of the corresponding storage unit; and

in response to the imbalance, adjusting the memory utilization for the inode table by transferring inode information for one or more files from the inode table to another inode table.

3. The computing device of claim 2, wherein the processing module is further operable to:

in response to the imbalance, determine whether to use one other of the inode tables as the other inode table or to create a new inode table; and

when the determination is to create a new inode table, allocate memory storage of the memory to the new inode table.

4. The computing device of claim 2, wherein the processing module is further operable to:

in response to the imbalance, adjust the memory utilization of the corresponding storage unit by transferring the one or more files from the corresponding storage unit to another storage unit.

5. The computing device of claim 1, wherein the processing module is further operable to process the per inode table memory utilization data and the per storage unit memory utilization data by:

detecting an imbalance between the per inode table memory utilization data and the per storage unit memory utilization data, wherein the imbalance indicates that the memory utilization for the inode table is being consumed at a greater rate than the memory utilization of the corresponding storage unit; and

in response to the imbalance, adjusting the memory utilization for the inode table by;

consolidating a set of inode table entries regarding a set of files into a single inode table entry, wherein the single inode table entry includes a new inumber and a new DSN address;

updating the directory with the new inumber; and

updating the corresponding storage unit with the new DSN address.

6. The computing device of claim 1, wherein the processing module is further operable to process the per inode table memory utilization data and the per storage unit memory utilization data by:

detecting an imbalance between the per inode table memory utilization data and the per storage unit memory utilization data, wherein the imbalance indicates that the memory utilization for the corresponding storage unit is being consumed at a greater rate than the memory utilization of the inode table; and

in response to the imbalance, adjusting the memory utilization for the corresponding storage unit by;

transferring inode information for one or more files from the inode table to another inode table; and

transferring the one or more files from the corresponding storage unit to another storage unit.

7. The computing device of claim 1, wherein the processing module is further operable to:

receive, via the interface, a data access request;

access the directory to identify one or more file identifiers corresponding to data of the data access request;

utilize one or more corresponding inumbers associated with the one or more file identifiers to identify one or more of the inode tables;

determining one or more DSN addresses from the one or more the inode tables based on the one or more corresponding inumbers; and

executing the data request utilizing the one or more DSN addresses.

8. A computer readable storage medium comprises:

a first memory that stores a directory that includes a file identifying field and an inumber field, wherein the directory stores a file identifier and a corresponding inumber for each of a plurality of files, wherein the plurality of files is stored in storage units of the DSN;

a second memory that stores inode tables corresponding to the storage units, wherein an inode table of the inode tables includes an inumber field, a meta data field, and a DSN address field and wherein the inode table stores an inumber, metadata, and a DSN address for each file stored in a corresponding storage unit of the storage units;

a third memory that stores operational instructions that, when executed by a computing device, causes the computing device to;

monitor, for each of the inode tables, utilization of the memory to produce per inode table memory utilization data;

monitor, for each of the storage units, utilization of memory of the storage units to produce per storage unit memory utilization data; and

process, for at least one of the inode table and the corresponding storage unit, the per inode table memory utilization data and the per storage unit memory utilization data to adjust at least one of memory utilization of the inode table and memory utilization of the corresponding storage unit.

9. The computer readable storage medium of claim 8, wherein the third memory further stores operational instructions that, when executed by the computing device, causes the computing device to process the per inode table memory utilization data and the per storage unit memory utilization data by:

detecting an imbalance between the per inode table memory utilization data and the per storage unit memory utilization data, wherein the imbalance indicates that the memory utilization for the inode table is being consumed at a greater rate than the memory utilization of the corresponding storage unit; and

in response to the imbalance, adjusting the memory utilization for the inode table by transferring inode information for one or more files from the inode table to another inode table.

10. The computer readable storage medium of claim 9, wherein the third memory further stores operational instructions that, when executed by the computing device, causes the computing device to:

in response to the imbalance, determine whether to use one other of the inode tables as the other inode table or to create a new inode table; and

when the determination is to create a new inode table, allocate memory storage of the memory to the new inode table.

11. The computer readable storage medium of claim 9, wherein the third memory further stores operational instructions that, when executed by the computing device, causes the computing device to:

in response to the imbalance, adjust the memory utilization of the corresponding storage unit by transferring the one or more files from the corresponding storage unit to another storage unit.

12. The computer readable storage medium of claim 8, wherein the third memory further stores operational instructions that, when executed by the computing device, causes the computing device to process the per inode table memory utilization data and the per storage unit memory utilization data by:

detecting an imbalance between the per inode table memory utilization data and the per storage unit memory utilization data, wherein the imbalance indicates that the memory utilization for the inode table is being consumed at a greater rate than the memory utilization of the corresponding storage unit; and

in response to the imbalance, adjusting the memory utilization for the inode table by;

consolidating a set of inode table entries regarding a set of files into a single inode table entry, wherein the single inode table entry includes a new inumber and a new DSN address;

updating the directory with the new inumber; and

updating the corresponding storage unit with the new DSN address.

13. The computer readable storage medium of claim 8, wherein the third memory further stores operational instructions that, when executed by the computing device, causes the computing device to process the per inode table memory utilization data and the per storage unit memory utilization data by:

detecting an imbalance between the per inode table memory utilization data and the per storage unit memory utilization data, wherein the imbalance indicates that the memory utilization for the corresponding storage unit is being consumed at a greater rate than the memory utilization of the inode table; and

in response to the imbalance, adjusting the memory utilization for the corresponding storage unit by;

transferring inode information for one or more files from the inode table to another inode table; and

transferring the one or more files from the corresponding storage unit to another storage unit.

14. The computer readable storage medium of claim 8, wherein the third memory further stores operational instructions that, when executed by the computing device, causes the computing device to:

receive, via the interface, a data access request;

access the directory to identify one or more file identifiers corresponding to data of the data access request;

utilize one or more corresponding inumbers associated with the one or more file identifiers to identify one or more of the inode tables;

determining one or more DSN addresses from the one or more the inode tables based on the one or more corresponding inumbers; and

executing the data request utilizing the one or more DSN addresses.