Controlling filling levels of storage pools

• US 9,361,300 B2
• Filed: 11/07/2014
• Issued: 06/07/2016
• Est. Priority Date: 10/18/2006
• Status: Expired due to Fees

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Kuhl, Fabian, Mueller, Christian
• Primary Examiner(s): JOSHI, SURAJ M
• Application Number:
  US14/536,226
• Publication Number:
  US 20150066864A1
• Time in Patent Office: 578 Days
• Field of Search: 709/217
• US Class Current: 1/1
• CPC Class Codes: G06F 16/113 Details of archiving lifecycle management in storage systems G06F3/0649; backup systems G06F11/1446
• CPC Class Codes: G06F 16/1744 using compression, e.g. sparse files
• CPC Class Codes: G06F 16/185 Hierarchical storage management [HSM] systems, e.g. file migration or policies thereof details of archiving G06F16/11
• CPC Class Codes: G06F 3/0688 Non-volatile semiconductor memory arrays
• CPC Class Codes: H04L 67/1097 for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]

Abstract

Method, system, and computer program product embodiments of controlling filling levels of a plurality of storage pools are provided. A plurality of files is selected from each storage pool of the plurality of storage pools. Each file is then copied to a server and a list is updated. The list comprises an entry for each file of the plurality of files. Each entry comprises a status information. The status information indicates that the corresponding file has been copied to the server. Each entry further comprises an identifier indicating the storage pool on which the corresponding file is stored. If it is detected that the filling level of a storage pool reaches a predefined first threshold value, then a first set of files is determined by use of the list. Each file of the first set of files is replaced by a stub file. The status information is updated.

Claims

1. A method for controlling filling levels of a plurality of storage pools, each storage pool comprising a plurality of files, using a processor device in a computing system including a first computing device storing the plurality of storage pools, a second computing device in communication with the first storage device and storing a list including size information corresponding to each storage pool and to size information and type of data in each file in each storage pool, and a server in communication with the first computing device and the second computing device, said method comprising:

providing a first threshold value for each respective storage pool representing a predefined maximum storage amount;

monitoring an associated filling level of each respective storage pool as each file in each storage pool is copied to the server;

if the filling level of a particular storage pool reaches a corresponding particular first threshold value;

searching the particular storage pool to determine which files are copied to the server,

determining, via the list, a type of data of each file that is copied to the server, and

selecting a first set of files in the particular storage pool based on the type of data and an amount of free space needed to reduce the associated filling level below the corresponding particular first threshold value to store a new set of files; and

replacing each file in the selected first set of files with a respective stub file for freeing the needed storage space in the respective storage pool such that the associated filling level is less than the corresponding particular first threshold value and such that the new set of files may be stored therein, wherein all of a first type of data is selected and replaced before any of a second type of data is selected and replaced based on a priority policy.

2. The method of claim 1, further comprising updating said list to indicate new size information corresponding to the particular storage pool and further indicating that each file in the selected first set of files has been replaced by a respective stub file.

3. The method of claim 1, wherein determining the first set of selected files comprises:

determining a minimum size sufficient to decrease the associated filling level below the corresponding particular first threshold value; and

basing the selection of the selected first set of files on the determined minimum size.

4. The method of claim 3, wherein the determined minimum size is a second threshold value and the selection of each file in the selected first set of files comprises selecting files with a combined size sufficient to exceed the second threshold value.

5. The method of claim 4, wherein selecting the files comprises:

determining a respective size of each file in the particular storage pool, each respective size representing an associated third threshold value that is less than the second threshold value; and

accumulating each selected file to obtain the selected first set of files such that a combined value of the accumulated third threshold values is greater than the second threshold value.

6. The method of claim 1, wherein said plurality of storage pools is linked with a policy engine, wherein said policy engine determines said plurality of selected files from said plurality of storage pools, wherein said policy engine provides an object-ID of each file of the plurality of selected files via a command line interface to a storage management system, wherein said command line interface is a component of said storage management system, wherein said storage management system updates said list and copies each selected file of said set of selected files to the server, wherein said storage management system further monitors the plurality of storage pools and if the filling level of said one storage pool reaches the first threshold value, the storage management system replaces each file in said first set of selected files with a stub file and updates the list.

7. The method of claim 1, wherein the first threshold value corresponds to an out of space condition.

8. A computer program product for controlling filling levels of a plurality of storage pools using a processor device in a computing system including a first computing device storing the plurality of storage pools, a second computing device in communication with the first storage device and storing a list including size information corresponding to each storage pool and to size information and type of data in each file in each storage pool, and a server in communication with the first computing device and the second computing device, said computer program product including a computer-readable memory comprising:

computer code for providing a first threshold value for each respective storage pool representing a predefined maximum storage amount;

computer code for monitoring an associated filling level of each respective storage pool as each file in each storage pool is copied to the server;

computer code for determining if the filling level of a particular storage pool reaches a corresponding particular first threshold value, and in response thereto;

computer code for searching the particular storage pool to determine which files are copied to the server,

computer code for determining, via the list, a type of data of each file that is copied to the server, and

computer code for selecting a first set of files in the particular storage pool based on the type of data and an amount of free space needed to reduce the associated filling level below the corresponding particular first threshold value to store a new set of files; and

computer code for replacing each file in the selected first set of files with a respective stub file for freeing the needed storage space in the respective storage pool such that the associated filling level is less than the corresponding particular first threshold value and such that the new set of files may be stored therein, wherein all of a first type of data is selected and replaced before any of a second type of data is selected and replaced based on a priority policy.

9. The computer program product of claim 8, further comprising computer code for updating said list to indicate new size information corresponding to the particular storage pool and further indicating that each file in the selected first set of files has been replaced by a respective stub file.

10. The computer program product of claim 9, wherein the determined minimum size is a second threshold value and the computer code for selecting each file in the selected first set of files comprises computer code for selecting files with a combined size sufficient to exceed the second threshold value.

11. The computer program product of claim 10, wherein computer code for selecting the files comprises:

computer code for determining a respective size of each file in the particular storage pool, each respective size representing an associated third threshold value that is less than the second threshold value; and

computer code for accumulating each selected file to obtain the selected first set of files such that a combined value of the accumulated third threshold values is greater than the second threshold value.

12. The computer program product of claim 8, wherein the computer code for determining the first set of selected files comprises:

computer code for determining a minimum size sufficient to decrease the associated filling level below the corresponding particular first threshold value; and

computer code for basing the selection of the selected first set of files on the determined minimum size.

13. The computer program product of claim 8, wherein said plurality of storage pools is linked with a policy engine, wherein said policy engine determines said plurality of selected files from said plurality of storage pools, wherein said policy engine provides an object-ID of each file of the plurality of selected files via a command line interface to a storage management system, wherein said command line interface is a component of said storage management system, wherein said storage management system updates said list and copies each selected file of said set of selected files to the server, wherein said storage management system further monitors the plurality of storage pools and if the filling level of said one storage pool reaches the first threshold value, the storage management system replaces each file in said first set of selected files with a stub file and updates the list.

14. A computing system for controlling storage pool filling levels, comprising:

a first computing device storing a plurality of storage pools, each storage pool comprising a plurality of files;

a second computing device in communication with the first storage device and storing a list including size information corresponding to each storage pool and to size information and type of data in each file in each storage pool; and

a server in communication with the first computing device and the second computing device; and

a processor device operable with the first computing device, the second computing device, and the server wherein the processor device is configured for;

providing a first threshold value for each respective storage pool representing a predefined maximum storage amount,

monitoring an associated filling level of each respective storage pool as each file in each storage pool is copied to the server,

determining if the filling level of a particular storage pool reaches a corresponding particular first threshold value, and in response thereto;

searching the particular storage pool to determine which files are copied to the server,

determining, via the list, a type of data of each file that is copied to the server,

selecting a first set of files in the particular storage pool based on the type of data and an amount of free space needed to reduce the associated filling level below the corresponding particular first threshold value to store a new set of files, and

replacing each file in the selected first set of files with a respective stub file for freeing storage space in the respective storage pool such that the associated filling level is less than the corresponding particular first threshold value and such that a new set of files may be stored therein, wherein all of a first type of data is selected and replaced before any of a second type of data is selected and replaced based on a priority policy.

15. The computing system of claim 14, wherein the processor device is further configured for updating said list to indicate new size information corresponding to the particular storage pool and further indicating that each file in the selected first set of files has been replaced by a respective stub file.

16. The computing system of claim 14, wherein when determining the first set of selected files the processor device is configured for:

determining a minimum size sufficient to decrease the associated filling level below the corresponding particular first threshold value; and

basing the selection of the selected first set of files on the determined minimum size.

17. The computing system of claim 16, wherein the determined minimum size is a second threshold value and wherein when selecting each file in the selected first set of files the processor device is configured for selecting files with a combined size sufficient to exceed the second threshold value.

18. The computing system of claim 17, wherein when selecting the files processor device is configured for:

determining a respective size of each file in the particular storage pool, each respective size representing an associated third threshold value that is less than the second threshold value; and

accumulating each selected file to obtain the selected first set of files such that a combined value of the accumulated third threshold values is greater than the second threshold value.

19. The computing system of claim 14, wherein said plurality of storage pools is linked with a policy engine, wherein said policy engine determines said plurality of selected files from said plurality of storage pools, wherein said policy engine provides an object-ID of each file of the plurality of selected files via a command line interface to a storage management system, wherein said command line interface is a component of said storage management system, wherein said storage management system updates said list and copies each selected file of said set of selected files to the server, wherein said storage management system further monitors the plurality of storage pools and if the filling level of said one storage pool reaches the first threshold value, the storage management system replaces each file in said first set of selected files with a stub file and updates the list.

20. The computing system of claim 14, wherein the first threshold value corresponds to an out of space condition.