Method of controlling filling levels of a plurality of storage pools

• US 8,909,730 B2
• Filed: 06/28/2007
• Issued: 12/09/2014
• Est. Priority Date: 10/18/2006
• Status: Expired due to Fees

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Kuhl, Fabian, Mueller, Christian
• Primary Examiner(s): Wang, Liangche A
• Assistant Examiner(s): JOSHI, SURAJ M
• Application Number:
  US12/443,533
• Publication Number:
  US 20100088392A1
• Time in Patent Office: 2,721 Days
• US Class Current: 709/217
• CPC Class Codes: G06F 16/113 Details of archiving lifecycle management in storage systems G06F3/0649; backup systems G06F11/1446
• CPC Class Codes: G06F 16/1744 using compression, e.g. sparse files
• CPC Class Codes: G06F 16/185 Hierarchical storage management [HSM] systems, e.g. file migration or policies thereof details of archiving G06F16/11
• CPC Class Codes: G06F 3/0688 Non-volatile semiconductor memory arrays
• CPC Class Codes: H04L 67/1097 for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]

Abstract

Method, system, and computer program product embodiments of controlling filling levels of a plurality of storage pools are provided. A plurality of files is selected from each storage pool of the plurality of storage pools. Each file is then copied to a server and a list is updated. The list comprises an entry for each file of the plurality of files. Each entry comprises a status information. The status information indicates that the corresponding file has been copied to the server. Each entry further comprises an identifier indicating the storage pool on which the corresponding file is stored. If it is detected that the filling level of a storage pool reaches a predefined first threshold value, then a first set of files is determined by use of the list. Each file of the first set of files is replaced by a stub file. The status information is updated.

Claims

1. A method of controlling filling levels of a plurality of storage pools using a processor device, each storage pool of said plurality of storage pools having assigned an identifier, said method comprising:

providing a predefined threshold filling level value for each respective storage pool;

selecting a plurality of files from each storage pool in accordance with a priority based on a characteristic of each file such that each storage pool comprises a set of selected files including at least one file and a set of unselected files including at least one file;

copying each set of selected files to a server before a filling level of each storage pool reaches each respective predefined threshold value such that each set of selected files from each storage pool is copied to the server and each set of unselected files is not copied to the server, wherein the sets of selected files are copied in an order based on the priority of each selected file;

updating a list comprising an entry for each selected file, each entry comprising a status information indicating a characteristic of each corresponding selected file and that the corresponding selected file has been copied to said server, each entry further comprising the identifier of each respective storage pool on which the corresponding selected file is stored;

monitoring a filling level of each respective storage pool;

if said filling level of each respective storage pool reaches said respective predefined first threshold value, determining a first set of selected files by use of said list and the characteristic of each selected file, said first set of selected files comprising files for which the status information in the corresponding entries indicate that the selected files have been copied to said server and for which the identifiers in the entries correspond to the identifier of said respective storage pool;

replacing said first set of selected files on each storage pool with a plurality of stub files, each selected file in said first set of selected files replaced by a respective stub file and each stub file configured to free storage space in a respective storage pool such that the filling level is less than the respective predefined threshold value and such that a new set of files may be stored therein; and

updating said status information comprised in the entries for each selected file in said first set of selected files, said updated status information indicating that each selected file has been replaced by a respective stub file.

2. The method of claim 1, wherein a storage space consumed by the selected files from each storage pool is larger than a preset second threshold value.

3. The method of claim 1, wherein storage space consumed by the first set of selected files is larger than a preset third threshold value.

4. The method of claim 1, wherein said selected files are selected by use of a given set of rules.

5. The method of claim 1, wherein said first threshold value corresponds to an out of space condition.

6. The method of claim 1, wherein said plurality of storage pools is linked with a policy engine, wherein said policy engine determines said plurality of selected files from said plurality of storage pools, wherein said policy engine provides an object-ID of each file of the plurality of selected files via a command line interface to a storage management system, wherein said command line interface is a component of said storage management system, wherein said storage management system updates said list and copies each selected file of said set of selected files to a server, wherein said storage management system further monitors the plurality of storage pools and if the filling level of said one storage pool reaches the first threshold value, the storage management system replaces the files of said first set of selected files by stub files and updates said list.

7. The method of claim 1, wherein said method is embodied in a computer program product comprising computer executable instructions for performing said method.

8. The method of claim 1, wherein the characteristic comprises a data type of each selected file.

9. The method of claim 8, wherein the data type comprises MPEG files and Stitch Pattern (VIP) files.

10. The method of claim 9, wherein files comprising MPEG files are selected before VIP files.

11. The method of claim 10, wherein larger MPEG files are selected before smaller MPEG files.

12. The method of claim 10, wherein older MPEG files are selected before newer MPEG files.

13. The method of claim 1, wherein larger files are selected before smaller files.

14. The method of claim 1, wherein older files are selected before newly stored files.

15. A computer program product for controlling filling levels of a plurality of storage pools using a processor device, said computer program product comprising a computer-readable memory including executable instructions, said instructions being configured to perform the steps of:

providing a predefined threshold filling level value for each respective storage pool;

selecting a plurality of files from each storage pool in accordance with a priority based on a characteristic of each file such that each storage pool comprises a set of selected files including at least one file and a set of unselected files including at least one file;

copying each set of selected files to a server before the filling level of each respective storage pool is detected to reach its respective predefined threshold value such that each set of selected files from each storage pool is copied to the server and each set of unselected files is not copied to the server, wherein the sets of selected files are copied in an order based on the priority of each selected file;

updating a list comprising an entry for each selected file, each entry comprising a status information indicating a characteristic of each corresponding selected file and that the corresponding selected file has been copied to said server, each entry further comprising the identifier of the storage pool on which the corresponding selected file is stored;

monitoring a filling level of each respective storage pool;

if the filling level of each respective storage pool is detected to reach a respective predefined threshold value, determining a first set of selected files by use of said list and the characteristic of each selected file, said first set of selected files comprising files for which the status information in the corresponding entries indicate that the selected files have been copied to said server and for which the corresponding identifier in the entries corresponds to the identifier of said respective storage pool;

replacing said first set of selected files on each storage pool with a plurality of stub files, each selected file in said first set of files replaced by a respective stub file and each stub file configured to free storage space in a respective storage pool such that the filling level is less than the respective predefined threshold value and such that a new set of files may be stored therein; and

updating the status information comprised in the entries for each selected file in said first set of selected files, said updated status information indicating that each selected file has been replaced by a respective stub file.

16. A data processing system for controlling filling levels of a plurality of storage pools, each of said storage pool having assigned an identifier, said data processing system comprising:

a processor device operable in the data processing system, wherein the processor device is configured for;

providing a predefined threshold filling level value for each respective storage pool;

selecting a plurality of files from each storage pool in accordance with a priority based on a characteristic of each file such that each storage pool comprises a set of selected files including at least one file and a set of unselected files including at least one file;

copying each set of selected files to a server before a filling level of each storage pool reaches each respective predefined threshold value such that each set of selected files from each storage pool is copied to the server and each set of unselected files is not copied to the server, wherein the set of selected files are copied in an order based on the priority of each selected file;

updating a list comprising an entry for each selected file, each entry comprising a status information indicating a characteristic of each corresponding selected file and that the corresponding selected file has been copied to said server, each entry further comprising the identifier of the storage pool on which the corresponding selected file is stored;

monitoring a filling level of each respective storage pool;

if the filling level of each respective storage pool is detected to reach a respective predefined threshold value, determining a first set of selected files by use of said list and the characteristic of each selected file, said first set of selected files comprising files for which the status information in the corresponding entries indicate that the selected files have been copied to said server and for which the corresponding identifier in the entries corresponds to the identifier of said respective storage pool;

replacing said first set of selected files on each storage pool with a plurality of stub files, each file in said first set of selected files replaced by a respective stub file and each stub file configured to free storage space in a respective storage pool such that the filling level is less than the respective predefined threshold value and such that a new set of files may be stored therein; and

updating the status information comprised in the entries for each selected file in said first set of selected files, said updated status information indicating that each selected file has been replaced by a respective stub file.

17. The data processing system of claim 16, wherein the processor device is further configured for selecting said files from one storage pool so that a storage space consumed by the selected files is larger than a preset second threshold value.

18. The data processing system of claim 16, wherein the processor device is further configured for determining the first set of selected files so that the storage space consumed by the first set of selected files is larger than a preset third threshold value.

19. The data processing system of claim 16, wherein the processor device is further configured for storing at least one set of rules, and selecting said plurality of selected files according to said at least one set of rules.

20. The data processing system of claim 16, further comprising a policy engine operable on the processor device and a storage management system, wherein the policy engine is linked with the storage management system, wherein said plurality of storage pools is linked with the policy engine, wherein said policy engine is configured for selecting said plurality of files from said plurality of storage pools, wherein said policy engine is able to provide information about each file of the set of selected files to said storage management system, wherein said storage management system is configured for updating said list and to copy each selected file of said set of selected files to the server, wherein said storage management system is further able to monitor the plurality of storage pools, and if the filling level of a storage pool reaches said first threshold value, the storage management system is able to replace the files of said first set of selected files by stub files and to update said list.