System and method for optimizing a storage system to support full utilization of storage space

• US 20060075007A1
• Filed: 09/17/2004
• Published: 04/06/2006
• Est. Priority Date: 09/17/2004
• Status: Abandoned Application

• Current Assignee: National Security Agency
• Original Assignee: International Business Machines Corporation
• Inventors: Tracey, John Michael, Douglis, Frederick, Halim, Nagui, Tetzlaff, William Harold, Richards, Elizabeth Suzanne, Palmer, John Davis, Tao, David, Wolf, Joel Leonard, Anderson, Kay Schwendimann
• Application Number:
  US10/943,397
• Publication Number:
  US 20060075007A1
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 3/0608 Saving storage space on storage systems
• CPC Class Codes: G06F 3/0649 Lifecycle management
• CPC Class Codes: G06F 3/0652 Erasing, e.g. deleting, data cleaning, moving of data to a wastebasket
• CPC Class Codes: G06F 3/0683 Plurality of storage devices

Abstract

A system and method for optimizing a storage system to support full utilization of storage space are provided. With the system and method, data objects/containers of data objects are assigned retention values when they are created. These retention values may be dynamically modified based on a modification function associated with the data objects/containers. When storage space needs to be freed for the storage of new data objects/containers, the retention values of existing data objects/containers provide a prioritization as to which data objects/containers should be deleted from the storage system and the order by which these data objects/containers are to be deleted to make available storage space for the new data objects/containers. The identification of the data objects/containers that are to be deleted may be based on a dynamically modified delete threshold, a sorted list of retention values, or the like.

Claims

1. A method of storing data in a data storage system, comprising:

receiving a plurality of data objects, wherein each data object has an associated retention value that identifies a relative importance for storing the data object in the storage system as compared to other data objects having different retention values;

storing the plurality of data objects in the storage system;

determining a relative priority for retention of data objects within the plurality of data objects based on the associated retention values of the data objects; and

deleting data objects of the plurality of data objects in accordance with the determined relative priority for retention of the data objects.

2. The method of claim 1, further comprising:

grouping the plurality of data objects into data containers based on the data objects having similar retention values.

3. The method of claim 1, further comprising:

receiving a change to a retention value of a data object, thereby generating a changed retention value;

determining whether to modify a state of the data object based on the changed retention value; and

modifying the state of the data object if it is determined that the state of the data object should be modified based on the changed retention value.

4. The method of claim 3, wherein the data object is grouped into a data container based on a retention value of the data object, and wherein modifying the state of the data object includes:

reassigning the data object to another data container based on the changed retention value.

5. The method of claim 4, wherein reassigning the data object to another data container includes at least one of generating a new data container for storing the data object and inserting the data object in an existing data container that has available storage space.

6. The method of claim 3, wherein the data object is grouped into a data container based on a retention value of the data object, and wherein modifying the state of the data object includes:

changing a retention value associated with the data container with which the data object is associated based on the changed retention value.

7. The method of claim 3, wherein the data object is grouped into a data container based on a retention value of the data object, and wherein modifying the state of the data object includes:

waiting for a predetermined aggregate change to retention values of data objects in the data container; and

modifying a retention value of the data container based retention values of the data objects in the data container in response to the predetermined aggregate change to retention values of data objects in the data container occurring.

8. The method of claim 3, wherein the change to the retention value is received from an application.

9. The method of claim 3, wherein the change to the retention value is received from applying a retention value modification function to the retention value of the data object.

10. The method of claim 2, wherein the data container is assigned a retention value based on retention values of data objects contained in the data container, and wherein deleting data objects of the plurality of data objects in accordance with the determined relative priority for retention of the data objects includes:

determining if the retention value of the data container has a predetermined relationship with a deletion threshold; and

deleting all of the data objects in the data container, if the retention value of the data container has the predetermined relationship with the deletion threshold.

11. The method of claim 10, further comprising:

dynamically updating a value of the deletion threshold based on a current utilization of the storage system.

12. The method of claim 11, wherein the predetermined relationship is that the retention value is less than or equal to the value of the deletion threshold, and wherein dynamically updating a value of the deletion threshold includes:

determining a current level of usage of the storage system;

increasing the value of the deletion threshold if the current level of usage of the storage system indicates an increase in usage of the storage system; and

decreasing the value of the deletion threshold if the current level of usage of the storage system indicates a decrease in usage of the storage system.

13. A computer program product in a computer readable medium for storing data in a data storage system, comprising:

first instructions for receiving a plurality of data objects, wherein each data object has an associated retention value that identifies a relative importance for storing the data object in the storage system as compared to other data objects having different retention values;

second instructions for storing the plurality of data objects in the storage system;

third instructions for determining a relative priority for retention of data objects within the plurality of data objects based on the associated retention values of the data objects; and

fourth instructions for deleting data objects of the plurality of data objects in accordance with the determined relative priority for retention of the data objects.

14. The computer program product of claim 13, further comprising:

fifth instructions for grouping the plurality of data objects into data containers based on the data objects having similar retention values.

15. The computer program product of claim 13, further comprising:

fifth instructions for receiving a change to a retention value of a data object, thereby generating a changed retention value;

sixth instructions for determining whether to modify a state of the data object based on the changed retention value; and

seventh instructions for modifying the state of the data object if it is determined that the state of the data object should be modified based on the changed retention value.

16. The computer program product of claim 15, wherein the data object is grouped into a data container based on a retention value of the data object, and wherein the seventh instructions for modifying the state of the data object include:

instructions for reassigning the data object to another data container based on the changed retention value.

17. The computer program product of claim 16, wherein the instructions for reassigning the data object to another data container include at least one of instructions for generating a new data container for storing the data object and instructions for inserting the data object in an existing data container that has available storage space.

18. The computer program product of claim 15, wherein the data object is grouped into a data container based on a retention value of the data object, and wherein the seventh instructions for modifying the state of the data object include:

instructions for changing a retention value associated with the data container with which the data object is associated based on the changed retention value.

19. The computer program product of claim 15, wherein the data object is grouped into a data container based on a retention value of the data object, and wherein the seventh instructions for modifying the state of the data object include:

instructions for waiting for a predetermined aggregate change to retention values of data objects in the data container; and

instructions for modifying a retention value of the data container based retention values of the data objects in the data container in response to the predetermined aggregate change to retention values of data objects in the data container occurring.

20. The computer program product of claim 15, wherein the change to the retention value is received from an application.

21. The computer program product of claim 15, wherein the change to the retention value is received from applying a retention value modification function to the retention value of the data object.

22. The computer program product of claim 14, wherein the data container is assigned a retention value based on retention values of data objects contained in the data container, and wherein the fourth instructions for deleting data objects of the plurality of data objects in accordance with the determined relative priority for retention of the data objects include:

instructions for determining if the retention value of the data container has a predetermined relationship with a deletion threshold; and

instructions for deleting all of the data objects in the data container, if the retention value of the data container has the predetermined relationship with the deletion threshold.

23. The computer program product of claim 22, further comprising:

instructions for dynamically updating a value of the deletion threshold based on a current utilization of the storage system.

24. The computer program product of claim 23, wherein the predetermined relationship is that the retention value is less than or equal to the value of the deletion threshold, and wherein the instructions for dynamically updating a value of the deletion threshold include:

instructions for determining a current level of usage of the storage system;

instructions for increasing the value of the deletion threshold if the current level of usage of the storage system indicates an increase in usage of the storage system; and

instructions for decreasing the value of the deletion threshold if the current level of usage of the storage system indicates a decrease in usage of the storage system.

25. A system for storing data in a data storage system, comprising:

means for receiving a plurality of data objects, wherein each data object has an associated retention value that identifies a relative importance for storing the data object in the storage system as compared to other data objects having different retention values;

means for storing the plurality of data objects in the storage system;

means for determining a relative priority for retention of data objects within the plurality of data objects based on the associated retention values of the data objects; and

means for deleting data objects of the plurality of data objects in accordance with the determined relative priority for retention of the data objects.

26. The system of claim 25, further comprising:

means for grouping the plurality of data objects into data containers based on the data objects having similar retention values.

27. The system of claim 25, further comprising:

means for receiving a change to a retention value of a data object, thereby generating a changed retention value;

determining whether to modify a state of the data object based on the changed retention value; and

modifying the state of the data object if it is determined that the state of the data object should be modified based on the changed retention value.