METHOD AND APPARATUS FOR OPTIMIZING DATA STORAGE

• US 20120284239A1
• Filed: 05/04/2011
• Published: 11/08/2012
• Est. Priority Date: 05/04/2011
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Jadav, Divyesh, Agarwala, Sandip, Bathen, Luis Angel D.
• Granted Patent: US 8,996,480 B2
• Time in Patent Office: Days
• US Class Current: 707/693
• CPC Class Codes: G06F 16/1744 using compression, e.g. sparse files
• CPC Class Codes: G06F 3/0608 Saving storage space on storage systems
• CPC Class Codes: G06F 3/061 Improving I/O performance
• CPC Class Codes: G06F 3/064 Management of blocks
• CPC Class Codes: G06F 3/0643 Management of files
• CPC Class Codes: G06F 3/067 Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]

Abstract

Embodiments of the invention relate to evaluation and storage of data in a computer system configured with a shared pool of resources. A multi-level adaptive compression technique is employed to minimize the cost of data storage based upon the type of data being stored and their access pattern. The costs of data storage include capacity, bandwidth, and compute cycles. Data is transformed local to a client in communication with the shared pool, local to the shared pool, or as a combination with a partial transformation local to the client and a partial transformation local to the shared pool.

Claims

1. A method comprising:

processing application data for storage;

evaluating the processed data including using a real-time analytical model for selecting a data format storage option and a storage location for the data, said evaluation including minimizing an overall cost associated with storage of said data;

said selection of data format storage option includes determining whether to compress said data, and if compression is unwarranted sending the data to a first selected location, if compression is warranted determining an appropriate compression technique that satisfies said analytical model evaluation, compressing the data based upon the appropriate compression technique, and sending the compressed data to a second selected storage location, wherein said first and second selected storage locations are a part of a shared pool of resources and selected from the group consisting of;

different storage locations in the shared pool, and the same storage location in the shared pool.

2. The method of claim 1, wherein evaluation of the data includes an application selecting and applying a compression technique to the data, said application is in a location with respect to a shared pool of resources selected from the group consisting of:

local, remote, and a combination thereof.

3. The method of claim 2, further comprising the local application and the remote application cooperatively achieving compression of the evaluated data.

4. The method of claim 1, further comprising profiling each prior data format selection and data storage location, and quantifying a cost component to the profile data and storing the profile in a data structure.

5. The method of claim 4, further comprising employing said quantified cost component of one or more stored profiles as a factor in evaluating the processed data and selecting a storage format and location.

6. The method of claim 4, wherein the quantified cost component includes a compression ratio of the data, storage location of data, and retrieval costs of the stored data.

7. The method of claim 1, said overall cost associated with storage of said data, includes a cost of transferring said data between a client machine and said shared pool of configurable computer resources and compressing and decompressing said data.

8. A system comprising:

a client machine in communication with application data;

an evaluation manager to process the application data, including use of a real-time analytical model for selection of a data format storage option and a storage location for the data, said evaluation manager to minimize an overall cost associated with storage of said data;

a compression manager in communication with the evaluation manager, the compression manager to determine whether to compress said data, and if compression is unwarranted to send the data to a first selected location, if compression is warranted to determine an appropriate compression technique that satisfies said analytical model evaluation, compress the data with the appropriate compression technique, and to send the compressed data to a second selected storage location, wherein said first and second selected storage locations are a part of a shared pool of resources and selected from the group consisting of;

different storage locations in the shared pool, and the same storage location in the shared pool.

9. The system of claim 8, further comprising the compression manager to select and apply a compression technique to the data, and wherein said compression manager is in a location with respect to a shared pool of resources selected from the group consisting of:

local, remote, and a combination thereof.

10. The system of claim 9, further comprising a first application local to the client machine and a second application remote from the client machine to cooperatively compress the evaluated data.

11. The system of claim 8, further comprising a profile manager in communication with the evaluation manager, the profile manager to profile each prior data format selection and data storage location, and to quantify a cost component to the profiled data and to store the profile in a data structure.

12. The system of claim 11, further comprising the evaluation manager to employ said quantified cost component of one or more stored profiles as a factor in evaluation of the processed data and selection of a storage format and location.

13. The system of claim 11, wherein the quantified cost component includes a compression ratio of the data, storage location of data, and retrieval costs of the stored data.

14. The system of claim 8, wherein said overall cost associated with storage of said data, includes a cost of transferring said data between said client machine and a shared pool of configurable computer resources and a cost of compressing and decompressing said data.

15. A computer program product, the computer program product comprising a computer readable storage medium having computer readable program code embodied therewith, the computer readable program code comprising:

computer readable program code configured to process data, including use of a real-time analytical model for selection of a data format storage option and a storage location for the data, the selection to minimize an overall cost associated with storage of said data;

computer readable program code configured to determine whether to compress said data, and if compression is unwarranted to send the data to a first selected storage location, if compression is warranted to determine an appropriate compression technique that satisfies said analytical model and computer readable program code to send the compressed data to a second selected storage location, wherein said first and second selected storage locations are a part of a shared pool of resources and selected from the group consisting of;

different storage locations in the shared pool, and the same storage location in the shared pool.

16. The computer program product of claim 15, further comprising computer readable program code to select and apply a compression technique to the data, and wherein said program code is delivered as a service over a network and resides in a location with respect to a shared pool of resources selected from the group consisting of:

local, remote, and a combination thereof.

17. The computer program product of claim 16, further comprising a first application of program code local to a client machine in communication with the shared pool of resources over the network and a second application of program code remote from the client machine, the first and second applications of program code to cooperatively compress the evaluated data.

18. The computer program product of claim 15, further comprising computer readable program code to profile each prior data format selection and data storage location, and to quantify a cost component to the profiled data and to store the profile in a data structure.

19. The computer program product of claim 18, further comprising computer readable program code to employ said quantified cost component of one or more stored profiles as a factor in evaluation of the processed data and selection of a storage format and location, and wherein the quantified cost component includes a compression ratio of the data, storage location of data, and retrieval costs of the stored data.

20. The computer program product of claim 15, wherein said overall cost associated with storage of said data, includes a cost of transferring said data between said client machine and a shared pool of configurable computer resources and a cost of compressing and decompressing said data.