Optimizing data storage among a plurality of data storage repositories

• US 9,191,437 B2
• Filed: 12/09/2009
• Issued: 11/17/2015
• Est. Priority Date: 12/09/2009
• Status: Active Grant

• Current Assignee: Daedalus Blue LLC
• Original Assignee: International Business Machines Corporation
• Inventors: Feder, Oshrit, Gordon, Abel, Goft, German
• Primary Examiner(s): HASAN, SYED HAROON
• Application Number:
  US12/633,924
• Publication Number:
  US 20110137870A1
• Time in Patent Office: 2,169 Days
• US Class Current: 1/1
• CPC Class Codes: G06F 16/174 Redundancy elimination performed by the file system management of the data involved in backup or backup restore using de-duplication of the data G06F11/14
• CPC Class Codes: H04L 67/1001 for accessing one among a plurality of replicated servers

Abstract

Machines and methods for optimizing data storage among a plurality of data storage repositories are provided. The method comprises processing a data request submitted for storing data on one or more data storage mediums; determining whether duplicate copies of the data are to be maintained on more than one storage medium so that established quality of service requirements are met; and managing duplicate copies of the data on said one or more data storage mediums to meet the established quality of service requirements while servicing the data request.

Claims

1. A method for optimizing data storage among a plurality of data storage repositories, the method comprising:

processing a data request submitted by a first application for storing data on one or more data storage mediums connected to a distributed communications network;

determining a number and location of duplicate copies of the data to be maintained on more than one storage medium so that established quality of service requirements associated with the first application are met;

managing duplicate copies of the data on said one or more data storage mediums to meet the established quality of service requirements for a group of applications that also use duplicated copies of said data while servicing the data request submitted by the first application,

wherein managing the duplicate copies of the data comprises deduplicating copies of the data on one or more data storage mediums, in response to determining that the deduplication does not adversely affect one or more of the following in the distributed communication network;

data access latency, data recovery reliability, data storage costs, and data storage bandwidth, wherein said one or more data storage mediums are positioned at various data paths in the distributed communications network such that maintaining duplicate copies of the data in at least some of the one or more data storage mediums helps at least one of data access latency, data recovery reliability, data storage costs, and data storage bandwidth in the distributed communications network, and

wherein said deduplicating comprises determining a unique ID associated with the data and a location type to remove the data item from.

2. The method of claim 1 wherein duplicate copies of the data are managed in a way to reduce data redundancy in copies of the data stored on the one or more data storage mediums in an effort to reduce costs associated with storing or retaining the data on said storage mediums.

3. The method of claim 1 wherein the managing of the duplicate copies of the data comprises storing the duplicate copies of the data on more than one data storage medium.

4. The method of claim 1 wherein the managing of the duplicate copies of the data comprises removing the duplicate copies of the data from one or more data storage medium.

5. The method of claim 1 wherein storage location of duplicate copies of the data is maintained in a directory, wherein said directory is updated when the data request is serviced.

6. The method of claim 1 wherein the data request is submitted in association with meta data, wherein the meta data defines quality of service requirements for the data.

7. The method of claim 6 wherein the meta data is stored in a meta data repository so that entries in an optimization directory are updated based on content of the meta data repository.

8. The method of claim 7 wherein the optimization directory comprises information about location of storage of the data and corresponding duplicates for the data.

9. The method of claim 7 wherein the optimization directory comprises information about quality of service associated with the data with respect to location of storage of the data among said one or more storage repositories.

10. The method of claim 1 wherein the data is stored in a temporary storage medium until it is determined how the duplicate copies of the data are to be managed.

11. The method of claim 1 wherein the unique ID associated with the data is determined based on a hash value.

12. The method of claim 1 wherein the unique ID associated with the data is determined based a pseudo-random value or an indexing value.

13. A system for optimizing data storage among a plurality of data storage repositories, the system comprising:

a logic unit to process a data request submitted by a first application for storing data on one or more data storage mediums connected to a distributed communications network;

a logic unit to determine a number and location of duplicate copies of the data to be maintained on more than one storage medium so that established quality of service requirements are met associated with the first application; and

a logic unit to manage duplicate copies of the data on said one or more data storage mediums to meet the established quality of service requirements for a group of applications that also use duplicated copies of said data while servicing the data request submitted by the first application, wherein managing the duplicate copies of the data comprises deduplicating copies of the data on one or more data storage mediums, in response to determining that the deduplication does not affect one or more of the following in the distributed communication network;

data access latency, data recovery reliability, data storage costs, and data storage bandwidth,

wherein said one or more data storage mediums are positioned at various data paths in the distributed communications network such that maintaining duplicate copies of the data in at least some of the one or more data storage mediums helps at least one of data access latency, data recovery reliability, data storage costs, and data storage bandwidth in the distributed communications network, and

wherein said deduplicating comprises determining a unique ID associated with the data and a location type to remove the data item from.

14. The system of claim 13 wherein duplicate copies of the data are managed in a way to reduce data redundancy in copies of the data stored on the one or more data storage mediums in an effort to reduce costs associated with storing or retaining the data on said storage mediums.

15. The system of claim 13 wherein the managing of the duplicate copies of the data comprises storing the duplicate copies of the data on more than one data storage medium.

16. The system of claim 13 wherein the managing of the duplicate copies of the data comprises removing the duplicate copies of the data from one or more data storage medium.

17. The system of claim 13 wherein storage location of duplicate copies of the data is maintained in a directory, wherein said directory is updated when the data request is serviced.

18. The system of claim 13 wherein the data request is submitted in association with meta data, wherein the meta data defines quality of service requirements for the data.

19. The system of claim 18 wherein the meta data is stored in a meta data repository so that entries in an optimization directory are updated based on content of the meta data repository.

20. The system of claim 19 wherein the optimization directory comprises information about location of storage of the data and corresponding duplicates for the data.

21. The system of claim 19 wherein the optimization directory comprises information about quality of service associated with the data with respect to location of storage of the data among said one or more storage repositories.

22. The system of claim 13 wherein the data is stored in a temporary storage medium until it is determined how the duplicate copies of the data are to be managed.

23. The system of claim 13 wherein the unique ID associated with the data is determined based on a hash value.

24. The system of claim 13 wherein the unique ID associated with the data is determined based a pseudo-random value or an indexing value.

25. A computer program product comprising a non-transitory computer readable storage medium having a computer readable program, wherein the computer readable program when executed on a computer causes the computer to:

process a data request submitted by a first application for storing data on one or more data storage mediums connected to a distributed communications network;

determine a number and location of duplicate copies of the data ae to be maintained on more than one storage medium so that established quality of service requirements associated with the first application are met; and

manage duplicate copies of the data on said one or more data storage mediums to meet the established quality of service requirements for a group of applications that also use duplicated copies of said data while servicing the data request submitted by the first application, wherein managing the duplicate copies of the data comprises deduplicating copies of the data on one or more data storage mediums, in response to determining that the deduplication does not adversely affect one or more of the following in the distributed communication network;

copies of the data on one or more data storage mediums based on factors that at least affect one or more of the following data access latency, data recovery, reliability, data storage costs, and data storage bandwidth,

wherein said one or more data storage mediums are positioned at various data paths in the distributed communications network such that maintaining duplicate copies of the data in at least some of the one or more data storage mediums helps at least one of data access latency, data recovery reliability, data storage costs, and data storage bandwidth m the distributed communications network, and

wherein said deduplicating comprises determining a unique ID associated with the data and a location type to remove the data item from.

26. The computer program product of claim 25, wherein duplicate copies of the data are managed in a way to reduce data redundancy in copies of the data stored on the one or more data storage mediums in an effort to reduce costs associated with storing or retaining the data on said storage mediums.

27. The computer program product of claim 25, wherein the managing of the duplicate copies of the data comprises storing the duplicate copies of the data on more than one data storage medium.

28. The computer program product of claim 25, wherein the managing of the duplicate copies of the data comprises removing the duplicate copies of the data from one or more data storage medium.

29. The computer program product of claim 25, wherein storage location of duplicate copies of the data is maintained in a directory, wherein said directory is updated when the data request is serviced.