SYSTEM AND METHOD FOR STORING REDUNDANT INFORMATION

• US 20080243914A1
• Filed: 03/28/2008
• Published: 10/02/2008
• Est. Priority Date: 12/22/2006
• Status: Abandoned Application

• Current Assignee: CommVault Systems Incorporated
• Original Assignee: CommVault Systems Incorporated
• Inventors: Gokhale, Parag, Attarde, Deepak R., Prahlad, Anand, Kottomtharayil, Rajiv, Vijayan Retnamma, Manoj K.
• Application Number:
  US12/058,317
• Publication Number:
  US 20080243914A1
• Time in Patent Office: Days
• US Class Current: 707/103Y
• CPC Class Codes: G06F 11/1453 using de-duplication of the data
• CPC Class Codes: G06F 16/1748 De-duplication implemented within the file system, e.g. based on file segments de-duplication techniques in storage systems for the management of data blocks G06F3/0641
• CPC Class Codes: G06F 3/061 Improving I/O performance
• CPC Class Codes: G06F 3/0638 Organizing or formatting or addressing of data
• CPC Class Codes: G06F 3/065 Replication mechanisms
• CPC Class Codes: G06F 3/067 Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
• CPC Class Codes: G11B 5/86 Re-recording, i.e. transcribing information from one magnetisable record carrier on to one or more similar or dissimilar record carriers by varying the order of the information G11B27/029, G11B27/036

Abstract

A method and system for reducing storage requirements and speeding up storage operations by reducing the storage of redundant data includes receiving a request that identifies one or more data objects to which to apply a storage operation. For each data object, the storage system determines if the data object contains data that matches another data object to which the storage operation was previously applied. If the data objects do not match, then the storage system performs the storage operation in a usual manner. However, if the data objects do match, then the storage system may avoid performing the storage operation.

Claims

1. A method in a computer system for reducing redundant storage of data, the method comprising:

identifying a first copy of data stored on a storage server, wherein the first copy is created at least in part by performing a storage operation on data objects stored on one or more client computers;

processing the first copy of data to identify one or more duplicate data objects stored within the first copy;

creating a new copy of the data that does not contain the duplicate data objects by storing a first instance of each data object within the new copy and replacing additional instances of the data object within the new copy with information for locating the first instance.

2. The method of claim 1 wherein the first copy is stored on random access media and the new copy is stored on sequential media.

3. The method of claim 1 including, after creating the new copy of the data, encrypting the data to prevent unauthorized parties from accessing the data.

4. The method of claim 1 including, before creating the new copy of the data, encrypting the data to prevent unauthorized parties from accessing the data and storing a digest value with each data object identifying the content of the data object, and wherein creating a new copy comprises determining which data objects are first instances and which data objects are additional instances using the digest values and without decrypting the data objects.

5. The method of claim 1 wherein the first copy of the data contains a hierarchy and wherein the method further comprises storing information indicating one or more locations within the hierarchy where each instance of each data object is stored.

6. The method of claim 1 wherein the first copy of the data creates a non-production copy of production data and is stored by a storage server separate from the client computers, and wherein creating a new copy comprises using the first non-production copy to create a new single instanced non-production copy.

7. The method of claim 1 wherein creating a new copy is performed on an auxiliary storage server so that the creating does not consume one or more resources of a production server.

8. A system for reducing redundant copies of files in a storage environment, the system comprising:

a storage manager component configured to create at least a first copy of data by duplicating primary data stored on at least one computer system;

a single instance database component configured to identify files stored by the system and maintain an index for determining if two data objects contain the same data; and

a single instancing component configured to create a second copy of data from the first copy of data, wherein the first copy of data contains more than one instance of at least one data object and the second copy contains only one instance of each data object.

9. The system of claim 8 wherein the index of the single instance database component contains a digest value that summarizes the data stored within each data object identified by the index.

10. The system of claim 8 wherein the storage manager component creates one or more copies of data from multiple computer systems, wherein at least some of the computer systems contain common files that are duplicated within the one or more copies and the second copy contains a single instance of each common file.

11. The system of claim 8 wherein the single instancing component is further configured to identify the data objects within the first copy of the data, query the single instance database component to determine if each data object is identified by the index, and store each data object one time in the second copy.

12. The system of claim 8 wherein the single instancing component is further configured to associate a reference to the one instance with each other instance of each data object contained in the second copy by storing header information that points to the actual location of the data object.

13. The system of claim 8 wherein the other instances do not contain the data object itself.

14. The system of claim 8 wherein the storage manager component is further configured to create an offline copy of a production data server and the single instancing component creates a second copy from the offline copy without impacting the performance of the production data server.

15. A computer-readable medium containing instructions for controlling a computer system to reduce redundant data, by a method comprising:

receiving a list of files contained in a first copy of data, wherein the list contains information associated with each file indicating if other instances of the file are stored within the system;

comparing the list of files and the information received with an index of files stored by the system;

creating a second copy of the data;

for each file in the list of files for which the index identifies a matching file stored by the system, storing a reference to the matching file in the second copy; and

for each file in the list of files for which the index does not identify a matching file stored by the system, storing the file in the second copy.

16. The computer-readable medium of claim 15 including for each file in the list of files for which the index does not identify a matching file stored by the system, updating the index with information about the file.

17. The computer-readable medium of claim 15 wherein receiving a list of files comprises receiving a digest value for each of the files and wherein the index contains a digest value for each file stored by the system such that the digest values for the files in the list can be compared with the digest values in the index.

18. The computer-readable medium of claim 15 wherein creating a second copy of the data comprises creating the second copy on sequential media and including sorting the list of files so that the sequential media contains the data of each file at a lower offset on the sequential media than each reference to the data on the sequential media.

19. The computer-readable medium of claim 15 including restoring the first copy of the data from the second copy of the data.

20. The computer-readable medium of claim 15 wherein the first copy of the data is produced by a third party data management application and the second copy of the data is smaller in size than the first copy of the data because at least some redundant instances of data are removed, and further comprising restoring the data from the second copy of the data.