SYSTEM AND METHOD FOR STORING REDUNDANT INFORMATION

• US 20080243879A1
• Filed: 12/21/2007
• Published: 10/02/2008
• Est. Priority Date: 12/22/2006
• Status: Active Grant

• Current Assignee: CommVault Systems Incorporated
• Original Assignee: CommVault Systems Incorporated
• Inventors: Gokhale, Parag, Kottomtharayil, Rajiv
• Granted Patent: US 7,840,537 B2
• Time in Patent Office: Days
• US Class Current: 707/100
• CPC Class Codes: G06F 11/1453 using de-duplication of the data
• CPC Class Codes: G06F 16/1748 De-duplication implemented within the file system, e.g. based on file segments de-duplication techniques in storage systems for the management of data blocks G06F3/0641
• CPC Class Codes: G06F 3/061 Improving I/O performance
• CPC Class Codes: G06F 3/0638 Organizing or formatting or addressing of data
• CPC Class Codes: G06F 3/065 Replication mechanisms
• CPC Class Codes: G06F 3/067 Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
• CPC Class Codes: G11B 5/86 Re-recording, i.e. transcribing information from one magnetisable record carrier on to one or more similar or dissimilar record carriers by varying the order of the information G11B27/029, G11B27/036

Abstract

A method and system for reducing storage requirements and speeding up storage operations by reducing the storage of redundant data includes receiving a request that identifies one or more data objects to which to apply a storage operation. For each data object, the storage system determines if the data object contains data that matches another data object to which the storage operation was previously applied. If the data objects do not match, then the storage system performs the storage operation in a usual manner. However, if the data objects do match, then the storage system may avoid performing the storage operation.

Claims

1. A method in a computer system for storing a data object to sequential media, comprising:

receiving a request to store in a destination location a copy of a data object, wherein the data object is stored at a source location on random access media and the destination location is on sequential media;

determining if a copy of the data object is already stored in a location associated with the sequential media; and

if the data object is already stored in a location associated with the sequential media, storing a reference to the data object in the destination location, and not storing at least a portion of the data object at the destination location.

2. The method of claim 1 wherein the determining comprises:

computing a hash value of the data object; and

comparing the computed hash value to another hash value of a data object previously stored by the system.

3. The method of claim 1 wherein the location associated with the sequential media is a tape that is part of a tape library that includes the destination location.

4. The method of claim 1 wherein storing a reference comprises storing index information containing at least a hash value of the data and a reference count indicating the number of references to the data.

5. The method of claim 1 wherein the reference comprises an address that points to the location different from the source location where the data object is already stored.

6. The method of claim 1 wherein the source location comprises a disk of a client computer system and the destination location comprises a tape library associated with a storage server.

7. The method of claim 1 wherein the source location is a first non-production copy of the data object and the destination location is a secondary non-production copy of the data object.

8. The method of claim 1 wherein the source location is backup data containing multiple instances of the data object.

9. A system for reducing redundant copies of files in a storage environment having sequential media, the system comprising:

a storage operation request component configured to receive requests to perform storage operations;

a digest generation component configured to compute a digest that provides summary of a file that distinguishes it from other files referred to by storage operation requests;

a digest comparison component configured to compare computed digests with previously stored digests to determine if a copy of a file already exists; and

a single instance data store configured to store computed digests and files on sequential media such that only one instance of the same file is stored in a set of sequential media that contains multiple references to the same file.

10. The system of claim 9 wherein the storage operation request component comprises a media agent component that stores an index of each of the files and computed digests associated with requests received by the storage operation request component.

11. The system of claim 9 wherein the digest generation computes digests using an MD5 hash algorithm.

12. The system of claim 9 wherein the digest comparison component compares the digest of a file being copied with a digest stored in the single instance data store.

13. The system of claim 9 wherein the single instance data store contains a reference count for each file indicating how many different locations tracked by the single instance data store refer to the file.

14. A computer-readable medium containing instructions for controlling a computer system to recover data, by a method comprising:

retrieving a backup object identifying a data object;

determining whether the backup object contains a reference to the data object or contains the data object instelf;

if the backup object refers to the data object stored in a location outside of the backup object, then locating the data object at the location outside of the backup object and copying the data object to a recovery location; and

if the backup object contains the data object, copying the data object from the backup object to the recovery location,

wherein a first instance of each data object is stored as a first backup object and each additional instance is stored as a reference to the first backup object.

15. The computer-readable medium of claim 14 wherein the backup object is a data structure containing header information identifying whether the object contains a data object or a reference.

16. The computer-readable medium of claim 14 wherein copying the data object to the recovery location comprises first copying the data object to an intermediate restored file cache, wherein the restored file cache contains a copy of the data object with any references to other data objects resolved.

17. The computer-readable medium of claim 14 wherein retrieving a backup object comprises retrieving the backup object from a tape, and wherein the backup object contains information for building an index of each of the backup objects and references to backup objects maintained by the computer system.

18. A system for storing data, comprising:

means for receiving a request to store on a destination sequential storage device a copy of a file stored on a source non-sequential data storage device;

means for determining if a copy of the file is already stored on the destination sequential storage device; and

means for storing a reference to the file on the destination sequential storage device if the file is already stored on the destination sequential storage device.

19. The system of claim 18 wherein the means for storing a reference stores a reference to data at one location on the destination storage device at a different location on the destination storage device.

20. The system of claim 18 wherein the means for a reference increments a reference count for the file if the file is already stored on the destination storage device.

21-80. -80. (canceled)