Systems and methods for stored data verification

• US 8,156,086 B2
• Filed: 10/09/2008
• Issued: 04/10/2012
• Est. Priority Date: 11/13/2003
• Status: Active Grant

• Current Assignee: CommVault Systems Incorporated
• Original Assignee: CommVault Systems Incorporated
• Inventors: Lu, Jun, Amarendran, Arun
• Primary Examiner(s): Kerzhner, Aleksandr
• Application Number:
  US12/248,779
• Publication Number:
  US 20090043830A1
• Time in Patent Office: 1,279 Days
• Field of Search: 707/690, 707/999.204
• US Class Current: 707/690
• CPC Class Codes: G06F 11/1448 Management of the data involved in backup or backup restore
• CPC Class Codes: Y10S 707/99943 Generating database or data structure, e.g. via user interface
• CPC Class Codes: Y10S 707/99953 Recoverability
• CPC Class Codes: Y10S 707/99955 Archiving or backup

Abstract

Methods and systems are described for verifying stored data by receiving a first set of metadata associated with a first set of stored data, generating a second set of metadata associated with a second set of stored data which is associated with the first set of stored data, and comparing the first set of metadata and second set of metadata. Alternatively, the storage system can also generate a first set of metadata associated with a first set of stored data, generate a second set of stored data which is a copy of the first set of stored data, generate a second set of metadata associated with the second set of stored data, and compare the first set of metadata and the second set of metadata.

Claims

1. A method for verifying data copies in a storage management system, the method comprising:

performing a first storage operation on a first set of data in a first location to generate a plurality of data chunks, wherein said performing the first storage operation for at least one of the plurality of data chunks further comprises;

generating a second set of data associated with the first set of data,

generating a first set of metadata associated with the first set of data, and

storing the second set of data and the first set of metadata on a storage device;

storing a copy of the first set of metadata in an index remote to the storage device;

generating a second set of metadata associated with the second set of data stored on the storage device;

comparing the first set of metadata in the index with the second set of metadata to verify whether or not differences exist between the first and second sets of data without accessing the first and second sets of data;

performing a second storage operation on the second set of data, the second storage operation comprising;

generating a third set of data associated with the second set of data,

generating a third set of metadata comprising information for retrieving the third set of data, and

storing the third set of data in a second location; and

comparing the first set of metadata in the index with the third set of metadata to verify whether or not differences exist between the first and third sets of data.

2. The method of claim 1, wherein the first and second storage operations comprise a same copy operation.

3. The method of claim 1, further comprising backing up the first set of data when said comparing the first and second sets of metadata indicates that the first and second sets of data are not equivalent.

4. The method of claim 1, additionally comprising updating, after said comparing, an entry in a verification table to identify that the second set of data has been verified.

5. The method of claim 4, additionally comprising updating, based on said comparing, the verification table to indicate whether or not the second set of data is inconsistent with the first set of data.

6. The method of claim 1, wherein the second storage operation comprises an archive operation.

7. The method of claim 1, wherein the first set of data comprises a plurality of files.

8. The method of claim 1, wherein the first set of metadata comprises information usable to extract the second set of data from the plurality of data chunks.

9. The method of claim 1, wherein generating the second set of metadata further comprises performing a partial storage operation on the second set of data such that the second set of metadata is generated without generating a copy of the second set of data.

10. A system for verifying data stored in a computer network environment, the system comprising:

a first media agent configured to manage storage-related operations;

a copy manager of the first media agent configured to perform a first storage operation on a first set of data by generating at least one data chunk, the at least one data chunk comprising (i) a payload portion including a second set of data associated with the first set of data and (ii) a header portion including a first set of metadata, the copy manager being further configured to generate the first set of metadata as a result of the first storage operation;

an index maintained by the first media agent and configured to store a copy of the first set of metadata;

at least one storage device in communication with the first media agent and configured to receive the at least one data chunk from the first media agent, process the at least one data chunk and store the second set of data and the first set of metadata;

a storage manager configured to generate a second set of metadata associated with the second set of data stored on the at least one storage device;

an archive check module configured to compare the first set of metadata stored in the index with the second set of metadata to verify whether or not differences exist between the first and second sets of data;

a verification data structure in communication with the archive check module, the verification data structure configured to identify whether or not the second set of data has been verified; and

a second media agent configured to perform a second storage operation on the second set of data by generating a third set of data and a third set of metadata associated with the second set of data, and wherein the archive check module is configured to compare the first set of metadata stored in the index with the third set of metadata to verify whether or not differences exist between the first and third sets of data.

11. The system of claim 10, wherein the first media agent further comprises the index configured to store the copy of the first set of metadata.

12. The system of claim 10, wherein the storage manager further comprises the index configured to store the copy of the first set of metadata.

13. The system of claim 10, wherein the verification data structure identifies the differences that exist in the second set of data.

14. The system of claim 10, wherein the storage manager comprises the archive check module, and wherein the archive check module is configured to generate the second set of metadata.

15. The system of claim 10, wherein the storage manager further comprises a jobs manager configured to direct performance of the first and second storage operations.

16. The system of claim 10, wherein the storage manager is configured to generate the second set of data by performing a partial storage operation on the second set of data such that the second set of metadata is generated without generating a copy of the second set of data.

17. The system of claim 10, wherein the first set of metadata comprises data recovery metadata usable to retrieve the second set of data from the at least one data chunk.