Systems and methods of unified reconstruction in storage systems

• US 9,996,430 B2
• Filed: 03/09/2017
• Issued: 06/12/2018
• Est. Priority Date: 12/19/2005
• Status: Active Grant

• Current Assignee: CommVault Systems Incorporated
• Original Assignee: CommVault Systems Incorporated
• Inventors: Gokhale, Parag
• Primary Examiner(s): Keehn, Richard G
• Application Number:
  US15/454,903
• Publication Number:
  US 20170177451A1
• Time in Patent Office: 460 Days
• CPC Class Codes: G06F 11/1435 using file system or storage system metadata
• CPC Class Codes: G06F 11/1464 for networked environments
• CPC Class Codes: G06F 11/1469 Backup restoration techniques
• CPC Class Codes: G06F 16/182 Distributed file systems
• CPC Class Codes: G06F 16/2365 Ensuring data consistency and integrity
• CPC Class Codes: G06F 16/25 Integrating or interfacing systems involving database management systems
• CPC Class Codes: G06F 16/27 Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
• CPC Class Codes: G06F 16/273 Asynchronous replication or reconciliation
• CPC Class Codes: G06F 16/907 Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
• CPC Class Codes: G06F 2201/84 Using snapshots, i.e. a logical point-in-time copy of the data

Abstract

Systems and methods for reconstructing unified data in an electronic storage network are provided which may include the identification and use of metadata stored centrally within the system. The metadata may be generated by a group of storage operation cells during storage operations within the network. The unified metadata is used to reconstruct data throughout the storage operation cells that may be missing, deleted or corrupt.

Claims

1. An apparatus that reconstructs missing metadata in an electronic storage network comprising:

primary data stored on at least a first storage device;

at least a secondary copy of the primary data stored on at least a second storage device;

a first set of metadata associated with a first portion of the secondary copy;

a database of metadata that stores a copy of the first set of metadata, the database of metadata stored in a different location than the first set of metadata; and

at least one storage operation controller comprising computer software executing on one or more computer processors, the storage operation controller configured to;

electronically comparing the first set of metadata with the first portion of the secondary copy to determine whether a portion of the first set of metadata is missing by identifying at least a file reference number in the first portion of the secondary copy that has no corresponding metadata in the first set of metadata; and

automatically reconstruct a missing portion of the first set of metadata using the copy of the first set of metadata stored in the database of metadata.

2. The apparatus of claim 1 wherein the storage operation controller determines that the missing portion of the first set of metadata is stored in the database of metadata.

3. The apparatus of claim 1 wherein the storage operation controller determines that the missing portion stored in the database of metadata is complete.

4. The apparatus of claim 1 wherein the storage operation controller determines whether the missing portion may be reconstructed based at least in part whether metadata associated with one or more storage operations has been identified.

5. The apparatus of claim 1 wherein the storage operation controller determines that the missing portion of the first set of metadata exists in the metadata database based on a sequence number.

6. The apparatus of claim 1 wherein the storage operation controller determines that the missing portion of the first set of metadata exists in the database of metadata based on an identifier associated with the first storage device.

7. The apparatus of claim 1 wherein the database of metadata further comprises a copy of a second set of metadata associated with a second portion of the secondary copy and the storage operation controller reconstructs a deleted portion of a second set of metadata using the copy of the second set of metadata stored in the database of.

8. The apparatus of claim 7 wherein the copies of the first and second set of metadata stored in the database of metadata are stored in a third storage device.

9. A method for reconstructing missing metadata in an electronic storage network, comprising:

copying primary data stored on at least a first storage device to at least a second storage device to create at least a secondary copy of the the primary data on the second storage device;

creating a first set of metadata associated with the secondary copy;

storing a copy of the first set of metadata in database of metadata wherein the database of metadata is stored in a different location than the first set of metadata; and

determining whether a portion of the first set of metadata is missing by identifying at least a file reference number in the first portion of the secondary copy that has no corresponding metadata in the first set of metadata; and

reconstructing a missing portion of the first set of metadata using the copy of the first set of metadata stored in the database of metadata.

10. The method of claim 9 further comprising determining that the missing portion of the first set of metadata is stored in the database of metadata.

11. The method of claim 9 further comprising analyzing the missing portion of the first set of metadata stored in the database of metadata to determine that the missing portion of the first set of metadata stored in the database of metadata is complete.

12. The method of claim 9 further comprising determining whether the missing portion may be reconstructed based at least in part whether metadata associated with one or more storage operations has been identified.

13. The method of claim 9 further comprising determining that a missing portion of the first set of metadata exists in the database of metadata storage device based on a sequence number.

14. The method of claim 9 further comprising creating a second set of metadata associated with a second portion of the secondary copy and storing a copy of the second set of metadata in the database of metadata, and reconstructing a loss of a portion of the second set of metadata using the copy of the second set of metadata stored in the database of metadata.

15. The method of claim 9 further comprising determining that a missing portion of the first set of metadata exists in the database of metadata based on an identifier associated with the first storage device.

16. The method of claim 9 further comprising storing copies of the first and second set of metadata stored in the database of metadata are copied to a third storage device.