Systems and methods of unified reconstruction in storage systems

• US 20070226535A1
• Filed: 12/15/2006
• Published: 09/27/2007
• Est. Priority Date: 12/19/2005
• Status: Active Grant

• Current Assignee: CommVault Systems Incorporated
• Original Assignee: CommVault Systems Incorporated
• Inventors: Gokhale, Parag
• Granted Patent: US 8,930,496 B2
• Time in Patent Office: Days
• US Class Current: 714/6
• CPC Class Codes: G06F 11/1435 using file system or storage system metadata
• CPC Class Codes: G06F 11/1464 for networked environments
• CPC Class Codes: G06F 11/1469 Backup restoration techniques
• CPC Class Codes: G06F 16/182 Distributed file systems
• CPC Class Codes: G06F 16/2365 Ensuring data consistency and integrity
• CPC Class Codes: G06F 16/25 Integrating or interfacing systems involving database management systems
• CPC Class Codes: G06F 16/27 Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
• CPC Class Codes: G06F 16/273 Asynchronous replication or reconciliation
• CPC Class Codes: G06F 16/907 Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
• CPC Class Codes: G06F 2201/84 Using snapshots, i.e. a logical point-in-time copy of the data

Abstract

Systems and methods for reconstructing unified data in an electronic storage network are provided which may include the identification and use of metadata stored centrally within the system. The metadata may be generated by a group of storage operation cells during storage operations within the network. The unified metadata is used to reconstruct data throughout the storage operation cells that may be missing, deleted or corrupt.

Claims

1. A system for maintaining metadata in an electronic storage network comprising:

a plurality of networks having a storage operation capability and which include client applications, wherein at least one of the plurality of networks includes a storage management component and wherein the plurality of networks are electronically linked to each other; and

a storage device associated with the storage manager for maintaining the metadata, wherein the plurality of networks and client applications generate metadata relating to the client applications and system operation, the storage management component collecting the metadata from the plurality of storage operation cells and storing the data on the storage device.

2. The system of claim 1 wherein the storage management component identifies, collects, and stores metadata relating to system operation.

3. The system of claim 1 wherein the storage management component identifies, collects, and stores metadata relating to the operation of one or more client applications.

4. The system of claim 1 wherein the storage management component detects an interruption of metadata transfer from a first system component to a second system component.

5. The system of claim 4 wherein the storage management component compares the metadata in the first system component with metadata in the second system component to determine differences, if any.

6. The system of claim 5 wherein the storage management component determines whether metadata from the first system component can be transferred to the second system such that the first and second system components have substantially the same metadata if differences are determined.

7. The system of claim 6 wherein the storage management component coordinates transfer metadata from the first system component to the second system component if the first system component has metadata necessary to make the metadata in the second system component substantially the same the metadata in the first system component.

8. The system of claim 6 wherein the storage management component searches the storage network for the metadata missing from the second system component if the storage management component determines that the missing metadata is not available from the first system component.

9. The system of claim 6 wherein the storage management component searches other components within a storage operation cell.

10. The system of claim 6 wherein the storage management component searches other storage operation cells.

11. The system of claim 6 wherein the storage management component searches other storage management components in other storage operation cells.

12. The system of claim 8 wherein the storage management component identifies portions of the missing metadata in different locations.

13. The system of claim 12 wherein the storage management component obtains copies the identified missing portions from the different locations to the storage device.

14. The system of claim 13 wherein the storage management component reconstructs the missing metadata from the identified missing portions.

15. The system of claim 14 wherein the storage management component reconstructs the missing metadata by integrating the identified missing portions into a metadata arrangement similar to or the same as an original arrangement of the missing metadata.

16. The system of claim 15 wherein the identified missing portions into a metadata are arranged based temporal criteria.

17. The system of claim 13 wherein the storage management component analyzes identified missing portions to determine whether the missing metadata may be reconstructed.

18. The system of claim 17 wherein the storage management component performs additional network searches if the identified missing portions of metadata are insufficient to reconstruct the missing metadata.

19. The system of claim 15 wherein the storage management component reconstructs the missing metadata on the storage devices and copies the reconstructed missing metadata from the storage device to the second system component.

20. The system of claim 8 wherein the missing metadata is not available from the first system component because the missing metadata is corrupted, has been deleted or is unavailable due to hardware or software failure.

21. A method for maintaining and restoring metadata in an electronic storage network comprising:

managing a storage operation network including client applications with a storage management component component; and

maintaining metadata associated with the storage management component and client applications that relate to the client applications and system operation, the storage management component collecting the metadata from a plurality of storage operation cells in the storage operation network and storing the data on the storage device.

22. The method of claim 21 further comprising identifying, collecting, and storing metadata relating to system operation.

23. The method of claim 21 further comprising identifying, collecting, and storing metadata relating to the operation of one or more client applications.

24. The method of claim 21 further comprising detecting an interruption of metadata transfer from a first system component to a second system component.

25. The method of claim 24 further comprising compares the metadata in the first system component with metadata in the second system component to determine differences, if any.

26. The method of claim 25 further comprising determining whether metadata from the first system component can be transferred to the second system such that the first and second system components have substantially the same metadata if differences are determined.

27. The method of claim 26 further comprising coordinating transfer metadata from the first system component to the second system component if the first system component has metadata necessary to make the metadata in the second system component substantially the same the metadata in the first system component.

28. The method of claim 26 further comprising searching the storage network for the metadata missing from the second system component if the storage management component determines that the missing metadata is not available from the first system component.

29. The method of claim 28 wherein the searching further comprises searching other components within a storage operation cell.

30. The method of claim 28 wherein the searching further comprises searching other storage operation cells.

31. The method of claim 28 wherein the searching further comprises searching other storage management components in other storage operation cells.

32. The method of claim 28 further comprising identifying portions of the missing metadata in different locations.

33. The method of claim 32 further comprising obtaining copies of the identified missing portions from the different locations to the storage device.

34. The method of claim 33 further comprising reconstructing the missing metadata from the identified missing portions.

35. The method of claim 34 wherein the reconstructing further comprises integrating the identified missing portions into a metadata arrangement similar to or the same as an original arrangement of the missing metadata.

36. The method of claim 35 further comprising arranging the identified missing portions into a metadata on a temporal basis.

37. The method of claim 33 further comprising analyzing identified missing portions to determine whether the missing metadata may be reconstructed.

38. The method of claim 37 further comprising performing additional network searches if the identified missing portions of metadata are insufficient to reconstruct the missing metadata.

39. The method of claim 35 further comprising:

reconstructing the missing metadata on the storage devices; and

copying the reconstructed missing metadata from the storage device to the second system component.

40. The method of claim 28 wherein the missing metadata is not available from the first network component because the missing metadata is corrupted, has been deleted or is unavailable due to hardware or software failure.

41. A computer-readable medium having stored thereon a plurality of sequences of instructions which, when executed by one or more processors cause an electronic device to:

assign a unique identifier to a set of metadata, the metadata generated by a storage operation cell, the storage operation cell having a local storage device;

determine whether a backup storage device includes a hardware identifier;

add the hardware identifier data to the unique identifier;

send the set of metadata to a central storage device;

detect a loss of metadata on the local storage device;

locate a corresponding set of metadata on the central storage device using the unique identifier;

reconstruct the lost metadata on the local storage device using a corresponding set of metadata on the central storage device; and

copy the reconstructed metadata to the local storage device.

42. A computer-readable medium having stored thereon a plurality of sequences of instructions which, when executed by one or more processors cause an electronic device to:

manage a storage operation network including client applications with a storage management component component; and

maintain metadata associated with the storage management component and client applications that relate to the client applications and system operation, the storage management component collecting the metadata from a plurality of storage operation cells in the storage operation network and storing the data on the storage device.