BACKUP USING A CLIENT-SIDE SIGNATURE REPOSITORY IN A NETWORKED STORAGE SYSTEM

• US 20130339299A1
• Filed: 06/12/2013
• Published: 12/19/2013
• Est. Priority Date: 06/13/2012
• Status: Active Grant

• Current Assignee: CommVault Systems Incorporated
• Original Assignee: CommVault Systems Incorporated
• Inventors: Muller, Marcus S., Ngo, David
• Granted Patent: US 9,251,186 B2
• Time in Patent Office: Days
• US Class Current: 707/640
• CPC Class Codes: G06F 11/1415 at system level
• CPC Class Codes: G06F 11/1448 Management of the data involved in backup or backup restore
• CPC Class Codes: G06F 11/1451 by selection of backup contents
• CPC Class Codes: G06F 11/1453 using de-duplication of the data
• CPC Class Codes: G06F 11/1458 Management of the backup or restore process
• CPC Class Codes: G06F 11/1464 for networked environments
• CPC Class Codes: G06F 16/119 Details of migration of file systems migration mechanisms in storage systems G06F3/0647
• CPC Class Codes: G06F 16/122 using management policies backup systems G06F11/1446; file migration policies for HSM systems G06F16/185
• CPC Class Codes: G06F 16/137 Hash-based content-based indexing of textual data G06F16/31
• CPC Class Codes: G06F 16/16 File or folder operations, e.g. details of user interfaces specifically adapted to file systems
• CPC Class Codes: G06F 16/1748 De-duplication implemented within the file system, e.g. based on file segments de-duplication techniques in storage systems for the management of data blocks G06F3/0641
• CPC Class Codes: G06F 16/1752 based on file chunks
• CPC Class Codes: G06F 16/176 Support for shared access to files; File sharing support
• CPC Class Codes: G06F 16/1847 specifically adapted to static storage, e.g. adapted to flash memory or SSD
• CPC Class Codes: G06F 16/185 Hierarchical storage management [HSM] systems, e.g. file migration or policies thereof details of archiving G06F16/11
• CPC Class Codes: G06F 16/217 Database tuning G06F16/2282 takes precedence; database performance monitoring G06F11/3409
• CPC Class Codes: G06F 16/22 Indexing; Data structures therefor; Storage structures
• CPC Class Codes: G06F 16/2379 Updates performed during online database operations; commit processing
• CPC Class Codes: G06F 16/24 Querying
• CPC Class Codes: G06F 16/27 Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
• CPC Class Codes: G06F 16/275 : Synchronous replication
• CPC Class Codes: G06F 16/90 : Details of database functions independent of the retrieved data types
• CPC Class Codes: G06F 16/95 : Retrieval from the web
• CPC Class Codes: G06F 2201/80 : Database-specific techniques

Abstract

A storage system according to certain embodiments includes a client-side signature repository that includes information representative of a set of data blocks stored in primary storage. During copy or backup operations, the system can use the client-side signature repository to identify data blocks located in primary storage that are new or that have changed. The system can also use the client-side signature repository to identify multiple locations within primary storage where different instances of the data blocks are located.

Claims

1. A method of generating a backup data set for a client computing device by using a signature repository residing in a primary storage subsystem, the method comprising:

for each respective client computing device of one or more client computing devices in a primary storage subsystem;

monitoring the storage of a plurality of files formed by data blocks generated by one or more software applications running on the respective client computing device,

wherein the plurality of files are stored in a data store associated with the respective client computing device;

maintaining, by a repository agent executing on one or more processors in the primary storage subsystem, a repository indicating at least which data blocks of the monitored files are stored in the primary storage subsystem;

in response to instructions to create a secondary copy in a secondary storage subsystem of at least a subset of the plurality of files stored in a data store associated with a first client computing device of the one or more client computing devices,

querying the repository to identify at least a first group of data blocks that form at least a portion of the subset of files and for which matching data blocks are not stored in the secondary storage subsystem;

identifying the location of the first group of data blocks within the primary storage subsystem; and

retrieving the first group of data blocks from one or more of the data stores associated with the one or more client computing devices.

2. The method of claim 1, further comprising:

generating a copy operation index that identifies one or more data blocks that form the subset of files including the retrieved first group of data blocks; and

communicating the retrieved first group of data blocks and the copy operation index to the secondary storage subsystem over a wide-area network (WAN), wherein the secondary storage subsystem creates the backup copy using at least the retrieved first group of data blocks.

3. The method of claim 1, wherein the repository agent resides on a computing device that is separate from each of the one or more client computing devices.

4. The method of claim 1, wherein the repository is maintained in one or more storage devices that are separate from the data stores of the one or more client computing devices.

5. The method of claim 1, wherein the repository comprises a central repository including signatures which correspond to the data blocks that are stored in the data store associated with all of the one or more client computing devices, wherein the signatures are generated by a signature generator.

6. The method of claim 5, wherein the first group of data blocks is identified using the signatures in the repository.

7. The method of claim 5, wherein the repository comprises signatures corresponding to all of the data blocks that are stored in the primary storage subsystem.

8. The method of claim 5, wherein each client computing device includes a signature generator.

9. The method of claim 5, wherein the repository agent comprises the signature generator.

10. The method of claim 5, further comprising maintaining in the repository, for each data block referenced in the repository,:

a storage location for the referenced data block within the data store associated with the client computing device that stores the referenced block.

11. The method of claim 1, wherein the repository includes only metadata and does not include copies of the data blocks referenced in the repository.

12. The method of claim 1, wherein the secondary storage subsystem comprises deduplicated data.

13. The method of claim 1, wherein the data store of one or more client computing devices comprises deduplicated data.

14. A storage system for generating a backup data set for a client computing device using a signature repository, the system comprising:

a repository indicating which data blocks are stored in a primary storage subsystem, the primary storage subsystem including one or more client computing devices each having an associated data store, wherein the data blocks are generated by one or more software applications running on the one or more client computing devices,

a repository agent executing on one or more processors and configured to;

maintain the repository;

in response to instructions to create a secondary copy in a secondary storage subsystem of at least a subset of the plurality of files stored in the data store associated with a first client computing device of the one or more client computing devices,

query the repository to identify at least a first group of data blocks that form at least a portion of the subset of files and for which matching data blocks are not stored in the secondary storage; and

provide retrieval information usable to locate the first group of data blocks within the primary storage subsystem.

15. The system of claim 14, wherein the repository agent is further configured to

generate a copy operation index that identifies one or more data blocks that form the subset of files including the retrieved first group of data blocks;

and

communicate the retrieved first group of data blocks and the copy operation index to the secondary storage subsystem which creates the backup copy using at least the retrieved first group of data blocks.

16. The system of claim 14, wherein the repository agent resides on a computing device that is separate from each of the one or more client computing devices.

17. The system of claim 14, wherein the repository is maintained in one or more storage devices that are separate from the data stores of the one or more client computing devices.

18. The system of claim 14, wherein the repository comprises a central repository including signatures which correspond to the data blocks that are stored in the data store associated with all of the one or more client computing devices, wherein the signatures are generated by a signature generator.

19. A computer-readable, non-transitory storage medium having one or more computer-executable modules for generating a backup data set for a client computing device, the one or more computer-executable modules comprising:

a first module in communication with one or more client computing devices and configured to;

maintain a repository indicating data blocks that are stored in a primary storage subsystem, the primary storage subsystem comprising one or more client computing devices each having an associated data store, wherein the data blocks are generated by one or more software applications running on the one or more client computing devices;

in response to instructions to create a secondary copy in a secondary storage subsystem of at least a subset of the plurality of files stored in the data store associated with a first client computing device of the plurality of client computing devices,

query the repository to identify at least a first group of data blocks that form at least a portion of the subset of files and for which matching data blocks are not stored in the secondary storage subsystem;

provide retrieval information usable to locate the first group of data blocks within the primary storage subsystem.

20. The storage medium of claim 25, wherein the first module is further configured to generate a copy operation index that identifies one or more data blocks that form the subset of files including the retrieved first group of data blocks.