Multi stream deduplicated backup of collaboration server data

• US 9,542,276 B2
• Filed: 09/18/2015
• Issued: 01/10/2017
• Est. Priority Date: 12/19/2012
• Status: Active Grant

• Current Assignee: Emc IP Holding Company LLC (Dell Technologies Inc.)
• Original Assignee: EMC Corporation (Dell Technologies Inc.)
• Inventors: Upadhyay, Navneet, Tadahal, Manjunath
• Primary Examiner(s): Waldron, Scott A
• Assistant Examiner(s): Skhoun, Hicham
• Application Number:
  US14/859,152
• Publication Number:
  US 20160011945A1
• Time in Patent Office: 480 Days
• Field of Search: 707/652
• US Class Current: 1/1
• CPC Class Codes: G06F 11/1453 using de-duplication of the data
• CPC Class Codes: G06F 11/1458 Management of the backup or restore process
• CPC Class Codes: G06F 11/1464 for networked environments
• CPC Class Codes: G06F 16/119 Details of migration of file systems migration mechanisms in storage systems G06F3/0647
• CPC Class Codes: G06F 16/172 Caching, prefetching or hoarding of files
• CPC Class Codes: G06F 16/1748 De-duplication implemented within the file system, e.g. based on file segments de-duplication techniques in storage systems for the management of data blocks G06F3/0641
• CPC Class Codes: G06F 16/27 Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
• CPC Class Codes: G06F 2201/84 Using snapshots, i.e. a logical point-in-time copy of the data
• CPC Class Codes: G06F 2201/845 Systems in which the redundancy can be transformed in increased performance

Abstract

Techniques to backup collaboration server data are disclosed. An indication to begin backup of a collaboration server dataset is received. An associated directory is walked in a prescribed order to divide the dataset into a prescribe number of approximately equal-sized subsets. A separate subset-specific thread is used to back up the subsets in parallel. In some embodiments in which the collaboration data is stored in multiple volumes, a volume-based approach is used to back up the volumes in parallel, e.g., one volume per thread. In some embodiments, transaction logs are backed up in parallel with volumes of collaboration data.

Claims

1. A method of backing up data, comprising:

receiving an indication associated with a backup of a collaboration server dataset;

dividing the dataset into a prescribed number of subsets; and

using a plurality of separate threads to back up the subsets in parallel, wherein the plurality of threads are configured to provide data included in the corresponding subset to a corresponding de-duplicating backup process instance that is configured to perform de-duplication processing with respect to the subset and a corresponding subset associated with a prior backup, wherein the corresponding subset was determined by walking an associated directory in the prescribed order at a prior time with which the prior backup is associated.

2. The method of claim 1, wherein the corresponding subset was determined based at least in part on a prescribed order according to which the dataset is divided.

3. The method of claim 1, wherein the associated directory is walked in the same prescribed order in a subsequent backup operation.

4. The method of claim 1, wherein each de-duplicating backup process instance uses a corresponding local cache located at the collaboration server.

5. The method of claim 4, wherein each de-duplicating backup process instance determines whether the corresponding local cache stores a file included in the subset corresponding to the thread.

6. The method of claim 1, wherein the prescribed number of subsets corresponds to a configured maximum number of concurrent threads used to back up the subsets in parallel.

7. The method claim 1, further comprising generating for each subset a corresponding list of files.

8. The method of claim 7, further comprising streaming each corresponding list of files to a corresponding thread.

9. The method of claim 1, wherein the de-duplicating backup process instances corresponding to the plurality of threads have associated therewith a corresponding deduplicating backup process instance-specific local cache.

10. The method of claim 9, wherein the de-duplicating backup process instances corresponding to the plurality of threads are configured to store in the associated corresponding de-duplicating backup process instance-specific local cache a hash value representative of data that has been transferred by the de-duplicating backup process instance to a remote backup server.

11. A system to back up collaboration server data, comprising:

a memory of other storage device configured to store a directory associated with a collaboration server dataset; and

a processor configured to;

receive an indication associated with a backup of a collaboration server dataset;

divide the dataset into a prescribed number of subsets; and

user a plurality of separate threads to back up the subsets in parallel, wherein the plurality of threads are configured to provide data included in the corresponding subset to a corresponding de-duplicating backup process instance that is configured to perform de-duplication processing with respect to the subset and a corresponding subset associated with a prior backup, wherein the corresponding subset was determined by walking an associated directory in the prescribed order at a prior time with which the prior backup is associated.

12. The system of claim 11, wherein the corresponding subset was determined based at least in part on a prescribed order according to which the dataset is divided.

13. The system of claim 11, wherein the associated directory is walked in the same prescribed order in a subsequent backup operation.

14. The system of claim 11, wherein each de-duplicating backup process instance uses a corresponding local cache located at the collaboration server.

15. The system of claim 14, wherein each de-duplicating backup process instance determines whether the corresponding local cache stores a file included in the subset corresponding to the thread.

16. The system of claim 11, wherein the prescribed number of subsets corresponds to a configured a maximum number of concurrent threads used to back up the subsets in parallel.

17. The system of claim 11, wherein the processor is further configured to generate for each subset a corresponding list of files and to stream each corresponding list of files to a corresponding thread with which that list of files is associated.

18. A computer program product to back up data, the computer program product being embodied in a non-transitory computer readable storage medium and comprising computer instructions for:

receiving an indication associated with a backup of a collaboration server dataset;

dividing the dataset into a prescribed number of subsets; and

using a plurality of separate threads to back up the subsets in parallel, wherein the plurality of threads are configured to provide data included in the corresponding subset to a corresponding de-duplicating backup process instance that is configured to perform de-duplication processing with respect to the subset and a corresponding subset associated with a prior backup, wherein the corresponding subset was determined by walking an associated directory in the prescribed order at a prior time with which the prior backup is associated.