Multi stream deduplicated backup of collaboration server data

• US 10,635,543 B2
• Filed: 11/30/2016
• Issued: 04/28/2020
• Est. Priority Date: 12/19/2012
• Status: Active Grant

• Current Assignee: Emc IP Holding Company LLC (Dell Technologies Inc.)
• Original Assignee: Emc IP Holding Company LLC (Dell Technologies Inc.)
• Inventors: Upadhyay, Navneet, Tadahal, Manjunath
• Primary Examiner(s): Skhoun, Hicham
• Application Number:
  US15/364,836
• Publication Number:
  US 20170083407A1
• Time in Patent Office: 1,245 Days
• Field of Search: 707652
• CPC Class Codes: G06F 11/1453 using de-duplication of the data
• CPC Class Codes: G06F 11/1458 Management of the backup or restore process
• CPC Class Codes: G06F 11/1464 for networked environments
• CPC Class Codes: G06F 16/119 Details of migration of file systems migration mechanisms in storage systems G06F3/0647
• CPC Class Codes: G06F 16/172 Caching, prefetching or hoarding of files
• CPC Class Codes: G06F 16/1748 De-duplication implemented within the file system, e.g. based on file segments de-duplication techniques in storage systems for the management of data blocks G06F3/0641
• CPC Class Codes: G06F 16/27 Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
• CPC Class Codes: G06F 2201/84 Using snapshots, i.e. a logical point-in-time copy of the data
• CPC Class Codes: G06F 2201/845 Systems in which the redundancy can be transformed in increased performance

Abstract

Techniques to backup collaboration server data are disclosed. A plurality of separate threads are used to back up a prescribed number of subsets of a dataset. Each subset of the prescribed number of subsets was determined by walking an associated directory in the prescribed order at a prior time with which a prior backup is associated. A separate thread is spawned to backup transaction logs associated with the dataset. A first pass backup of the transaction logs is performed in parallel with the backup of the prescribed number of subsets.

Claims

1. A method of backing up data, comprising:

using a plurality of separate threads to back up a prescribed number of subsets of a dataset, wherein each subset of the prescribed number of subsets was determined by walking an associated directory in a prescribed order at a prior time with which a prior backup is associated;

spawning a separate thread to backup transaction logs associated with the dataset; and

performing a first pass backup of the transaction logs in parallel with the backup of the prescribed number of subsets.

2. The method of claim 1, further comprising performing a second pass backup of the transaction logs.

3. The method of claim 1, wherein the plurality of separate threads are configured to provide data included in a corresponding subset to a corresponding de-duplicating backup process instance that is configured to perform de-duplication processing with respect to the corresponding subset and a previous corresponding subset associated with a prior backup.

4. The method of claim 3, wherein each de-duplicating instance uses a corresponding local cache located at a collaboration server.

5. The method of claim 1, wherein the prescribed number of subsets corresponds to a configured maximum number of concurrent threads used to back up the subsets in parallel.

6. The method of claim 1, further comprising generating for each subset a corresponding list of files.

7. The method of claim 6, further comprising streaming each corresponding list of files to a corresponding thread.

8. The method of claim 1, further comprising:

determining one or more changes made to the prescribed number of subsets while the first pass backup of the transaction logs is performed in parallel with the backup of the prescribed number of subsets; and

performing a second pass backup of the transaction logs by backing up the one or more changes.

18. The method of claim 1, further comprising performing a second pass backup of the transaction logs.

19. The method of claim 1, wherein the plurality of separate threads are configured to provide data included in a corresponding subset to a corresponding de-duplicating backup process instance that is configured to perform de-duplication processing with respect to the corresponding subset and a previous corresponding subset associated with a prior backup.

20. The method of claim 3, wherein each de-duplicating instance uses a corresponding local cache located at a collaboration server.

9. A system for backing up data, comprising:

a processor configured to;

use a plurality of separate threads to back up a prescribed number of subsets of a dataset, wherein each subset of the prescribed number of subsets was determined by walking an associated directory in a prescribed order at a prior time with which a prior backup is associated;

spawn a separate thread to backup transaction logs associated with the dataset; and

perform a first pass backup of the transaction logs in parallel with the backup of the prescribed number of subsets.

10. The system of claim 9, wherein the processor is further configured to perform a second pass backup of the transaction logs.

11. The system of claim 9, further comprising performing a second pass backup of the transaction logs.

12. The system of claim 9, wherein the plurality of separate threads are configured to provide data included in a corresponding subset to a corresponding de-duplicating backup process instance that is configured to perform de-duplication processing with respect to the corresponding subset and a previous corresponding subset associated with a prior backup.

13. The system of claim 12, wherein each de-duplicating instance uses a corresponding local cache located at a collaboration server.

14. The system of claim 9, wherein the prescribed number of subsets corresponds to a configured maximum number of concurrent threads used to back up the subsets in parallel.

15. The system of claim 9, further comprising generating for each subset a corresponding list of files.

16. The system of claim 15, further comprising streaming each corresponding list of files to a corresponding thread.

17. A computer program product to back up data, the computer program product being embodied in a non-transitory computer readable storage medium and comprising computer instructions for:

using a plurality of separate threads to back up a prescribed number of subsets of a dataset, wherein each subset of the prescribed number of subsets was determined by walking an associated directory in a prescribed order at a prior time with which a prior backup is associated;

spawning a separate thread to backup transaction logs associated with the dataset; and

performing a first pass backup of the transaction logs in parallel with the backup of the prescribed number of subsets.