Dividing data for multi-thread backup

• US 7,574,461 B1
• Filed: 12/28/2005
• Issued: 08/11/2009
• Est. Priority Date: 12/28/2005
• Status: Active Grant

• Current Assignee: EMC Corporation (Dell Technologies Inc.)
• Original Assignee: EMC Corporation (Dell Technologies Inc.)
• Inventors: Zajac, Suavek, Kilday, Roger W., Ozekinci, Yasemin Ugur, Owens, Ken, Barrero, Anselmo, Gardner, Brian, Armorer, Peter
• Primary Examiner(s): Breene; John E
• Assistant Examiner(s): Ly; Anh
• Application Number:
  US11/322,176
• Time in Patent Office: 1,322 Days
• Field of Search: 707200-205
• US Class Current: 1/1
• CPC Class Codes: G06F 11/1458 Management of the backup or restore process
• CPC Class Codes: G06F 11/1469 Backup restoration techniques

Abstract

Performing a backup is disclosed. One or more directories at a lowest level in a file directory associated with a storage is identified. The identified lowest level directories are divided into one or more directory groups. The backup of the directory groups is performed using parallel threads until all directory groups have been backed up.

Claims

1. A method of performing a backup, comprising:

identifying one or more directories at a lowest level in a file directory associated with a storage using a processor, wherein the storage communicates with the processor via an interface, and wherein the file directory is constrained to grow in a balanced manner;

dividing the identified lowest level directories into one or more directory groups based at least in part on a determination that the total size of the identified lowest level directories is greater than a threshold size using a processor; and

performing the backup of the directory groups using parallel threads until all directory groups have been backed up using a processor, wherein the backup is performed without traversing the entire file directory and wherein the number of parallel threads used to perform the backup is determined using one or more of the following;

a preconfigured value, a configuration value, a configuration preference, the number of lowest level directories that have been identified, and characteristics of one or more systems running the parallel threads.

2. A method as recited in claim 1, wherein the file directory includes a content storage area root directory.

3. A method as recited in claim 1, wherein the amount of processing required to backup each directory groups is similar.

4. A method as recited in claim 1, wherein each of the parallel threads is associated with a different backup data stream.

5. A method as recited in claim 1, wherein the file directory is constrained to include a maximum, a minimum, or a specified number of subdirectories in a directory of the file directory.

6. A method as recited in claim 1, wherein the file directory is constrained to include a maximum, a minimum, or a specified number of subdirectory levels.

7. A method as recited in claim 1, wherein the file directory is constrained to include a maximum, a minimum, or a specified number of files in a directory of the file directory.

8. A method as recited in claim 1, wherein the file directory is constrained to only include content files in the lowest level directory.

9. A method as recited in claim 1, wherein each of the one or more directory groups is constrained to be less than, equal to, or less than or equal to a preconfigured or an indicated maximum size per workload group.

10. A method as recited in claim 9, wherein the size is associated with a maximum data size one of the parallel threads is configured to process, and the number of files is associated with a maximum number of files one of the parallel threads is configured to process.

11. A method as recited in claim 9, wherein the size is determined at least in part using one or more of the following:

the number of the identified lowest level directories.

12. A method as recited in claim 1, wherein identifying the one or more directories at the lowest level includes identifying files and directories that have been modified or created since a previous backup.

13. A system for performing a backup, comprising:

a processor configured to

identify one or more directories at a lowest level in a file directory associated with a storage, wherein the file directory is constrained to grow in a balanced manner,

divide the identified lowest level directories into one or more directory group based at least in part on a determination that the total size of the identified lowest level directories is greater than a threshold size, and

perform the backup of the directory groups using parallel threads until all directory groups have been backed up, wherein the backup is performed without traversing the entire file directory and wherein the number of parallel threads used to perform the backup is determined using one or more of the following;

a preconfigured value, a configuration value, a configuration preference, the number of lowest level directories that have been identified, the size of the identified lowest level directories, and characteristics of the system; and

an interface coupled to the processor for communicating with the storage;

an interface coupled to the processor for communication with the storage.

14. A system as recited in claim 13, wherein the amount of processing required to backup each of the one or more directory groups is similar.

15. A system as recited in claim 13, wherein each of the one or more directory groups is constrained be less than, equal to, or less than or equal to a preconfigured or an indicated maximum size per workload group.

16. A system as recited in claim 15, wherein the size is associated with a maximum data size one of the parallel threads is configured to process, and the number of files is associated with a maximum number of files one of the parallel threads is configured to process.

17. A system as recited in claim 15, wherein the size is determined at least in part using one or more of the following:

the number of the identified lowest level directories.

18. A system as recited in claim 13, wherein the file directory includes a content storage area root directory.

19. A system as recited in claim 13, wherein each of the parallel threads is associated with a different backup data stream.

20. A system as recited in claim 13, wherein the file directory is constrained to include a maximum, a minimum, or a specified number of subdirectories in a directory of the file directory.

21. A system as recited in claim 13, wherein the file directory is constrained to include a maximum, a minimum, or a specified number of subdirectory levels.

22. A system as recited in claim 13, wherein the file directory is constrained to include a maximum, a minimum, or a specified number of files in a directory of the file directory.

23. A system as recited in claim 13, wherein the file directory is constrained to only include content files in the lowest level directory.

24. A system as recited in claim 13, wherein identifying the one or more directories at the lowest level includes identifying files and directories that have been modified or created since a previous backup.

25. A computer program product for performing a backup, the computer program product being embodied in a computer readable storage medium and comprising computer instructions for:

identifying one or more directories at a lowest level in a file directory associated with a storage, wherein the file directory is constrained to grow in a balanced manner;

dividing the identified lowest level directories into one or more directory groups, based at least in part on a determination that the total size of the identified lowest level directories is greater than a threshold size; and

performing the backup of the directory groups using parallel threads until all directory groups have been backed up, wherein the backup is performed without traversing the entire file directory and wherein the number of parallel threads used to perform the backup is determined using one or more of the following;

a preconfigured value, a configuration value, a configuration preference, the number of lowest level directories that have been identified, and characteristics of one or more systems running the parallel threads.