Backup and restoration for a deduplicated file system

• US 9,959,275 B2
• Filed: 11/30/2016
• Issued: 05/01/2018
• Est. Priority Date: 12/28/2012
• Status: Active Grant

• Current Assignee: CommVault Systems Incorporated
• Original Assignee: CommVault Systems Incorporated
• Inventors: Vijayan, Manoj Kumar
• Primary Examiner(s): Mobin, Hasanul
• Application Number:
  US15/364,954
• Publication Number:
  US 20170083408A1
• Time in Patent Office: 517 Days
• CPC Class Codes: G06F 11/1451 by selection of backup contents
• CPC Class Codes: G06F 11/1453 using de-duplication of the data
• CPC Class Codes: G06F 11/1469 Backup restoration techniques
• CPC Class Codes: G06F 16/113 Details of archiving lifecycle management in storage systems G06F3/0649; backup systems G06F11/1446
• CPC Class Codes: G06F 16/1748 De-duplication implemented within the file system, e.g. based on file segments de-duplication techniques in storage systems for the management of data blocks G06F3/0641
• CPC Class Codes: G06F 16/1752 based on file chunks

Abstract

The disclosed techniques that can use deduplication information on a source computer platform to improve the process of performing data backups or restoration from/to the computer platform. In one example aspect, a data backup operation can re-use some of the work already done by a source computer'"'"'s deduplication system. For example, a storage operation could read a deduplication database on the source computer platform to determine the duplicativeness of a given data chunk being transferred to a backup storage system, without having to perform computations such as data chunk hashing and comparison with previously generated hashes. The technique may additionally or alternatively reuse hashes generated by the source computer during deduplication of the data file on the source computer'"'"'s file system during deduplication at the external backup storage system.

Claims

1. A method of backing up data from a source file system of a computer device to a backup data storage system, the method comprising:

checking whether a source file is locally stored on the source file system in a local deduplicated format of the source file system;

when the checking indicates that the source file is stored in the local deduplicated format, then

determining a block size value used to store the source file in the deduplicated format;

accessing, without a file read/write assistance from an operating system running on the computer device, a local deduplication database to determine a location of a first data chunk of the source file stored in the deduplicated format on a local storage device; and

,

backing up the source file by accessing and selectively transferring the first data chunk and successive data chunks of the source file by;

transferring a given data chunk, if the local deduplication database indicates that the given data chunk was not deduplicated; and

transferring a deduplication record, without transferring the given data chunk, if the local deduplication database indicates that the given data chunk was deduplicated; and

when the check indicates that the source file is locally stored without deduplication on the computer device, then backing up the source file by transferring data chunks of the source file to the backup data storage system and performing deduplication on the data chunks of the source file.

2. The method of claim 1, further comprising:

updating a backup transaction log at the backup storage system with a first entry type when the given data chunk is transferred, and with a second entry type when the deduplication record is transferred instead of the given data chunk

wherein multiple files of the source file system are backed up using the method.

3. The method recited in claim 2, wherein, when at least one file in a source directory of the source file system is determined to be locally stored in the deduplicated format, then it is determined that all remaining files in the source directory are also stored in the deduplicated format.

4. The method recited in claim 2, where, when at least one file in the source directory of the source file system is determined to be locally stored in the deduplicated format, then it is determined that all remaining files in child directories under the source directory are also stored in the deduplicated format.

5. The method of claim 1 wherein the determining whether the source file is locally stored on the source file system in the deduplicated format is performed without assistance from the operating system.

6. The method of claim 1 further comprising:

when it is determined that the source file is locally stored in a deduplicated format, transferring hash values corresponding to the source file in the deduplicated format to the backup storage system.

7. The method of claim 1, wherein different hash functions are used for deduplication at the source file system and the backup data storage system, and wherein the determining the block size value is performed without assistance from the operating system.

8. A computing system for backing up data from a source file system of a computer device to a backup data storage system, the system comprising:

at least one processor;

memory coupled to the at least one processor, wherein the memory stores contents that, when executed by the at least one processor performs a method of;

determining a list of source files to be backed up in a source directory;

for each source file on the list of source files to be backed up;

checking whether the source file is locally stored on the source file system in a deduplicated format,

wherein at least some file on the source file system are deduplicated by a deduplication module of the source file system; and

when the checking indicates that the source file is stored in the deduplicated format, then

accessing a local deduplication database to determine locations of data chunks of the source file stored in the deduplicated format on a local storage device;

backing up the source file to the backup data storage system by;

 

transferring a given data chunk of the source file to the backup data storage system, if the local deduplication database indicates that the given data chunk was not deduplicated; and

 

transferring a deduplication record, without transferring the given data chunk, if the local deduplication database indicates that the given data chunk was deduplicated.

9. The system recited in claim 8, wherein, when at least one source file in the source directory of the source file system is determined to be locally stored in the deduplicated format, then it is determined that all remaining files in the source directory are also stored in the deduplicated format.

10. The system recited in claim 8, where, when at least one source file in the source directory of the source file system is determined to be locally stored in the deduplicated format, then it is determined that all remaining files in child directories under the source directory are also stored in the deduplicated format.

11. The system of claim 8 wherein the determining whether the source file is locally stored on the source file system in the deduplicated format is performed without assistance from an operating system running on the computer device.

12. The system of claim 8 further comprising:

when it is determined that the source file is locally stored in a deduplicated format, transferring hash values corresponding to the source file in the deduplicated format to the backup storage system, and wherein different hash functions are used for deduplication at the source file system and the backup data storage system.

13. The system of claim 8 further comprising:

when it is determined that the source file is locally stored in a deduplicated format, transferring hash values corresponding to the source file in the deduplicated format to the backup storage system.

14. The system of claim 8 further comprising:

when the check indicates that the source file is locally stored on the source file system without deduplication on the computer device, then backing up the source file by transferring data chunks of the source file to the backup data storage system and performing deduplication on the data chunks of the source file.

15. A non-transitory computer-readable medium carrying instructions to perform a method in a computing system for backing up data from a source file system of a computer device to a backup data storage system, the method comprising:

checking whether a source file is locally stored on the source file system in a local deduplicated format of the source file system; and

when the checking indicates that the source file is stored in the local deduplicated format, then

determining a block size value used to store the source file in the deduplicated format;

accessing, without a file read/write assistance from an operating system running on the computer device, a local deduplication database to determine locations of data chunks of the source file stored in the deduplicated format on a local storage device; and

,

backing up the source file by accessing and selectively transferring the data chunks of the source file by;

transferring a given data chunk, if the local deduplication database indicates that the given data chunk was not deduplicated; and

transferring a deduplication record, without transferring the given data chunk, if the local deduplication database indicates that the given data chunk was deduplicated.

16. The non-transitory computer-readable medium of claim 15, wherein the method further comprises:

updating a backup transaction log at the backup storage system with a first entry type when the given data chunk is transferred, and with a second entry type when the deduplication record is transferred instead of the given data chunk

wherein multiple files of the source file system are backed up using the method.

17. The non-transitory computer-readable medium recited in claim 15, wherein, when at least one file in a source directory of the source file system is determined to be locally stored in the deduplicated format, then it is determined that all remaining files in the source directory are also stored in the deduplicated format.

18. The non-transitory computer-readable medium recited in claim 15, where, when at least one source file in the source directory of the source file system is determined to be locally stored in the deduplicated format, then it is determined that all remaining files in child directories under the source directory are also stored in the deduplicated format.

19. The non-transitory computer-readable medium of claim 15, wherein the method further comprises:

when it is determined that the source file is locally stored in a deduplicated format, transferring hash values corresponding to the source file in the deduplicated format to the backup storage system.

20. The non-transitory computer-readable medium of claim 15, wherein different hash functions are used for deduplication at the source file system and the backup data storage system, and wherein the determining the block size value is performed without assistance from the operating system.