Data deduplication using chunk files

• US 9,684,569 B2
• Filed: 03/30/2015
• Issued: 06/20/2017
• Est. Priority Date: 03/30/2015
• Status: Active Grant

• Current Assignee: JP Morgan Chase Bank, N.A. (JP Morgan Chase & Co)
• Original Assignee: Western Digital Technologies Incorporated (Western Digital Corporation)
• Inventors: Dolan, Michael A., Kochan, Linh, Rohr, Sean, Tu, Kent, Miller, Shawn, Ram, Tamir
• Primary Examiner(s): NGUYEN, THAN VINH
• Application Number:
  US14/673,785
• Publication Number:
  US 20160292048A1
• Time in Patent Office: 813 Days
• Field of Search: 711103, 711162
• CPC Class Codes: G06F 11/1451 by selection of backup contents
• CPC Class Codes: G06F 11/1453 using de-duplication of the data
• CPC Class Codes: G06F 11/1458 Management of the backup or restore process
• CPC Class Codes: G06F 11/1469 Backup restoration techniques
• CPC Class Codes: G06F 2201/80 Database-specific techniques
• CPC Class Codes: G06F 2201/84 Using snapshots, i.e. a logical point-in-time copy of the data
• CPC Class Codes: G06F 3/0608 Saving storage space on storage systems
• CPC Class Codes: G06F 3/0619 in relation to data integrity, e.g. data losses, bit errors
• CPC Class Codes: G06F 3/0641 De-duplication techniques
• CPC Class Codes: G06F 3/065 Replication mechanisms
• CPC Class Codes: G06F 3/0683 Plurality of storage devices
• CPC Class Codes: G06F 3/0688 Non-volatile semiconductor memory arrays

Abstract

Systems and methods are disclosed for performing file backup and restore in a computing system. Certain embodiments provide a communication interface for communicating with a non-volatile memory and a controller configured to determine that a file stored in the non-volatile memory has been modified, identify a chunk of the file that has been modified, determine a new chunk associated with the modified chunk, the new chunk reflecting the modification, generate a separate chunk file including the new chunk and a filename, and storing the chunk file in the non-volatile memory using the communication interface.

Claims

1. A computing system comprising:

a communication interface for communicating with a non-volatile memory; and

a controller configured to;

determine that a file stored in the non-volatile memory has been modified, the file comprising a plurality of chunks;

identify a chunk of the plurality of chunks that has been modified, the plurality of chunks consisting of the identified modified chunk and a plurality of other chunks;

determine a new chunk associated with the modified chunk, the new chunk reflecting the modification;

generate a stand-alone chunk file including the new chunk and a filename; and

store the chunk file in the non-volatile memory separate from the plurality of other chunks using the communication interface.

2. The computing system of claim 1, wherein the filename indicates a storage location within a file directory associated with the non-volatile memory, wherein said storing the chunk file comprises storing the chunk file in the location.

3. The computing system of claim 2, wherein no path value for the location is stored by the controller separately from the filename.

4. The computing system of claim 1, wherein:

the controller is a component of a host device connected to a data storage device over the communication interface;

the data storage device includes the non-volatile memory.

5. The computing system of claim 1, wherein the non-volatile memory and the controller are components of a network-attached storage (NAS) drive or a direct-attached storage (DAS) drive.

6. The computing system of claim 1, wherein the controller is further configured to generate a hash value associated with the new chunk.

7. The computing system of claim 6, wherein the hash value is the filename.

8. The computing system of claim 1, wherein the chunk file is a hash file.

9. The computing system of claim 1, wherein the controller is further configured to reconstruct the file at least in part by:

receiving a request to restore a version of the file;

identifying one or more chunk files associated with the version of the file, the one or more chunk files including the chunk file;

retrieving the one or more identified chunk files; and

appending the retrieved one or more chunk files to a restored file.

10. The computing system of claim 9, wherein said retrieving the one or more identified chunk files comprises determining one or more directory locations based on filenames of the one or more identified chunk files.

11. The computing system of claim 9, wherein the controller is further configured to retrieve a table associating files with chunks.

12. The computing system of claim 9, wherein the controller is further configured to provide the restored file to a host device over a network, wherein the computing system is a backup server system.

13. A method of backing up data in a computing system, the method comprising:

determining that a file stored in non-volatile memory of a computing system has been modified, the file comprising a plurality of chunks;

identifying a chunk of the plurality of chunks that has been modified, the plurality of chunks consisting of the identified modified chunk and a plurality of other chunks;

determining a new chunk associated with the modified chunk, the new chunk reflecting the modification;

generating a stand-alone chunk file including the new chunk and a filename; and

storing the chunk file in the non-volatile memory separate from the plurality of other chunks.

14. The method of claim 13, wherein the filename indicates a storage location within a file directory associated with the non-volatile memory.

15. The method of claim 13, further comprising generating a hash value associated with the new chunk.

16. The method of claim 13, wherein the chunk file is a hash file.

17. A method of restoring backed-up data in a computing system, the method comprising:

receiving a request to restore a version of a file stored in non-volatile memory of a computing system;

identifying one or more stand-alone chunk files stored in the non-volatile memory associated with the version of the file, each of the one or more chunk files consisting of a single respective chunk and metadata associated therewith, the metadata including a filename;

retrieving the one or more identified chunk files; and

appending the retrieved one or more chunk files to a restored file.

18. The method of claim 17, wherein said retrieving the one or more identified chunk files comprises determining one or more directory locations based on filenames of the one or more identified chunk files.

19. The method of claim 17, further comprising retrieving a table associating files with chunks.

20. The method of claim 17, wherein the filename of each of the one or more chunk files is a hash value associated with a respective chunk file.