Operating system and file system independent incremental data backup

• US 8,595,188 B2
• Filed: 11/06/2009
• Issued: 11/26/2013
• Est. Priority Date: 11/06/2009
• Status: Active Grant

• Current Assignee: Dropbox, Inc.
• Original Assignee: International Business Machines Corporation
• Inventors: Gruhl, Daniel, Pieper, Jan H., Smith, Mark Andrew
• Primary Examiner(s): Corrielus, Jean M
• Assistant Examiner(s): ASPINWALL, EVAN S
• Application Number:
  US12/614,134
• Publication Number:
  US 20110113012A1
• Time in Patent Office: 1,481 Days
• Field of Search: 707/624, 707/625, 707/640, 707/641, 707/643, 707/644, 707/646, 707/647, 707/648, 707/649, 711/154, 711/162
• US Class Current: 707/646
• CPC Class Codes: G06F 11/1451 by selection of backup contents
• CPC Class Codes: G06F 2201/815 Virtual middleware or OS functionality using virtual machines to implement generic software techniques for error detection or fault masking G06F11/1484

Abstract

Embodiments of the invention relate to creating an operating system and file system independent incremental data backup. A first data backup of a source system and second version of the data on the source system is received. A second data backup of the second version of the data is created by determining differences between the first data backup and the second version of the data. Each portion of the second version of the data that is the same as a portion of the first data backup is referenced in the second data backup. Each portion of the second version of the data that is different than all portions of the first data backup is included in the second data backup. The second data backup is appended to the first data backup to create an incremental data backup.

Claims

1. A method of creating an operating system and file system independent incremental data backup, comprising:

receiving a first data backup of a source system, the first data backup being a byte-level data backup of the source system'"'"'s storage;

receiving a second version of the source system'"'"'s data as a byte stream, the second version being a verbatim binary copy of the source system'"'"'s storage and received independent of the source system'"'"'s file system and operating system;

partitioning said second version using a chunking method selected from the group consisting of;

content-agnostic chunking, content-aware chunking, and fixed-size chunking;

creating a second data backup of the second version of the data, comprising;

determining differences between the first data backup and the second version of the data,

for a portion at a first position of the second version of the data that is the same as a portion of the first data backup, referencing the portion of the first data backup in the second data backup,

for a portion at a first position of the second version of the data that is different than all portions of the first data backup, including the portion of the second version of the data in the second data backup, and

creating reconstruction metadata for assembling an incremental data backup, of the first data backup, into a replication of the second version of the data; and

appending the second data backup to the first data backup, the appending creates a data backup incremental of the first data backup and the second data backup.

2. The method of claim 1, wherein said determining comprises comparing a byte stream of the first data backup to a byte stream of the second version of the data.

3. The method of claim 1, wherein said determining comprises comparing a signature for a potion of the second version of data to signatures for a plurality of portions of the first data backup.

4. The method of claim 1, wherein the byte stream is received without knowledge of the source system'"'"'s file system and operating system.

5. The method of claim 1, wherein creating an incremental data backup independent of the source system'"'"'s operating system and file system is done without file-level access to the source system'"'"'s storage.

6. The method of claim 3, wherein the signatures for comparison are signatures for the current version of the data and signatures for a most recently backed up version of the data.

7. The method of claim 3, wherein:

the signatures for the portions of the first data backup and the second version of the data are hashes,

the hashes for the portions of the first data backup are stored in a first hash table, and

the hashes for the portions of the second version of the data are stored in the first hash table or and a second hash table.

8. The method of claim 3, further comprising:

creating the first data backup of a first version of the data, wherein creating the first data backup of the first version of the data comprises;

comparing hashes of portions of the first version of the data to hashes of portions of previously stored data,

for portions of the first version of the data that have a same hash as portions of the previously stored data, referencing the portions of the first version of the data in the first data backup, and

for portions of the first version of the data that have a different hash than portions of the previously stored data, storing the portions of the first version of the data in the first data backup.

9. A computer program product of creating an operating system and file system independent incremental data backup, said program product comprising a non-transitory computer readable storage medium having computer usable program code embodied therewith, said program code being readable/executable by a computer to:

receive a first data backup of a source system, the first data backup being a byte-level data backup of the source system'"'"'s storage;

receive a second version of the source system'"'"'s data as a byte stream, the second version being a verbatim binary copy of the source system'"'"'s storage and received independent of the source system'"'"'s file system and operating system;

partition said second version using a chunking method selected from the group consisting of;

content-agnostic chunking, content-aware chunking, and fixed-size chunking;

create a second data backup of the second version of the data, comprising;

determine differences between the first data backup and the second version of the data,

for a portion at a first position of the second version of the data that is the same as a portion of the first data backup, reference the portion of the first data backup in the second data backup,

for a portion at a first position of the second version of the data that is different than all portions of the first data backup, include the portion of the second version of the data in the second data backup, and

create reconstruction metadata for assembling an incremental data backup, of the first data backup, into a replication of the second version of the data; and

append the second data backup to the first data backup, the append creates a data backup incremental of the first data backup and the second data backup.

10. The computer program product of claim 9, wherein said determine comprises comparing a signature for a potion of the second version of data to signatures for a plurality of portions of the first data backup.

11. The computer program product of claim 9, wherein the byte stream is received without knowledge of the source system'"'"'s file system and operating system.

12. The computer program product of claim 9, wherein creating an incremental data backup independent of the source system'"'"'s operating system and file system is done without file-level access to the source system'"'"'s storage.

13. The computer program product of claim 10, wherein:

the signatures for the portions of the first data backup and the second version of the data are hashes,

the hashes for the portions of the first data backup are stored in a first hash table, and

the hashes for the portions of the second version of the data are stored in the first hash table or and a second hash table.

14. The computer program product of claim 10, said program code being further readable/executable by a computer to:

create the first data backup of a first version of the data, including;

compare hashes of portions of the first version of the data to hashes of portions of previously stored data,

for portions of the first version of the data that have a same hash as portions of the previously stored data, reference the portions of the first version of the data in the first data backup, and

for portions of the first version of the data that have a different hash than portions of the previously stored data, store the portions of the first version of the data in the first data backup.

15. A system of creating an operating system and file system independent incremental data backup, comprising:

a memory device for buffering data to be backed up by the system;

a storage device for storing data being backed up by the system;

a processor for executing computer usable program code; and

a computer readable storage medium having computer usable program code embodied therewith, said program code being executable by the processor to;

receive a first data backup of a source system, the first data backup being a byte-level data backup of the source system'"'"'s storage,

receive a second version of the source system'"'"'s data as a byte stream, the second version being a verbatim binary copy of the source system'"'"'s storage and received independent of the source system'"'"'s file system and operating system,

partition said second version using a chunking method selected from the group consisting of;

content-agnostic chunking, content-aware chunking, and fixed-size chunking

create a second data backup of the second version of the data, comprising;

determine differences between the first data backup and the second version of the data,

for a portion at a first position of the second version of the data that is the same as a portion of the first data backup, reference the portion of the first data backup in the second data backup,

for a portion at a first position of the second version of the data that is different than all portions of the first data backup, include the portion of the second version of the data in the second data backup, and

create reconstruction metadata for assembling an incremental data backup, of the first data backup, into a replication of the second version of the data; and

append the second data backup to the first data backup, the append creates a data backup incremental of the first data backup and the second data backup.

16. The system of claim 15, wherein said determine comprises comparing a signature for a potion of the second version of data to signatures for a plurality of portions of the first data backup.

17. The system of claim 15, wherein the byte stream is received without knowledge of the source system'"'"'s file system and operating system.

18. The system of claim 15, wherein creating an incremental data backup independent of the source system'"'"'s operating system and file system is done without file-level access to the source system'"'"'s storage.

19. The system of claim 16, wherein said program code is further executable by the processor to create the first data backup of the first version of the data, and said create the first data backup comprises:

compare hashes of portions of the first version of the data to hashes of portions of previously stored data,

for portions of the first version of the data that have a same hash as portions of the previously stored data, reference the portions of the first version of the data in the first data backup, and

for portions of the first version of the data that have a different hash than portions of the previously stored data, store the portions of the first version of the data in the first data backup.