Partial Block Based Backups

• US 20110218967A1
• Filed: 03/08/2010
• Published: 09/08/2011
• Est. Priority Date: 03/08/2010
• Status: Abandoned Application

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Sliger, Michael, Suriyanarayanan, Guhan, Lyon, James M., Deb, Bodhi, Bindal, Anuj
• Application Number:
  US12/719,837
• Publication Number:
  US 20110218967A1
• Time in Patent Office: Days
• US Class Current: 707/647
• CPC Class Codes: G06F 11/00 Error detection; Error correction; Monitoring error detection, correction or monitoring in information storage based on relative movement between record carrier and transducer G11B20/18; monitoring, i.e. supervising the progress of recording or reproducing G11B27/36; in static stores G11C29/00
• CPC Class Codes: G06F 11/14 Error detection or correction of the data by redundancy in operation G06F11/16 takes precedence
• CPC Class Codes: G06F 11/1451 by selection of backup contents
• CPC Class Codes: G06F 11/1453 using de-duplication of the data
• CPC Class Codes: G06F 11/1464 for networked environments
• CPC Class Codes: G06F 11/1469 Backup restoration techniques
• CPC Class Codes: G06F 12/00 Accessing, addressing or allocating within memory systems or architectures digital input from, or digital output to record carriers, e.g. to disk storage units, G06F3/06
• CPC Class Codes: G06F 12/16 Protection against loss of memory contents contains no material, see G06F11/00
• CPC Class Codes: G06F 17/00 Digital computing or data processing equipment or methods, specially adapted for specific functions information retrieval, database structures or file system structures therefor G06F16/00
• CPC Class Codes: G06F 2201/81 Threshold

Abstract

A block based backup system may perform several partial backups to incrementally transfer backup information to a backup system. Each partial backup may build on the previous backup and the partial backups may be marked as unable to be used for restoration. In some cases, the partial backups may be portions of a file system snapshot, while in other cases, the partial backups may include any changes that occurred since a last partial backup. The size of the partial backups may be dynamically changed depending on network connections, workloads, and other factors.

Claims

1. A method of backing up a file system to a backup storage system, said file system being stored on a storage system as a plurality of blocks of data, said method comprising:

analyzing said file system to identify a set of said blocks of data to backup;

performing one or more partial backups by selecting a subset of said blocks of data to perform a partial backup, backing up each of said subset of blocks of data to create a partial backup on said backup storage system, said partial backup being unusable for restore operations;

performing a final partial backup by selecting remaining blocks of data that have not been backed up by said one or more partial backups, backing up said remaining blocks of data, and creating a final backup comprising said one or more partial backup, said final backup being usable for restore operations.

2. The method of claim 1 further comprising:

sorting said blocks of data into a sorted list; and

selecting said subset in order of said sorted list to create said subset.

3. The method of claim 2, said sorting being in order of block placement on said storage system.

4. The method of claim 1, said analyzing said file system comprising:

identifying a file that has not been backed up;

identifying one or more blocks of data associated with said file; and

adding said one or more blocks of data to said set of said blocks of data.

5. The method of claim 1, said backing up comprising:

determining that a first block is currently stored by said backup storage system and not transferring said first block to said storage system; and

determining that a second block is not stored by said backup storage system and transferring said second block to said storage system.

6. The method of claim 5, said determining being performed by:

calculating hash values for each said first block and said second block and querying said backup storage system to determine that said first block is present in said backup storage system and said second block is not present in said backup storage system.

7. The method of claim 1 further comprising:

attempting to perform a first partial backup and detecting a failure of said first partial backup; and

retrying said first partial backup until said first partial backup is successful and proceeding to a next partial backup.

8. The method of claim 1, said subset being selected by using a threshold to determine a limit for said subset.

9. The method of claim 8, said threshold being defined as a maximum size of data to transfer.

10. The method of claim 8 further comprising:

attempting to perform a first partial backup and detecting a failure of said first partial backup;

changing said threshold based on said failure to create a modified threshold; and

retrying said first partial backup using said modified threshold.

11. The method of claim 10 further comprising:

classifying a network connection to said backup storage system to determine said threshold.

12. The method of claim 11, said classifying comprising determining a bandwidth for said network connection.

13. The method of claim 11, said classifying comprising determining a reliability for said network connection.

14. The method of claim 1, said analyzing said file system comprising marking each of said blocks of data as one of a group composed of empty, backed up, and suspect.

15. The method of claim 14, after completing said partial backup, marking each of said subset of blocks of data as backed up.

16. A system comprising:

a connection to a backup storage system;

a file storage system comprising a file system comprising a plurality of files, each of said files being stored in at least one blocks of data on said file storage system;

a processor that performs a method comprising;

analyzing said file system to identify a set of said blocks of data to backup;

determining a threshold for a partial backup;

performing one or more partial backups by selecting a subset of said blocks of data to perform a partial backup, said subset being determined using said threshold, backing up each of said subset of blocks of data to create a partial backup on said backup storage system, said partial backup being unusable for restore operations;

performing a final partial backup by selecting remaining blocks of data that have not been backed up by said one or more partial backups, backing up said remaining blocks of data, and creating a final backup comprising said one or more partial backup, said final backup being usable for restore operations.

17. The system of claim 16 further comprising:

a threshold management system that determines a performance parameter for said connection and sets said threshold based on said performance parameter.

18. The system of claim 17, said threshold management system determining said performance parameter using an active method to determine said performance parameter.

19. A method of backing up a file system to a backup storage system, said file system being stored on a storage system as a plurality of blocks of data, said method comprising:

analyzing said file system to identify a set of said blocks of data to backup, said analyzing comprising comparing blocks of data contained in said storage system to blocks of data stored on said backup storage system to identify a first set of blocks of data that are not stored on said backup storage system and are contained in said storage system;

performing one or more partial backups by selecting a subset of said first set of blocks of data to perform a partial backup, backing up each of said subset of blocks of data to create a partial backup on said backup storage system, said partial backup being unusable for restore operations;

performing a final partial backup by selecting remaining blocks of data that have not been backed up by said one or more partial backups, backing up said remaining blocks of data, and creating a final backup comprising said one or more partial backup, said final backup being usable for restore operations.

20. The method of claim 19, said first set of blocks of data being identified by performing a process for each of said blocks of data on said storage system, said process comprising:

determining a hash for said block of data;

comparing said hash to a table of hashes stored on said backup storage system; and

determining that said block of data is present on said backup storage system when said hash is found in said table of hashes and determining that said block of data is not present on said backup storage system when said hash is not found in said table of hashes.