Creation of synthetic backups within deduplication storage system

• US 9,697,222 B2
• Filed: 03/13/2013
• Issued: 07/04/2017
• Est. Priority Date: 12/01/2010
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Aronovich, Lior, Hirsch, Michael, Toaff, Yair
• Primary Examiner(s): Uddin, Md. I
• Application Number:
  US13/801,756
• Publication Number:
  US 20130232119A1
• Time in Patent Office: 1,574 Days
• Field of Search: 707640, 707641, 707696, 707698
• CPC Class Codes: G06F 11/1453 using de-duplication of the data
• CPC Class Codes: G06F 16/1748 De-duplication implemented within the file system, e.g. based on file segments de-duplication techniques in storage systems for the management of data blocks G06F3/0641
• CPC Class Codes: G06F 16/2255 Hash tables
• CPC Class Codes: G06F 16/24556 Aggregation; Duplicate elimination
• CPC Class Codes: G06F 16/907 Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually

Abstract

A deduplication storage system and a backup application create a synthetic backup. Metadata instructions are provided to the deduplication storage system. Each of the metadata instructions specifies the data segment of an originating backup and a designated location of the data segment in the synthetic backup. Each of the metadata instructions are processed by locating those data sub-segments in the deduplication storage system specified by the data segment in each of the metadata instructions, and creating metadata references to each of the data sub-segments and adding the metadata references to metadata of the synthetic backup being created.

Claims

1. A method for creating a synthetic backup by a deduplication storage system and a backup application using a processor device, comprising:

providing the deduplication storage system a plurality of metadata instructions, wherein each of the plurality of metadata instructions specifies a data segment of an originating backup and a designated location of the data segment in the synthetic backup;

processing each of the plurality of metadata instructions by each of;

partitioning each data segment input into each of a plurality of fixed-sized data sub-segments, each sub-segment referencing a plurality of stored sub-segments,

for each of the plurality of data sub-segments, during the construction of the synthetic backup, calculating each of a plurality of input deduplication digests based on a retrieved plurality of stored deduplication digests,

locating those of the plurality of data sub-segments in the deduplication storage system specified by the data segment in each of the plurality of metadata instructions, and

creating metadata references to each of the plurality of data sub-segments and adding the metadata references to metadata of the synthetic backup being created wherein the references includes physical and logical data patterns;

constructing the synthetic backup by consolidating the plurality of metadata instructions that reference adjacent backup data segments into a single metadata instruction;

wherein the synthetic backup includes data from already existing full backup and its subsequent incremental backups dating until a specific point in time.

2. The method of claim 1, further including using a plurality of metadata calculations and metadata operations to process each of the plurality of metadata instructions.

3. The method of claim 1, further including incorporating a plurality of storage data sub-segments in the deduplication storage system in all data segments of a stored backup.

4. The method of claim 3, further including performing one of:

associating a backup data segment with one of the plurality of storage data sub-segments during a deduplication process, and

storing association information of the backup data segment with one of the plurality of storage data sub-segments in metadata associated with the backup data segment.

5. The method of claim 4, further including performing reference counts in a granularity of the plurality of storage data sub-segments.

6. The method of claim 5, further including incrementing values of the reference counts of the plurality of storage data sub-segments that are referenced by the data segment specified in each of the plurality of metadata instructions in accordance with a number of new references to each of the plurality of storage data sub-segments.

7. A system for creating a synthetic backup by a deduplication storage system and a backup application, comprising:

the deduplication storage system;

at least one processor device, operable in the deduplication computing storage environment, wherein the at least one processor device;

provides the deduplication storage system a plurality of metadata instructions, wherein each of the plurality of metadata instructions specifies a data segment of an originating backup and a designated location of the data segment in the synthetic backup, processes each of the plurality of metadata instructions by each of;

partitioning each data segment input into each of a plurality of fixed-sized data sub-segments, each sub-segment referencing a plurality of stored sub-segments,

for each of the plurality of data sub-segments, during the construction of the synthetic backup, calculating each of a plurality of input deduplication digests based on a retrieved plurality of stored deduplication digests,

locating those of the plurality of data sub-segments in the deduplication storage system specified by the data segment in each of the plurality of metadata instructions, and

creating metadata references to each of the plurality of data sub-segments and adding the metadata references to metadata of the synthetic backup being created wherein the references includes physical and logical data patterns,

constructing the synthetic backup by consolidates the plurality of metadata instructions that reference adjacent backup data segments into a single metadata instruction;

wherein the synthetic backup includes data from already existing full backup and its subsequent incremental backups dating until a specific point in time.

8. The system of claim 7, wherein the at least one processor device uses a plurality of metadata calculations and metadata operations to process each of the plurality of metadata instructions.

9. The system of claim 7, wherein the at least one processor device incorporates a plurality of storage data sub-segments in the deduplication storage system all data segments of a stored backup.

10. The system of claim 9, wherein the at least one processor device performs one of:

associating a backup data segment with one of the plurality of storage data sub-segments during a deduplication process, and

storing association information of the backup data segment with one of the plurality of storage data sub-segments in metadata associated with the backup data segment.

11. The system of claim 10, wherein the at least one processor device performs reference counts in a granularity of the plurality of storage data sub-segments.

12. The system of claim 11, wherein the at least one processor device increments values of the reference counts of the plurality of storage data sub-segments that are referenced by the data segment specified in each of the plurality of metadata instructions in accordance with a number of new references to each of the plurality of storage data sub-segments.

13. A computer program product for creating a synthetic backup by a deduplication storage system and a backup application using at least one processor device, the computer program product comprising a non-transitory computer-readable storage medium having computer-readable program code portions stored therein, the computer-readable program code portions comprising:

an executable portion that provides the deduplication storage system a plurality of metadata instructions, wherein each of the plurality of metadata instructions specifies a data segment of an originating backup and a designated location of the data segment in the synthetic backup;

an executable portion that processes each of the plurality of metadata instructions by each of;

partitioning each data segment input into each of a plurality of fixed-sized data sub-segments, each sub-segment referencing a plurality of stored sub-segments,

for each of the plurality of data sub-segments, during the construction of the synthetic backup, calculating each of a plurality of input deduplication digests based on a retrieved plurality of stored deduplication digests,

locating those of the plurality of data sub-segments in the deduplication storage system specified by the data segment in each of the plurality of metadata instructions, and

creating metadata references to each of the plurality of data sub-segments and adding the metadata references to metadata of the synthetic backup being created wherein the references includes physical and logical data patterns;

an executable portion that constructing the synthetic backup by consolidates the plurality of metadata instructions that reference adjacent backup data segments into a single metadata instruction;

wherein the synthetic backup includes data from already existing full backup and its subsequent incremental backups dating until a specific point in time.

14. The computer program product of claim 13, further including an executable portion that uses a plurality of metadata calculations and metadata operations to process each of the plurality of metadata instructions.

15. The computer program product of claim 13, further including an executable portion that incorporates a plurality of storage data sub-segments in the deduplication storage system in all data segments of a stored backup.

16. The computer program product of claim 15, further including an executable portion that performs one of:

associating a backup data segment with one of the plurality of storage data sub-segments during a deduplication process, and

storing association information of the backup data segment with one of the plurality of storage data sub-segments in metadata associated with the backup data segment.

17. The computer program product of claim 16, further including an executable portion that performs reference counts in a granularity of the plurality of storage data sub-segments.

18. The computer program product of claim 17, further including an executable portion that increments values of the reference counts of the plurality of storage data sub-segments, that are referenced by the data segment specified in each of the plurality of metadata instructions, in accordance with a number of new references to each of the plurality of storage data sub-segments.