Archive and backup virtualization

• US 9,002,800 B1
• Filed: 06/30/2007
• Issued: 04/07/2015
• Est. Priority Date: 06/30/2007
• Status: Active Grant

• Current Assignee: Emc IP Holding Company LLC (Dell Technologies Inc.)
• Original Assignee: EMC Corporation (Dell Technologies Inc.)
• Inventors: Yueh, Jedidiah
• Primary Examiner(s): Alam, Hosain
• Assistant Examiner(s): JOHNSON, JOHNESE T
• Application Number:
  US11/772,182
• Time in Patent Office: 2,838 Days
• Field of Search: 707/609./640/661
• US Class Current: 707/661
• CPC Class Codes: G06F 11/1448 Management of the data involved in backup or backup restore
• CPC Class Codes: G06F 11/1456 Hardware arrangements for backup
• CPC Class Codes: G06F 12/145 the protection being virtual, e.g. for virtual blocks or segments before a translation mechanism
• CPC Class Codes: G06F 16/27 Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor

Abstract

A data storage and protection system includes secondary storage and at least one instance of a high efficiency storage application (“HESA”). The HESA backs up and archives client data stored in primary storage of a client computer system or client node to secondary storage. Archive files generated by the HESA re-use previously backed up client data stored in the secondary storage. In one embodiment, previously backed up client data is re-used for an archive file by organizing the archive file as a hash tree having hash values pointing to the previously backed up client data. In addition, the HESA can maximize available space in the primary storage by replacing previously backed up and/or archived client data in the primary storage with pointers that point to the previously backed up and/or archived client data in secondary storage.

Claims

1. A method for providing data protection in a computer architecture including a client computer system and secondary storage, the method comprising:

integrating a data backup process together with a data archive process, where the data backup process is different from the data archive process, and the integration of the data backup process with the data archive process is performed by;

creating a de-duplicated backup data set from a raw data set that includes both active and inactive portions stored in primary storage of a client computer system, wherein the de-duplicated backup data set includes part of the active portion of the raw data set;

storing the de-duplicated backup data set in a secondary storage remote from the client computer system;

creating a de-duplicated archive data set from at least the inactive portion of the raw data set stored in the primary storage, wherein the archive data set re-uses one or more pieces of raw data currently stored in the secondary storage by referencing the one or more pieces of raw data in the de-duplicated backup data set, the one or more pieces of raw data having been stored in the secondary storage during one or more previous backups; and

storing the de-duplicated archive data set in the secondary storage such that at least some of the raw data included in the de-duplicated archive data set is the same raw data included in the de-duplicated backup data set.

2. The method of claim 1, wherein creating the de-duplicated backup data set from the raw data set includes:

breaking the raw data set into one or more pieces;

performing a hash function on each piece; and

comparing each resulting hash value to a set of hashes corresponding to pieces of data previously stored in secondary storage.

3. The method of claim 2, wherein creating the de-duplicated backup data set further includes building a hash tree of the raw data set, the hash tree including a plurality of hash values representative of the one or more pieces of the raw data set.

4. The method of claim 3, wherein the de-duplicated backup data set includes the hash tree of the raw data set and one or more pieces of data whose hash values did not match any of the hashes in the set of hashes corresponding to pieces of data previously stored in secondary storage.

5. The method of claim 1, wherein the archive data set re-uses the one or more pieces of raw data currently stored in the secondary storage by pointing to the one or more pieces of raw data with one or more hash values organized in a hash tree representative of the archive data set.

6. The method of claim 1, further comprising, deleting the at least a portion of the raw data set from the primary storage after creating the archive data set.

7. The method of claim 1, wherein the at least a portion of the raw data set from which the archive data set is created comprises a first portion of the raw data set, the method further comprising, replacing a second portion of the raw data stored in primary storage with one or more pointers, wherein the second portion of the raw data set has previously been stored to the secondary storage during one or more prior backups and wherein the one or more pointers point to the second portion of the raw data set in the secondary storage.

8. The method of claim 7, wherein the one or more pointers comprise one or more hash values obtained by performing a hash function on one or more pieces of data of the second portion of the raw data set.

9. A method for reducing redundant data in a data storage system by re-using backup data for archiving, the method comprising:

integrating a data backup process together with a data archive process, where the data backup process is different from the data archive process, and the integration of the data backup process with the data archive process is performed by;

storing client data that includes both active and inactive portions in primary storage of a client system, the client data including data previously backed up and stored in secondary storage and data not previously backed up or stored in secondary storage;

generating a backup file including active data from the active portion of the client data, wherein the backup file includes the not previously backed up or stored client data and a hash tree organization of the client data from which the backup file is generated, the backup file hash tree organization including one or more hash values pointing to one or more corresponding pieces of client data;

storing the backup file in secondary storage;

generating an archive file from at least the inactive portion of the client data stored in primary storage, the archive file including a hash tree organization of the at least the first portion of the client data, wherein the archive file hash tree organization includes one or more hash values pointing to one or more corresponding pieces of client data previously backed up and stored in secondary storage, wherein at least some of the one or more hash values included in the archive file hash tree organization are included in the backup file hash tree organization;

storing the archive file in secondary storage such that at least some of the raw data included in the archive file is the same raw data included in the backup file; and

removing the first portion of the client data that is inactive from the primary storage.

10. The method of claim 9, wherein generating the backup file from the client data includes:

breaking the client data into one or more pieces;

performing a hash function on each piece;

comparing each resulting hash value to a plurality of hashes corresponding to pieces of data previously backed up and stored in secondary storage to identify pieces of the client data that are new or modified; and

organizing the hash values for all of the pieces of the client data into the backup file hash tree.

11. The method of claim 9, wherein the generating and storing of backup and archive files is performed by:

a high efficiency storage application installed and operating on the client system;

a high efficiency storage application installed and operating on a backup and archive server managing the secondary storage;

or

two high efficiency storage applications, one each installed and operating on the client system and the backup and archive server.

12. The method of claim 9, further comprising:

deleting a second portion of the client data from primary storage, the second portion of the client data having previously been backed up and stored in secondary storage; and

storing a plurality of pointers in the primary storage that point to the second portion of the client data stored in secondary storage.

13. The method of claim 12, wherein the plurality of pointers comprise a plurality of probabilistically unique hash values obtained by hashing a plurality of pieces of data corresponding to the second portion of the client data.

14. The method of claim 12, wherein the first portion of the client data includes the second portion of the client data and wherein generating an archive file from at least a first portion of the client data includes copying the pointers and using the copied pointers in the archive file hash tree organization.

15. A data storage and protection system, comprising:

a computer system that includes a secondary storage, the computer system including a high efficiency storage application configured to integrate a data backup process together with a data archive process, where the data backup process is different from the data archive process, wherein in operation, the high efficiency storage application operates to;

create a de-duplicated backup data set from a raw data set that includes both active and inactive portions stored in primary storage of a client computer system;

store the de-duplicated backup data set in a secondary storage remote from the client computer system;

create a de-duplicated archive data set from at least the inactive portion of the raw data set stored in the primary storage, wherein the de-duplicated archive data set re-uses one or more pieces of raw data currently stored in the secondary storage by referencing the one or more pieces of raw data in the de-duplicated backup data set, the one or more pieces of raw data having been stored in the secondary storage during one or more previous backups; and

store the de-duplicated archive data set in the secondary storage, wherein the de-duplicated backup data set and the de-duplicated archive data set are different data sets and are configured to reference some of the same raw data such that at least some of the raw data included in the de-duplicated archive data set is the same raw data included in the de-duplicated backup data set.

16. The data storage and protection system of claim 15, wherein the de-duplicated archive data set re-uses previously backed up raw data by pointing to the previously backed up raw data with one or more hash values organized in a hash tree representative of the de-duplicated archive data set.

17. The data storage and protection system of claim 15, wherein the high efficiency storage application is installed and operating on a client computer system or a client node.

18. The data storage and protection system of claim 17, further comprising a backup and archive server configured to manage the de-duplicated backup data set and the de-duplicated archive data set received in the secondary storage.

19. The data storage and protection system of claim 17, wherein the client computer system or client node includes primary storage for storing active or key data sets.

20. The data storage and protection system of claim 19, wherein at least a portion of the active or key data sets have been backed up in secondary storage and wherein the high efficiency storage application is further configured to delete the at least a portion of the active or key data sets from the primary storage and represent them in the primary storage using one or more pointers that point to the at least a portion of the active or key data sets in secondary storage.