Initializing backup snapshots on deduplicated storage

• US 10,031,690 B1
• Filed: 12/16/2013
• Issued: 07/24/2018
• Est. Priority Date: 12/16/2013
• Status: Active Grant

• Current Assignee: Emc IP Holding Company LLC (Dell Technologies Inc.)
• Original Assignee: Emc IP Holding Company LLC (Dell Technologies Inc.)
• Inventors: Panidis, Anestis, Natanzon, Assaf, Cohen, Saar
• Primary Examiner(s): Tsui, Daniel
• Assistant Examiner(s): Wei, Jane
• Application Number:
  US14/108,002
• Time in Patent Office: 1,681 Days
• CPC Class Codes: G06F 3/0608 Saving storage space on storage systems
• CPC Class Codes: G06F 3/0619 in relation to data integrity, e.g. data losses, bit errors
• CPC Class Codes: G06F 3/0641 De-duplication techniques
• CPC Class Codes: G06F 3/067 Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]

Abstract

The system, process, and methods herein describe a mechanism for creating an initial backup snapshot on deduplicated storage. Initialization IO'"'"'s may be transmitted to the deduplicated storage, and those initialization IO'"'"'s may be synthesized into a snapshot. Application IO'"'"'s may also be transmitted in case the source side data changes while the backup is synthesized.

Claims

1. A method for initializing a backup snapshot on deduplicated storage comprising:

receiving sequential initialization input/outputs (“

IOs”

) from a scanned storage volume during a continuous data replication process;

transmitting the sequential initialization IOs to the deduplicated storage, the sequential initialization IOs comprising a data stream and a metadata stream;

writing the sequential initialization IOs to a data journal and a metadata journal on the deduplicated storage that are both used to synthesize a backup snapshot on the deduplicated storage;

receiving application IOs, wherein the application IOs are writes made to the scanned storage volume;

determining a data block associated with an individual application IO;

determining whether the data block is associated with a previously streamed initialization IO;

transmitting the application IOs concurrently with the sequential initialization IOs to the deduplicated storage when the data block is associated with a previously streamed initialization IO;

writing the application IOs to the journal;

synthesizing a backup snapshot from initialization IOs and the application IOs to allow for point-in-time recovery of backed up files of the continuous data replication process by synthesizing new backup snapshots from at least one previous backup snapshot and new entries to a journal, wherein a last backup snapshot of the new backup snapshots is used to restore the storage volume up to a point-in-time of a corresponding last journal entry, and further wherein the synthesizing comprises copying or creating a data pointer to the data journal indicating only changed data for the last journal entry;

querying the metadata journal to identify most recent data associated with each offset specified by the metadata; and

defining a user policy to specify when, on a periodic basis, the new backup snapshots are taken, wherein synthesizing the backup snapshot comprises synthesizing the initialization IOs and the application IOs from the data and metadata journals into the snapshot using the most recent data, and further wherein the synthesizing comprises synthesizing the last backup snapshot entirely from journal entries rather than from an initial backup snapshot.

2. The method of claim 1, wherein the data journal comprises data stored within an offset, and the metadata includes metadata associated with the offset, comprising at least an offset identifier, size, write time, and device ID.

3. The method of claim 1, wherein the application IOs comprise a data stream and a metadata stream that are written to the data journal and the metadata journal.

4. A computer program product for initializing a backup snapshot on deduplicated storage, the computer program product comprising a non-transitory computer readable medium encoded with computer executable program, the program enabling:

receiving sequential initialization input/outputs (“

IOs”

) from a scanned storage volume during a continuous data replication process;

transmitting the sequential initialization IOs to the deduplicated storage, the sequential initialization IOs comprising a data stream and a metadata stream;

writing the sequential initialization IOs to a data journal and a metadata journal on the deduplicated storage that are both used to synthesize a backup snapshot on the deduplicated storage;

receiving application IOs, wherein the application IOs are writes made to the scanned storage volume;

determining a data block associated with an individual application IO;

determining whether the data block is associated with a previously streamed initialization IO;

transmitting the application IOs concurrently with the sequential initialization IOs to the deduplicated storage when the data block is associated with a previously streamed initialization IO;

writing the application IOs to the journal;

synthesizing a backup snapshot from initialization IOs and the application IOs to allow for point-in-time recovery of backed up files of the continuous data replication process by synthesizing new backup snapshots from at least one previous backup snapshot and new entries to a journal, wherein a last backup snapshot of the new backup snapshots is used to restore the storage volume up to a point-in-time of a corresponding last journal entry, and further wherein the synthesizing comprises copying or creating a data pointer to the data journal indicating only changed data for the last journal entry;

querying the metadata journal to identify most recent data associated with each offset specified by the metadata; and

defining a user policy to specify when, on a periodic basis, the new backup snapshots are taken, wherein synthesizing the backup snapshot comprises synthesizing the initialization IOs and the application IOs from the data and metadata journals into the snapshot using the most recent data, and further wherein the synthesizing comprises synthesizing the last backup snapshot entirely from journal entries rather than from an initial backup snapshot.

5. The computer program product of claim 4, wherein the data journal comprises data stored within an offset, and the metadata includes metadata associated with the offset, comprising at least an offset identifier, size, write time, and device ID.

6. The computer program product of claim 4, wherein the application IOs comprise a data stream and a metadata stream that are written to the data journal and the metadata journal.

7. A system for initializing a backup snapshot on deduplicated storage, the system comprising a computer processor configured to execute instructions comprising:

receiving sequential initialization input/outputs (“

IOs”

) from a scanned storage volume during a continuous data replication process;

transmitting the sequential initialization IOs to the deduplicated storage, the sequential initialization IOs comprising a data stream and a metadata stream;

writing the sequential initialization IOs to a data journal and a metadata journal on the deduplicated storage that are both used to synthesize a backup snapshot on the deduplicated storage;

determining a data block associated with an individual application IO;

determining whether the data block is associated with a previously streamed initialization IO;

transmitting the application IOs concurrently with the sequential initialization IOs to the deduplicated storage when the data block is associated with a previously streamed initialization IO;

writing the application IOs to the journal; and

synthesizing a backup snapshot from initialization IOs and the application IOs to allow for point-in-time recovery of backed up files of the continuous data replication process by synthesizing new backup snapshots from at least one previous backup snapshot and new entries to a journal, wherein a last backup snapshot of the new backup snapshots is used to restore the storage volume up to a point-in-time of a corresponding last journal entry, and further wherein the synthesizing comprises copying or creating a data pointer to the data journal indicating only changed data for the last journal entry;

querying the metadata journal to identify most recent data associated with each offset specified by the metadata; and

defining a user policy to specify when, on a periodic basis, the new backup snapshots are taken, wherein synthesizing the backup snapshot comprises synthesizing the initialization IOs and the application IOs from the data and metadata journals into the snapshot using the most recent data, and further wherein the synthesizing comprises synthesizing the last backup snapshot entirely from journal entries rather than from an initial backup snapshot.

8. The system of claim 7, wherein the data journal comprises data stored within an offset, and the metadata includes metadata associated with the offset, comprising at least an offset identifier, size, write time, and device ID.

9. The system of claim 7, wherein the application IOs comprise a data stream and a metadata stream that are written to the data journal and the metadata journal.