Deletion and restoration of archived data in cloud/object storage

• US 10,503,602 B2
• Filed: 01/12/2018
• Issued: 12/10/2019
• Est. Priority Date: 01/12/2018
• Status: Active Grant

• Current Assignee: Vmware LLC (Broadcom, Inc.)
• Original Assignee: VMware, Inc. (Broadcom, Inc.)
• Inventors: Kashi Visvanathan, Satish Kumar, Sarda, Pooja, Langouev, Ilya
• Primary Examiner(s): Doan, Khoa D
• Application Number:
  US15/870,728
• Publication Number:
  US 20190220360A1
• Time in Patent Office: 697 Days
• CPC Class Codes: G06F 11/1435 using file system or storage system metadata
• CPC Class Codes: G06F 11/1448 Management of the data involved in backup or backup restore
• CPC Class Codes: G06F 11/1469 Backup restoration techniques
• CPC Class Codes: G06F 11/2094 Redundant storage or storage space G06F11/2056 takes precedence
• CPC Class Codes: G06F 16/162 Delete operations erasing in storage systems G06F3/0652
• CPC Class Codes: G06F 16/907 Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
• CPC Class Codes: G06F 2201/82 Solving problems relating to consistency
• CPC Class Codes: G06F 2201/84 Using snapshots, i.e. a logical point-in-time copy of the data
• CPC Class Codes: G06F 3/0608 Saving storage space on storage systems
• CPC Class Codes: G06F 3/0652 Erasing, e.g. deleting, data cleaning, moving of data to a wastebasket
• CPC Class Codes: G06F 3/067 Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
• CPC Class Codes: H04L 67/1095 Replication or mirroring of data, e.g. scheduling or transport for data synchronisation between network nodes
• CPC Class Codes: H04L 67/1097 for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]

Abstract

Techniques for deleting a snapshot of a data set archived in cloud/object storage are provided. In one set of embodiments, a computer system can identify data blocks that are part of the snapshot and are referred to by an incremental child snapshot of the snapshot. The computer system can further create a new metadata object that includes metadata entries for the identified data blocks, each metadata entry comprising an identifier of the incremental child snapshot, an identifier of a data block, and an identifier of a data object of the incremental child snapshot where the data block is stored. The computer system can then associate the new metadata object with the incremental child snapshot in the cloud/object storage.

Claims

1. A method comprising:

deleting, by a computer system, a snapshot of a data set archived in cloud storage, the deleting comprising;

identifying data blocks that are part of the snapshot and are referred to by an incremental child snapshot of the snapshot;

creating a new metadata object that includes metadata entries for the identified data blocks, each metadata entry comprising an identifier of the incremental child snapshot, an identifier of a data block, and an identifier of a data object of the snapshot where the data block is stored; and

associating the new metadata object with the incremental child snapshot in the cloud storage; and

concurrently with the deleting of the snapshot, restoring the data set from the incremental child snapshot without locking any data objects of the snapshot.

2. The method of claim 1 wherein the deleting further comprises:

deleting all metadata objects for the snapshot from the cloud storage.

3. The method of claim 1 wherein the deleting further comprises, at a later time:

identifying data objects of the snapshot that do not include any data blocks referred to by child snapshots; and

deleting the identified data objects from the cloud storage.

4. The method of claim 1 wherein the restoring comprises:

traversing up a hierarchy of snapshots needed to restore the data set from the incremental child snapshot; and

for each snapshot in the hierarchy;

if one or more metadata objects for the snapshot can be found in the cloud storage, incorporating the one or more snapshots into a metadata view for the data set; and

if one or more metadata objects for the snapshot cannot be found in the cloud storage, returning to an immediate child snapshot of the snapshot in the hierarchy and examining the immediate child snapshot to identify any new metadata objects.

5. The method of claim 1 wherein the incremental child snapshot is restored from a replicated region of the cloud storage that is different from a region of the cloud storage where the incremental child snapshot was originally uploaded.

6. The method of claim 5 wherein the restoring comprises:

verifying, prior to initiating the restore, that all objects needed for restoring the data set are present in the replicated region.

7. A non-transitory computer readable storage medium having stored thereon program code executable by a computer system, the program code embodying a method comprising:

deleting a snapshot of a data set archived in cloud storage, the deleting comprising;

identifying data blocks that are part of the snapshot and are referred to by an incremental child snapshot of the snapshot;

creating a new metadata object that includes metadata entries for the identified data blocks, each metadata entry comprising an identifier of the incremental child snapshot, an identifier of a data block, and an identifier of a data object of the snapshot where the data block is stored; and

associating the new metadata object with the incremental child snapshot in the cloud storage; and

concurrently with the deleting of the snapshot, restoring the data set from the incremental child snapshot without locking any data objects of the snapshot.

8. The non-transitory computer readable storage medium of claim 7 wherein the deleting further comprises:

deleting all metadata objects for the snapshot from the cloud storage.

9. The non-transitory computer readable storage medium of claim 7 wherein the deleting further comprises, at a later time:

identifying data objects of the snapshot that do not include any data blocks referred to by child snapshots; and

deleting the identified data objects from the cloud storage.

10. The non-transitory computer readable storage medium of claim 7 wherein the restoring comprises:

traversing up a hierarchy of snapshots needed to restore the data set from the incremental child snapshot; and

for each snapshot in the hierarchy;

if one or more metadata objects for the snapshot can be found in the cloud storage, incorporating the one or more snapshots into a metadata view for the data set; and

if one or more metadata objects for the snapshot cannot be found in the cloud storage, returning to an immediate child snapshot of the snapshot in the hierarchy and examining the immediate child snapshot to identify any new metadata objects.

11. The non-transitory computer readable storage medium of claim 7 wherein the incremental child snapshot is restored from a replicated region of the cloud storage that is different from a region of the cloud storage where the incremental child snapshot was originally uploaded.

12. The non-transitory computer readable storage medium of claim 11 wherein the restoring comprises:

verifying, prior to initiating the restore, that all objects needed for restoring the data set are present in the replicated region.

13. A computer system comprising:

a processor; and

a non-transitory computer readable medium having stored thereon program code that causes the processor to;

delete a snapshot of a data set archived in cloud storage, the deleting comprising;

identifying data blocks that are part of the snapshot and are referred to by an incremental child snapshot of the snapshot;

creating a new metadata object that includes metadata entries for the identified data blocks, each metadata entry comprising an identifier of the incremental child snapshot, an identifier of a data block, and an identifier of a data object of the incremental child snapshot where the data block is stored; and

associating the new metadata object with the incremental child snapshot in the cloud storage; and

concurrently with the deleting of the snapshot, restore the data set from the incremental child snapshot without locking any data objects of the snapshot.

14. The computer system of claim 13 wherein the deleting further comprises:

deleting all metadata objects for the snapshot from the cloud storage.

15. The computer system of claim 13 wherein the deleting further comprises, at a later time:

identifying data objects of the snapshot that do not include any data blocks referred to by child snapshots; and

deleting the identified data objects from the cloud storage.

16. The computer system of claim 13 wherein the restoring comprises:

traversing up a hierarchy of snapshots needed to restore the data set from the incremental child snapshot; and

for each snapshot in the hierarchy;

if one or more metadata objects for the snapshot can be found in the cloud storage, incorporating the one or more snapshots into a metadata view for the data set; and

if one or more metadata objects for the snapshot cannot be found in the cloud storage, returning to an immediate child snapshot of the snapshot in the hierarchy and examining the immediate child snapshot to identify any new metadata objects.

17. The computer system of claim 13 wherein the incremental child snapshot is restored from a replicated region of the cloud storage that is different from a region of the cloud storage where the incremental child snapshot was originally uploaded.

18. The computer system of claim 17 wherein the restoring comprises:

verifying, prior to initiating the restore, that all objects needed for restoring the data set are present in the replicated region.