NAMESPACE POLICY BASED DEDUPLICATION INDEXES

• US 20170315875A1
• Filed: 04/29/2016
• Published: 11/02/2017
• Est. Priority Date: 04/29/2016
• Status: Abandoned Application

• Current Assignee: NetApp, Inc.
• Original Assignee: NetApp, Inc.
• Inventors: Tirupati Nagaraj, Sudhindra Prasad, Pisupati, Pramodh, Taleck, Gregory Thomas
• Application Number:
  US15/143,133
• Publication Number:
  US 20170315875A1
• Time in Patent Office: Days
• CPC Class Codes: G06F 11/1451 by selection of backup contents
• CPC Class Codes: G06F 11/1453 using de-duplication of the data
• CPC Class Codes: G06F 11/1464 for networked environments
• CPC Class Codes: G06F 16/215 Improving data quality; Data cleansing, e.g. de-duplication, removing invalid entries or correcting typographical errors
• CPC Class Codes: G06F 16/22 Indexing; Data structures therefor; Storage structures
• CPC Class Codes: G06F 16/2272 Management thereof
• CPC Class Codes: G06F 16/2365 Ensuring data consistency and integrity
• CPC Class Codes: G06F 2201/82 Solving problems relating to consistency ensuring consistency in mirrored systems G06F11/2064

Abstract

A cloud storage gateway device can be used to deduplicate data across different namespaces while complying with SLOs that govern data of the different namespaces. A cloud storage gateway device can use multiple fingerprint indexes to comply with different SLOs. Each fingerprint index corresponds to a different SLO. Thus, the cloud storage gateway device deduplicates data against other data governed by a same SLO. Assuming an SLO aligns or indicates a cloud storage target, the cloud storage gateway device will deduplicate data against other data that will eventually migrate from the device to a same cloud storage target. The cloud storage gateway device ensures satisfaction of the governing SLO(s) from receipt of data, through deduplication, to the migration of the data to a cloud storage target.

Claims

1. A method comprising:

after receipt of a first dataset, determining a first service level objective associated with a first namespace corresponding to the first dataset;

selecting a first deduplication index from a plurality of deduplication indexes based, at least in part, on the service level objective; and

tagging metadata of the first dataset to indicate a cloud storage target corresponding to the first deduplication index.

2. The method of claim 1 further comprising, for a first data block of the first data set that does not have a matching entry in the first deduplication index, tagging the first data block to indicate the cloud storage target.

3. The method of claim 2 further comprising updating a log to indicate the first data block and the cloud storage target and storing the first data block in storage of a cloud storage gateway device that received the first dataset to back up to cloud storage.

4. The method of claim 1 further comprising receiving the first dataset to back up to the cloud storage target.

5. The method of claim 1, wherein tagging the metadata of the first dataset comprises updating a metadata block for the first dataset with an indication of the cloud storage target.

6. The method of claim 1 further comprising:

determining that the first service level objective indicates that data of the first namespace can be deduplicated with data of another namespace; and

determining that the first deduplication index is used to deduplicate data of namespaces to be migrated or already migrated to the cloud storage target.

7. The method of claim 1 further comprising determining a second service level object associated with the first namespace, wherein selection of the first deduplication index is also based on the second service level objective.

8. The method of claim 1 further comprising:

after receipt of a second dataset and after migration of a second data block of the first dataset to the cloud storage target, determining that a first data block of the second dataset is a duplicate of the second data block when deduplicating the second dataset with the first deduplication index.

9. The method of claim 8 further comprising updating metadata of the second dataset with an identifier of the second data block.

10. The method of claim 1 further comprising:

after detection of a migration trigger, reading data blocks of the first dataset from storage of a gateway device that received the first dataset;

for each of the data blocks of the first dataset,

determining that the data block is tagged with the indication of the cloud storage target;

migrating the data block to the cloud storage target based on the indication of the cloud storage target;

reading the metadata of the first dataset from the storage;

determining that the metadata is tagged with an indication of the cloud storage target; and

migrating the metadata of the first dataset to the cloud storage target based on the indication of the cloud storage target in the metadata.

11. The method of claim 1 further comprising:

after detection of an eviction related trigger, selecting one of the plurality of deduplication indexes;

determining a set of one or more entries of the selected deduplication index for eviction according to an eviction algorithm;

determining cloud migration status of each data block corresponding to the set of one or more entries; and

evicting those of the set of one or more entries corresponding to a data block that has been migrated to cloud storage.

12. One or more non-transitory machine-readable media comprising program code for namespace policy based deduplication, the program code to:

maintain a deduplication index for each of a plurality of cloud storage targets;

for deduplication of a dataset, select from the deduplication indexes based, at least in part, on a policy associated with a namespace of the dataset, wherein the policy comprises at least one service level objective that corresponds to at least one of the plurality of cloud storage targets; and

indicate, in metadata of the dataset the cloud storage target of the plurality of cloud storage targets that corresponds to the deduplication index selected to deduplicate the dataset.

13. The machine-readable storage media of claim 12, further comprising program code to deduplicate datasets of multiple namespaces with one of the deduplication indexes when the datasets are to be migrated to a same cloud storage target.

14. The machine-readable storage media of claim 12, further comprising program code to deduplicate datasets of multiple namespaces with one of the deduplication indexes when the multiple namespaces are governed by a same policy

15. The machine-readable media of claim 12, wherein the program code to maintain the deduplication indexes comprises allowing at least some entries to remain in deduplication indexes after corresponding data blocks have migrated to cloud storage targets.

16. The machine-readable media of claim 12, wherein the program code to maintain the deduplication indexes comprises allowing at least some entries to remain in deduplication indexes after corresponding data blocks have migrated to cloud storage targets.

17. The machine-readable media of claim 12 further comprising program code to:

after detection of an eviction related trigger, selecting one of the deduplication indexes;

apply an eviction algorithm to the selected deduplication index to determine a set of one or more entries of for eviction;

determine cloud migration status of each data block corresponding to the set of one or more entries; and

evict those of the set of one or more entries corresponding to a data block that has been migrated to cloud storage.

18. A storage gateway device comprising:

a processor; and

a machine-readable medium comprising program code executable by the processor to cause the storage gateway device to,

maintain a deduplication index for each of a plurality of cloud storage targets configured on the storage gateway device;

for deduplication of a dataset, select from the deduplication indexes based, at least in part, on a policy associated with a namespace of the dataset, wherein the policy comprises at least one service level objective that corresponds to at least one of the plurality of cloud storage targets; and

indicate, in metadata of the dataset the cloud storage target of the plurality of cloud storage targets that corresponds to the deduplication index selected to deduplicate the dataset.

19. The storage gateway device of claim 18, wherein the machine-readable medium further comprises program code to deduplicate datasets of multiple namespaces with one of the deduplication indexes when the datasets are to be migrated to a same cloud storage target.

20. The storage gateway device of claim 18, wherein the machine-readable medium further comprises program code to maintain the deduplication indexes comprises allowing at least some entries to remain in deduplication indexes after corresponding data blocks have migrated to cloud storage targets.