Efficient and flexible organization and management of file metadata

• US 9,529,808 B1
• Filed: 07/09/2013
• Issued: 12/27/2016
• Est. Priority Date: 07/16/2012
• Status: Active Grant

• Current Assignee: Tintri Incorporated
• Original Assignee: Tintri, Inc.
• Inventors: Liu, Rose F., Sudarsanam, Raj A., Singh, Pratap V., Lee, Edward K., Gritter, Mark G.
• Primary Examiner(s): OBERLY, VAN HONG
• Application Number:
  US13/938,005
• Time in Patent Office: 1,267 Days
• Field of Search: 707/649
• US Class Current: 1/1
• CPC Class Codes: G06F 11/1451 by selection of backup contents
• CPC Class Codes: G06F 11/1464 for networked environments
• CPC Class Codes: G06F 16/128 Details of file system snapshots on the file-level, e.g. snapshot creation, administration, deletion use of snapshots for error detection or correction G06F11/14, G06F11/16
• CPC Class Codes: G06F 16/211 Schema design and management
• CPC Class Codes: G06F 16/215 Improving data quality; Data cleansing, e.g. de-duplication, removing invalid entries or correcting typographical errors
• CPC Class Codes: G06F 16/245 Query processing
• CPC Class Codes: G06F 16/258 Data format conversion from or to a database
• CPC Class Codes: G06F 2201/80 Database-specific techniques
• CPC Class Codes: G06F 2201/84 Using snapshots, i.e. a logical point-in-time copy of the data

Abstract

Storing data is disclosed, including: receiving a request to create a clone of a snapshot included in a first set of metadata associated with a source data; and generating a second set of metadata for the clone, wherein the second set of metadata associated with the clone includes, for at least some metadata values, a data associating with the second set of metadata one or more corresponding values in the first set of metadata associated with the source data.

Claims

1. A storage system, comprising:

a processor configured to;

receive a request to create a clone of a snapshot included in a first set of metadata associated with a source data;

in response to the request to create the clone of the snapshot;

generate a second set of metadata for the clone, wherein the second set of metadata associated with the clone includes, for at least some metadata values, a data associating with the second set of metadata one or more corresponding values in the first set of metadata associated with the source data,

wherein the second set of metadata for the clone comprises a first active index and a second active index, wherein the first active index is configured to store one or more data values that are written to the clone since a creation of a previous snapshot associated with the clone, wherein the second active index is configured to store one or more data values that at least partially reflect a current state of the clone; and

modify a stored reference clone value associated with the snapshot included in the first set of metadata associated with the source data, wherein the stored reference clone value is used to determine whether to delete the snapshot;

receive a write operation to write a requested data value to the clone; and

in response to the write operation, update the first active index and the second active index associated with the clone with the requested data value; and

a memory coupled to the processor and configured to store the first set of metadata and the second set of metadata.

2. The storage system of claim 1, wherein the processor is further configured to generate the data associating with the second set of metadata the one or more corresponding values in the first set of metadata.

3. The storage system of claim 1, wherein the requested data value comprises a first requested data value, and wherein the processor is further configured to:

receive a read operation to read a second requested data value from the clone; and

in response to the read operation, search the second set of metadata associated with the clone and in the event that the second requested data value is not found in the second set of metadata, search the snapshot included in the first set of metadata associated with the source data.

4. The storage system of claim 3, in the event the second requested data value is not found in the snapshot included in the first set of metadata associated with the source data, the snapshot comprises a first snapshot and wherein the processor is further configured to search a second snapshot included in the first set of metadata associated with the source data, the second snapshot being older than the first snapshot.

5. The storage system of claim 1, wherein the snapshot comprises a first snapshot and wherein the processor is further configured to:

determine that requested data associated with a second snapshot meets a selective promotion criterion, wherein the second snapshot is older than the first snapshot;

generate a piece of metadata associated with the requested data based on one or more selective promotion parameters; and

associate the piece of metadata with the first snapshot.

6. The storage system of claim 5, wherein the selective promotion parameters comprise one or more of the following:

whether the piece of metadata to be generated to reference the requested data references a physical location of the requested data or an existing mapping to the requested data;

whether to promote only mapping associated with the requested data that was previously requested or to promote mappings associated with the requested data that was previously requested in addition to data that is in temporal locality, spatial locality, or grouped with the requested data based on another characteristic;

whether the requested data should be promoted to a shared snapshot or to a live index of the clone, and how long the requested data is to be promoted for.

7. The storage system of claim 1, wherein the processor is further configured to:

determine that a snapshot creation event has occurred;

determine that no changes have been made to the source data since creation of a previous snapshot associated with the source data;

create a label associated with a new snapshot; and

link the label associated with the new snapshot to the previous snapshot associated with the source data, wherein the label is configured to redirect a search of the new snapshot to the linked previous snapshot associated with the source data.

8. The storage system of claim 1, wherein the processor is further configured to:

determine to delete a snapshot to be deleted; and

merge a mapping of the snapshot to be deleted associated with not overwritten data with a next younger snapshot.

9. The storage system of claim 8, wherein whether the snapshot to be deleted is to be merged into the next younger snapshot or the next younger snapshot is to be superimposed onto the snapshot to be deleted is determined dynamically based on one or more criteria including on a first size associated with the snapshot to be deleted and a second size associated with the next younger snapshot.

10. The storage system of claim 1, wherein the second set of metadata associated with the clone does not replicate mappings included in the first set of metadata associated with the source data.

11. The system of claim 1, wherein the requested data value comprises a first requested data value, and wherein the processor is further configured to:

receive a read operation to read a second requested data value from the clone; and

in response to the read operation, search the second active index included in the second set of metadata associated with the clone for with the second requested data value.

12. The system of claim 11, wherein the processor is further configured to:

in the event that a mapping associated with the second requested data value is found in the second active index included in the second set of metadata associated with the clone, return the second requested data value in the second active index.

13. The system of claim 11, wherein the processor is further configured to:

in the event that a mapping associated with the second requested data value is not found in the second active index included in the second set of metadata, use the data associating with the second set of metadata one or more corresponding values in the first set of metadata to search the snapshot included in the first set of metadata associated with the source data for the mapping associated with the second requested data value.

14. A method for storage, comprising:

receiving a request to create a clone of a snapshot included in a first set of metadata associated with a source data;

in response to the request to create the clone of the snapshot;

generating, by a processor, a second set of metadata for the clone, wherein the second set of metadata associated with the clone includes, for at least some metadata values, a data associating with the second set of metadata one or more corresponding values in the first set of metadata associated with the source data,

wherein the second set of metadata for the clone comprises a first active index and a second active index, wherein the first active index is configured to store one or more data values that are written to the clone since a creation of a previous snapshot associated with the clone, wherein the second active index is configured to store one or more data values that at least partially reflect a current state of the clone; and

modifying a stored reference clone value associated with the snapshot included in the first set of metadata associated with the source data, wherein the stored reference clone value is used to determine whether to delete the snapshot;

receiving a write operation to write a requested data value to the clone; and

in response to the write operation, updating the first active index and the second active index associated with the clone with the requested data value.

15. The method of claim 14, further comprising generating the data associating with the second set of metadata the one or more corresponding values in the first set of metadata.

16. The method of claim 14, wherein the requested data value comprises a first requested data value, and further comprising:

receiving a read operation to read a second requested data value from the clone; and

in response to the read operation, searching the second set of metadata associated with the clone and in the event that the second requested data value is not found in the second set of metadata, searching the snapshot included in the first set of metadata associated with the source data.

17. The method of claim 16, in the event the second requested data value is not found in the snapshot included in the first set of metadata associated with the source data, the snapshot comprises a first snapshot and further comprising searching a second snapshot included in the first set of metadata associated with the source data, the second snapshot being older than the first snapshot.

18. The method of claim 14, wherein the snapshot comprises a first snapshot and further comprising:

determining that requested data associated with a second snapshot meets a selective promotion criterion, wherein the second snapshot is older than the first snapshot;

generating a piece of metadata associated with the requested data based on one or more selective promotion parameters; and

associating the piece of metadata with the first snapshot.

19. The method of claim 18, wherein the selective promotion parameters comprise one or more of the following:

whether the piece of metadata to be generated to reference the requested data references a physical location of the requested data or an existing mapping to the requested data;

whether to promote only mapping associated with the requested data that was previously requested or to promote mappings associated with the requested data that was previously requested in addition to data that is in temporal locality, spatial locality, or grouped with the requested data based on another characteristic;

whether the requested data should be promoted to a shared snapshot or to a live index of the clone, and how long the requested data is to be promoted for.

20. The method of claim 14, further comprising:

determining that a snapshot creation event has occurred;

determining that no changes have been made to the source data since creation of a previous snapshot associated with the source data;

creating a label associated with a new snapshot; and

linking the label associated with the new snapshot to the previous snapshot associated with the source data, wherein the label is configured to redirect a search of the new snapshot to the linked previous snapshot associated with the source data.

21. The method of claim 14, further comprising:

determining to delete a snapshot to be deleted; and

merging a mapping of the snapshot to be deleted associated with not overwritten data with a next younger snapshot.

22. The method of claim 14, wherein the second set of metadata associated with the clone does not replicate mappings included in the first set of metadata associated with the source data.

23. A computer program product for storage, the computer program product being embodied in a non-transitory computer readable storage medium and comprising computer instructions for:

receiving a request to create a clone of a snapshot included in a first set of metadata associated with a source data;

in response to the request to create the clone of the snapshot;

generating a second set of metadata for the clone, wherein the second set of metadata associated with the clone includes, for at least some metadata values, a data associating with the second set of metadata one or more corresponding values in the first set of metadata associated with the source data,

wherein the second set of metadata for the clone comprises a first active index and a second active index, wherein the first active index is configured to store one or more data values that are written to the clone since a creation of a previous snapshot associated with the clone, wherein the second active index is configured to store one or more data values that at least partially reflect a current state of the clone; and

modifying a stored reference clone value associated with the snapshot included in the first set of metadata associated with the source data, wherein the stored reference clone value is used to determine whether to delete the snapshot;

receiving a write operation to write a requested data value to the clone; and

in response to the write operation, updating the first active index and the second active index associated with the clone with the requested data value.