Managing transformed snapshots in a storage system

• US 10,430,434 B2
• Filed: 01/20/2017
• Issued: 10/01/2019
• Est. Priority Date: 01/20/2016
• Status: Active Grant

• Current Assignee: Delphix Corporation (Perforce Software Incorporated)
• Original Assignee: Delphix Corporation (Perforce Software Incorporated)
• Inventors: Sun, Hubert Ken, Siden, Christopher G., Cackett, Kyle
• Primary Examiner(s): Thammavong, Prasith
• Application Number:
  US15/411,471
• Publication Number:
  US 20170206016A1
• Time in Patent Office: 984 Days
• CPC Class Codes: G06F 16/258 Data format conversion from or to a database
• CPC Class Codes: G06F 3/0607 by facilitating the process of upgrading existing storage systems, e.g. for improving compatibility between host and storage device
• CPC Class Codes: G06F 3/0661 Format or protocol conversion arrangements
• CPC Class Codes: G06F 3/067 Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]

Abstract

A storage system performs transformations of data stored as source snapshots to obtain transformed snapshots. The storage system stores relations between source snapshots, for example, parent child relationships. The storage system analyzes relationships between source snapshots to determine relationships between transformed snapshots. The storage system creates a transformed snapshot based on a source snapshot. The storage system traverses the graph representing source snapshots to identify ancestor or descendants of the source snapshots that were previously transformed. The storage system determines the transformed snapshots corresponding to the ancestor and descendant source snapshots and relates them to the newly created transformed snapshot.

Claims

1. A method for transforming snapshots, the method comprising:

storing, in a storage system, a plurality of source snapshots, wherein a source snapshot shares data blocks with one or more other source snapshots;

maintaining links representing relationships between source snapshots from the plurality of source snapshots, each link relating a snapshot with another snapshot created using data of the snapshot;

identifying a set of previously created transformed snapshots associated with a transform operation and stored in the storage system, each transformed snapshot of the set of transformed snapshots obtained by performing the transform operation on data of a respective source snapshot of the plurality of source snapshots to change a format of the respective source snapshot;

receiving a request to perform the transform operation on a first source snapshot of the plurality of source snapshots;

generating a first transformed snapshot by transforming data of the first source snapshot;

determining relations between the first transformed snapshot and the previously created transformed snapshots, the determining comprising;

identifying a second source snapshot of the plurality of source snapshots, the second source snapshot connected to the first source snapshot via one or more links, and wherein the second source snapshot was transformed to obtain a second transformed snapshot of the previously created transformed snapshots; and

responsive to identifying the second source snapshot, storing at least one link between the first transformed snapshot and the second transformed snapshot based upon the identified one or more links connecting the first source snapshot and the second source snapshot; and

sending information describing relations between one or more transformed snapshots for presentation via a user interface.

2. The method of claim 1, wherein identifying the second source snapshot comprises traversing links from the first source snapshot and selecting a nearest source snapshot that was previously transformed to generate a transformed snapshot.

3. The method of claim 2, wherein the second source snapshot is connected to the first source snapshot by at least a plurality of links.

4. The method of claim 1, wherein a link is a parent-child link that associates a parent source snapshot with a child source snapshot, wherein the child source snapshot was created from the parent source snapshot.

5. The method of claim 4, wherein identifying the second source snapshot comprises traversing the parent-child links starting from the first source snapshot to identify a lowest ancestor of the first source snapshot that was previously transformed to generate a transformed snapshot.

6. The method of claim 4, wherein identifying the second source snapshot comprises traversing the parent-child links starting from the first source snapshot to identify a highest descendant of the first source snapshot that was previously transformed to generate a transformed snapshot.

7. The method of claim 4, wherein identifying the second source snapshot comprises traversing the parent-child links starting from the first source snapshot to identify a plurality of descendants, wherein each of the plurality of descendants is a highest descendant of the first source snapshot along a particular branch that was previously transformed to generate a transformed snapshot.

8. The method of claim 1, wherein the transform operation masks at least a subset of data of a source snapshot to create a transformed snapshot storing masked data.

9. The method of claim 1, wherein the transform operation receives as input a source snapshot storing data that can be processed by a first version of an application and generates a transformed snapshot storing data that can be processed by a second version of the application.

10. The method of claim 9, wherein the application is a database system.

11. The method of claim 1, wherein the transform operation receives as input a source snapshot storing data that can be processed by an application configured to execute on a first operating system and generates a transformed snapshot storing data that can be processed by an application configured to execute on a second operating system.

12. The method of claim 1, wherein the transform operation receives as input a source snapshot storing data formatted for processing by a first application and generates a transformed snapshot storing data that is formatted for processing by a second application.

13. The method of claim 1, wherein transforming data of a source snapshot comprises:

creating a copy of the source snapshot;

creating a virtual database representing data of the copy of the source snapshot;

executing the transform operation to modify the virtual database, the transform operation comprising one or more database commands; and

using the data of the modified virtual database as the transformed snapshot corresponding to the source snapshot.

14. A non-transitory computer-readable storage medium storing instructions for:

storing, in a storage system, a plurality of source snapshots, wherein a source snapshot shares data blocks with one or more other source snapshots;

maintaining links representing relationships between source snapshots from the plurality of source snapshots, each link relating a snapshot with another snapshot created using data of the snapshot;

identifying a set of previously created transformed snapshots associated with a transform operation and stored in the storage system, each transformed snapshot of the set of transformed snapshots obtained by performing the transform operation on data of a respective source snapshot of the plurality of source snapshots to change a format of the respective source snapshot;

receiving a request to perform the transform operation on a first source snapshot of the plurality of source snapshots;

generating a first transformed snapshot by transforming data of the first source snapshot;

determining relations between the first transformed snapshot and the previously created transformed snapshots, the determining comprising;

identifying a second source snapshot of the plurality of source snapshots, the second source snapshot connected to the first source snapshot via one or more links, and wherein the second source snapshot was transformed to obtain a second transformed snapshot of the previously created transformed snapshots; and

responsive to identifying the second source snapshot, storing at least one link between the first transformed snapshot and the second transformed snapshot based upon the identified one or more links connecting the first source snapshot and the second source snapshot; and

sending information describing relations between one or more transformed snapshots for presentation via a user interface.

15. The non-transitory computer-readable storage medium of claim 14, wherein identifying the second source snapshot comprises traversing the links starting from the first source snapshot and selecting the nearest source snapshot that was previously transformed to generate a transformed snapshot.

16. The non-transitory computer-readable storage medium of claim 14, wherein a link is a parent-child link that associates a parent source snapshot with a child source snapshot, wherein the child source snapshot was created from the parent source snapshot.

17. The non-transitory computer-readable storage medium of claim 16, wherein identifying the second source snapshot comprises traversing the parent-child links starting from the first source snapshot to identify a lowest ancestor of the first source snapshot that was previously transformed to generate a transformed snapshot.

18. The non-transitory computer-readable storage medium of claim 16, wherein identifying the second source snapshot comprises traversing the parent-child links starting from the first source snapshot to identify a highest descendant of the first source snapshot that was previously transformed to generate a transformed snapshot.

19. The non-transitory computer-readable storage medium of claim 16, wherein identifying the second source snapshot comprises traversing the parent-child links starting from the first source snapshot to identify a plurality of descendants, wherein each of the plurality of descendants is a highest descendant of the first source snapshot along a particular branch that was previously transformed to generate a transformed snapshot.

20. A computer system, comprising:

an electronic processors; and

a non-transitory computer-readable storage medium storing instructions for;

storing, in a storage system, a plurality of source snapshots, wherein a source snapshot shares data blocks with one or more other source snapshots;

maintaining links representing relationships between source snapshots from the plurality of source snapshots, each link relating a snapshot with another snapshot created using data of the snapshot;

identifying a set of previously created transformed snapshots associated with a transform operation and stored in the storage system, each transformed snapshot of the set of transformed snapshots obtained by performing the transform operation on data of a respective source snapshot of the plurality of source snapshots to change a format of the respective source snapshot;

receiving a request to perform the transform operation on a first source snapshot of the plurality of source snapshots;

generating a first transformed snapshot by transforming data of the first source snapshot;

determining relations between the first transformed snapshot and the previously created transformed snapshots, the determining comprising;

identifying a second source snapshot of the plurality of source snapshots, the second source snapshot connected to the first source snapshot via one or more links, and wherein the second source snapshot was transformed to obtain a second transformed snapshot of the previously created transformed snapshots; and

responsive to identifying the second source snapshot, storing at least one link between the first transformed snapshot and the second transformed snapshot based upon the identified one or more links connecting the first source snapshot and the second source snapshot; and

sending information describing relations between one or more transformed snapshots for presentation via a user interface.

21. A method for transforming data, the method comprising:

storing, in a storage system, a plurality of source entities, wherein a source entity represents data stored as a plurality of data blocks in the storage system, wherein an entity shares data blocks with one or more other entities represented in the storage system;

maintaining links representing relationships between entities from the plurality of source entities, each link relating an entity with another entity created using data of the entity;

identifying a set of previously created transformed entities associated with a transform operation and stored in the storage system, each transformed entity of the set of transformed entities obtained by performing the transform operation on data of a respective source entity of the plurality of source entities to change a format of the respective source entity;

receiving a request to perform the transform operation on a first source entity of the plurality of source entities;

generating a first transformed entity by transforming data of the first source entity;

determining relations between the first transformed entity and the previously created transformed entities, the determining comprising;

identifying a second source entity of the plurality of source entities, the second source entity connected to the first source entity via one or more links, and wherein the second source entity was transformed to obtain a second transformed entity of the previously created transformed entities; and

responsive to identifying the second source entity, storing at least one link between the first transformed entity and the second transformed entity based upon the identified one or more links connecting the first source entity and the second source entity; and

sending information describing relations between one or more transformed entity for presentation via a user interface.