Replication to the cloud

• US 10,545,987 B2
• Filed: 12/19/2014
• Issued: 01/28/2020
• Est. Priority Date: 12/19/2014
• Status: Active Grant

• Current Assignee: Pure Storage, Inc.
• Original Assignee: Pure Storage, Inc.
• Inventors: Borowiec, Benjamin, Miller, Ethan L., Hodgson, Steve, Bernat, Andrew R., Ramanarayanan, Ganesh, Sharpe, Malcolm, Driscoll, Alan S.
• Primary Examiner(s): Ehichioya, Irete F
• Assistant Examiner(s): Johnson, Johnese T
• Application Number:
  US14/577,110
• Publication Number:
  US 20160196320A1
• Time in Patent Office: 1,866 Days
• Field of Search: 707624
• CPC Class Codes: G06F 11/1451 by selection of backup contents
• CPC Class Codes: G06F 11/1453 using de-duplication of the data
• CPC Class Codes: G06F 11/1456 Hardware arrangements for backup
• CPC Class Codes: G06F 11/1458 Management of the backup or restore process
• CPC Class Codes: G06F 16/1744 using compression, e.g. sparse files
• CPC Class Codes: G06F 16/1748 De-duplication implemented within the file system, e.g. based on file segments de-duplication techniques in storage systems for the management of data blocks G06F3/0641
• CPC Class Codes: G06F 16/27 Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
• CPC Class Codes: G06F 2201/84 Using snapshots, i.e. a logical point-in-time copy of the data
• CPC Class Codes: G06F 3/0608 Saving storage space on storage systems
• CPC Class Codes: G06F 3/061 Improving I/O performance
• CPC Class Codes: G06F 3/0617 in relation to availability
• CPC Class Codes: G06F 3/065 Replication mechanisms
• CPC Class Codes: G06F 3/067 Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]

Abstract

Systems, methods, and computer readable storage mediums for replicating data to the cloud. A first dataset is replicated from a first storage subsystem to the cloud, with only the changes made to the first dataset since a previous replication event being replicated to the cloud. Additionally, the data may be deduplicated and compressed before being replicated from the first storage subsystem to the cloud. In some cases, the first dataset may be replicated to the cloud in a stream format which is not directly usable and then, at a later point in time, loaded onto a second storage subsystem from the cloud.

Claims

1. A storage system for replicating data to a cloud-based server, the system configured to:

by a storage controller,

take a first snapshot of a local dataset stored on one or more storage devices,

take a second snapshot of the local dataset;

wherein the second snapshot includes only changes made to the local dataset since the first snapshot was taken;

deduplicate and compress the second snapshot; and

send to the cloud-based server, the deduplicated and compressed second snapshot in a stream of data including first data that references second data within the stream, the stream further including third data that references previously replicated data already stored.

2. The storage system as recited in claim 1, wherein the cloud-based server is configured to apply the deduplicated and compressed data to a remote dataset.

3. The storage system as recited in claim 1, wherein the system is further configured to send an identifier (ID) to the cloud-based server, wherein the ID is associated with the second snapshot of the local dataset.

4. The storage system as recited in claim 1, wherein the system is further configured to encrypt the deduplicated and compressed second snapshot prior to sending the deduplicated and compressed second snapshot to the cloud-based server.

5. The storage system as recited in claim 1, wherein the cloud-based server is further configured to:

select, from a plurality of systems, a second system for restoring the remote dataset responsive to an indication of failure of the local dataset; and

send the stream to the second system for restoration.

6. A method comprising:

performing, by a storage controller;

taking a first snapshot of a local dataset stored on one or more storage devices,

taking a second snapshot of the local dataset;

wherein the second snapshot includes only changes made to the local dataset since the first snapshot was taken;

deduplicating and compressing the second snapshot; and

sending to the cloud-based server, the deduplicated and compressed second snapshot in a stream of data including first data that references second data within the stream, the stream further including third data that references previously replicated data already stored.

7. The method as recited in claim 6, further comprising applying the deduplicated and compressed data to a remote dataset at the cloud-based server.

8. The method as recited in claim 6, further comprising sending an identifier (ID) to the cloud-based server, wherein the ID is associated with the second snapshot of the local dataset.

9. The method as recited in claim 6, further comprising encrypting the deduplicated and compressed second snapshot prior to sending the deduplicated and compressed snapshot to the cloud-based server.

10. The method as recited in claim 6, further comprising:

selecting, from a plurality of storage systems, a second storage system by the cloud-based server for restoring the remote dataset responsive to an indication of failure of the local dataset; and

sending the stream to the second storage system from the cloud-based server for restoration.

11. A non-transitory computer readable storage medium storing program instructions, wherein the program instructions are executable by a processor to:

by a storage controller;

take a first snapshot of a local dataset stored on one or more storage devices,

take a second snapshot of the local dataset;

wherein the second snapshot includes only changes made to the local dataset since the first snapshot was taken;

deduplicate and compress the second snapshot; and

send to the cloud-based server, the deduplicated and compressed second snapshot in a stream of data including first data that references second data within the stream, the stream further including third data that references previously replicated data already stored.

12. The non-transitory computer readable storage medium as recited in claim 11, wherein the program instructions are further executable by a processor to apply the deduplicated and compressed data to a remote dataset at the cloud-based server.

13. The non-transitory computer readable storage medium as recited in claim 11, wherein the program instructions are further executable by a processor to send an identifier (ID) to the cloud-based server, wherein the ID is associated with the second snapshot of the local dataset.

14. The non-transitory computer readable storage medium as recited in claim 11, wherein the program instructions are further executable by a processor to:

select, from a plurality of storage systems, a second storage system by the cloud-based server for restoring the remote dataset responsive to an indication of failure of the local dataset; and

send the stream to the second storage system from the cloud-based server for restoration.