System and method for secure and reliable multi-cloud data replication

• US 8,762,642 B2
• Filed: 01/28/2010
• Issued: 06/24/2014
• Est. Priority Date: 01/30/2009
• Status: Active Grant

• Current Assignee: Emc IP Holding Company LLC (Dell Technologies Inc.)
• Original Assignee: TwinStrata, Inc. (Dell Technologies Inc.)
• Inventors: Bates, John W., Aldred, Mark
• Primary Examiner(s): Kindred, Alford W.
• Assistant Examiner(s): TSUI, DANIEL D
• Application Number:
  US12/695,250
• Publication Number:
  US 20100199042A1
• Time in Patent Office: 1,608 Days
• Field of Search: None
• US Class Current: 711/118
• CPC Class Codes: G06F 11/2064 while ensuring consistency
• CPC Class Codes: G06F 11/2076 Synchronous techniques
• CPC Class Codes: G06F 11/2087 with a common controller
• CPC Class Codes: G06F 12/0804 with main memory updating G06F12/0806 takes precedence
• CPC Class Codes: G06F 12/0866 for peripheral storage systems, e.g. disk cache
• CPC Class Codes: G06F 2201/82 Solving problems relating to consistency ensuring consistency in mirrored systems G06F11/2064
• CPC Class Codes: G06F 2212/263 Network storage, e.g. SAN or NAS
• CPC Class Codes: G06F 3/0614 Improving the reliability of storage systems
• CPC Class Codes: G06F 3/0623 in relation to content
• CPC Class Codes: G06F 3/065 Replication mechanisms
• CPC Class Codes: G06F 3/067 Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
• CPC Class Codes: H04L 63/0428 wherein the data content is protected, e.g. by encrypting or encapsulating the payload
• CPC Class Codes: H04L 63/10 for controlling access to devices or network resources
• CPC Class Codes: H04L 67/1095 Replication or mirroring of data, e.g. scheduling or transport for data synchronisation between network nodes
• CPC Class Codes: H04L 67/1097 for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]
• CPC Class Codes: H04L 67/288 Distributed intermediate devices, i.e. intermediate devices for interaction with other intermediate devices on the same level
• CPC Class Codes: H04L 67/289 Intermediate processing functionally located close to the data consumer application, e.g. in same machine, in same home or in same sub-network
• CPC Class Codes: H04L 67/5682 Policies or rules for updating, deleting or replacing the stored data
• CPC Class Codes: H04L 67/5683 Storage of data provided by user terminals, i.e. reverse caching
• CPC Class Codes: H04L 9/0625 with splitting of the data block into left and right halves, e.g. Feistel based algorithms, DES, FEAL, IDEA or KASUMI
• CPC Class Codes: H04L 9/088 : Usage controlling of secret information, e.g. techniques for restricting cryptographic keys to pre-authorized uses, different access levels, validity of crypto-period, different key- or password length, or different strong and weak cryptographic algorithms network architectures or network communication protocols for using time-dependent keys in a packet data network H04L63/068
• CPC Class Codes: H04L 9/0891 : Revocation or update of secret information, e.g. encryption key update or rekeying
• CPC Class Codes: H04L 9/0894 : Escrow, recovery or storing of secret information, e.g. secret key escrow or cryptographic key storage
• CPC Class Codes: H04L 9/321 : involving a third party or a trusted authority

Abstract

A multi-cloud data replication method includes providing a data replication cluster comprising at least a first host node and at least a first online storage cloud. The first host node is connected to the first online storage cloud via a network and comprises a server, a cloud array application and a local cache. The local cache comprises a buffer and a first storage volume comprising data cached in one or more buffer blocks of the local cache'"'"'s buffer. Next, requesting authorization to perform cache flush of the cached first storage volume data to the first online storage cloud. Upon receiving approval of the authorization, encrypting the cached first storage volume data in each of the one or more buffer blocks with a data private key. Next, assigning metadata comprising at lest a unique identifier to each of the one or more buffer blocks and then encrypting the metadata with a metadata private key. Next, transmitting the one or more buffer blocks with the encrypted first storage volume data to the first online cloud storage. Next, creating a sequence of updates of the metadata, encrypting the sequence with the metadata private key and then transmitting the sequence of metadata updates to the first online storage cloud.

Claims

1. A method for replicating data to one or more online storage clouds comprising:

providing a data replication cluster comprising at least a first host node and at least a first online storage cloud, wherein said first host node is connected to said first online storage cloud via a network and comprises a server, a cloud array application and a local cache and wherein said local cache comprises a buffer and a first storage volume and wherein said first storage volume comprises data cached in one or more buffer blocks of said local cache'"'"'s buffer;

entering a first request by a user to “

write”

data in said first storage volume;

receiving said first request by said server;

identifying said first storage volume'"'"'s internal structure and initiating processing of said first request;

verifying first authorization of user to “

write”

data in said first storage volume, and upon approval said first authorization, passing said first request to said local cache;

verifying second authorization of said user to “

write”

data in a specific region of said first storage volume by said local cache;

upon approval of said second authorization, determining if a specific extent exists in said local cache'"'"'s buffer for receiving said “

write”

data;

upon confirmation that said specific extent exists in said local cache'"'"'s buffer, storing said “

write”

data in said extent, wherein said extent comprises said one or more buffer blocks of said local cache'"'"'s buffer;

requesting authorization to perform cache flush of the cached first storage volume data to said first online storage cloud, by said cloud array application;

upon receiving approval of said authorization, encrypting said cached first storage volume data in each of said one or more buffer blocks with a data private key, by said cloud array application;

assigning metadata comprising at least a unique identifier to each of said one or more buffer blocks, by said cloud array application;

encrypting said metadata with a metadata private key, by said cloud array application;

transmitting said one or more buffer blocks with the encrypted first storage volume data to said first online cloud storage, by said cloud array application;

creating a sequence of updates of said metadata, encrypting said sequence with said metadata private key, by said cloud array application; and

transmitting said sequence of metadata updates to said first online storage cloud, by said cloud array application.

2. The method of claim 1 further comprising compressing said cached first storage volume data in each of said one or more buffer blocks.

3. The method of claim 1 wherein said metadata further comprise one of a volume index, a region index, page index, maximum number of pages in a region, number of valid pages in a region, size of a page, file name of a page or a logical time stamp.

4. The method of claim 1 further comprising upon receiving approval of said authorization creating a copy of each of said one or more buffer blocks comprising said cached first storage volume data.

5. The method of claim 1 wherein said one or more buffer blocks with the encrypted first storage volume data are transmitted to said first online cloud storage according to a RAID-1 replication algorithm.

6. The method of claim 1 wherein said sequence of metadata is transmitted to said first online cloud storage according to a RAID-1 replication algorithm.

7. The method of claim 1, further comprising analyzing said one or more buffer blocks to determine if data were previously written in said buffer blocks and upon determination that data were previously written in said buffer blocks, backfilling said previously written data by reading said previously written data from said online storage cloud.

8. The method of claim 1, further comprising upon confirmation that said specific extent does not exist in said local cache, allocating a new extent for receiving said “

write”

data.

9. The method of claim 1, further comprising upon confirmation that said specific extent does not exist in said local cache, freeing an existing extent for receiving said “

write”

data.

10. The method of claim 1, wherein said identifying and verifying are performed by said cloud array application.

11. The method of claim 1 wherein said first host node further comprises a local storage device and said method further comprises flushing said one or more buffer blocks with said cached first storage volume data to said local storage device.

12. The method of claim 1, further comprising acknowledging processing of said first request.

13. The method of claim 1 further comprising acknowledging completion of data replication to said first online storage cloud.

14. The method of claim 1 wherein said first online storage cloud comprises a cloud storage device and said method further comprises storing said first storage volume data in said cloud storage device.

15. The method of claim 1, wherein said first request is entered via a block based interface.

16. A method for replicating data to one or more online storage clouds comprising:

providing a data replication cluster comprising at least a first host node and at least a first online storage cloud, wherein said first host node is connected to said first online storage cloud via a network and comprises a server, a cloud array application and a local cache and wherein said local cache comprises a buffer and a first storage volume comprising data cached in one or more buffer blocks of said local cache'"'"'s buffer;

entering a first request by a user to “

write”

data in said first storage volume;

receiving said first request by said server;

identifying said first storage volume'"'"'s internal structure and initiating processing of said first request;

verifying first authorization of said user to “

write”

data in said first storage volume, and upon approval of said first authorization, passing said first request to said local cache;

verifying second authorization of said user to “

write”

data in a specific region of said first storage volume by said local cache;

upon approval of said second authorization, determining if a specific extent exists in said local cache'"'"'s buffer for receiving said “

write”

data;

upon confirmation that said specific extent exists in said local cache'"'"'s buffer, storing said “

write”

data in said extent, wherein said extent comprises one or more buffer blocks of said local cache'"'"'s buffer;

requesting third authorization to perform cache flush of the cached first storage volume data to said first online storage cloud;

upon receiving said third authorization, creating a copy of each of said buffer blocks with said cached first storage volume data, compressing said cached first storage volume data in each of said buffer blocks and encrypting said cached first storage volume data in each of said buffer blocks with a data private key;

assigning metadata comprising at least a unique identifier and a logical time stamp to each of said buffer blocks;

encrypting said metadata with a metadata private key;

transmitting said encrypted buffer blocks to said first online cloud storage;

creating a sequence of updates of said metadata, encrypting said sequence with said metadata private key; and

transmitting said sequence of metadata updates to said first online storage cloud.

17. A system for replicating data to one or more online storage clouds comprising:

a data replication cluster comprising at least a first host node and at least a first online storage cloud, wherein said first host node is connected to said first online storage cloud via a network and comprises a server, a cloud array application and a local cache and wherein said local cache comprises a buffer and a first storage volume comprising data cached in one or more buffer blocks of said local cache'"'"'s buffer;

means for entering a first request by a user to “

write”

data in said first storage volume;

means for receiving said first request by said server;

means for identifying said first storage volume'"'"'s internal structure;

means for initiating processing of said first request;

means for verifying first authorization of said user to “

write”

data in said first storage volume, and upon approval of said first authorization, passing said first request to said local cache;

means for verifying second authorization of said user to “

write”

data in a specific region of said first storage volume by said local cache;

means for determining if a specific extent exists in local cache'"'"'s buffer for receiving said “

write”

data;

means for storing “

write”

in extent;

wherein said cloud array application is configure to request authorization to perform cache flush of the cached first storage volume data to said first online storage cloud;

wherein said cloud array application is configure to encrypt said cached first storage volume data in each of said one or more buffer blocks with a data private key;

wherein said cloud array application is configure to assign metadata comprising at least a unique identifier to each of said one or more buffer blocks;

wherein said cloud array application is configure to encrypt said metadata with a metadata private key;

wherein said cloud array application is configure to transmit said one or more buffer blocks with the encrypted first storage volume data to said first online cloud storage;

wherein said cloud array application is configure to create a sequence of updates of said metadata;

wherein said cloud array application is configure to encrypt said sequence with said metadata private key; and

wherein said cloud array application is configure to transmit said sequence of metadata updates to said first online storage cloud.

18. The system of claim 17, wherein said means for entering a first request by a user to “

write”

data in said first storage volume comprise a block based interface.

19. The system of claim 17 further comprising means for analyzing said one or more buffer blocks to determine if data were previously written in said buffer blocks and means for backfilling said previously written data by reading said previously written data from said online storage cloud.

20. The system of claim 17 wherein said host node further comprises a local storage device and said system further comprises means for flushing said one or more buffer blocks with said cached first storage volume data to said local storage device.