Metadata favored replication in active topologies

• US 10,387,449 B2
• Filed: 05/30/2014
• Issued: 08/20/2019
• Est. Priority Date: 05/30/2014
• Status: Active Grant

• Current Assignee: SABIC Global Technologies BV (Government of Saudi Arabia), Hitachi Vantara, LLC (Hitachi, Ltd.)
• Original Assignee: Hitachi Vantara Corporation (Hitachi, Ltd.)
• Inventors: Isherwood, Benjamin, Grimaldi, Kevin Canuette
• Primary Examiner(s): Choi, Yuk Ting
• Application Number:
  US15/103,892
• Publication Number:
  US 20160321338A1
• Time in Patent Office: 1,908 Days
• CPC Class Codes: G06F 11/2094 Redundant storage or storage space G06F11/2056 takes precedence
• CPC Class Codes: G06F 11/2097 maintaining the standby controller/processing unit updated initialisation or re-synchronisation thereof G06F11/1658 and subgroups
• CPC Class Codes: G06F 16/00 Information retrieval; Database structures therefor; File system structures therefor
• CPC Class Codes: G06F 16/27 Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
• CPC Class Codes: H04L 67/1095 Replication or mirroring of data, e.g. scheduling or transport for data synchronisation between network nodes
• CPC Class Codes: H04L 67/1097 for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]
• CPC Class Codes: H04L 67/56 Provisioning of proxy services store-and-forward switching systems in data switching networks H04L12/54

Abstract

A method and apparatus for metadata favored replication in an active/active topology. When replicating data objects between multiple storage systems, metadata is sent first while the associated data content is sent at a subsequent time. Metadata is stored and made visible on a target storage system thereby reserving the object path in a namespace to reduce collisions in the namespace. After metadata is transferred to the target storage system, the target system performs collision processing to determine if an object already exists in the namespace prior to receiving the transferred metadata. After collision processing is performed, the winning object is made visible for access on the target storage system regardless of whether the associated data content has been transferred to the target system.

Claims

1. A storage system, comprising:

a first storage system and a second storage system connected to a network,

wherein the first storage system includes a first central processing unit, a first memory, and a first storage device coupled to the first central processing unit and first memory, the first central processing unit is configured to manage a first namespace which contains data and metadata associated with the data, the data and associated metadata are stored in the first storage device,

wherein the second storage system includes a second central processing unit, a second memory, and a second storage device coupled to the second memory and the second processing unit, the second central processing unit is configured to manage the first namespace which contains data and metadata associated with the data, the data and associated metadata are stored in the second storage device,

wherein the first storage system is configured to;

receive a write request of first data,

store the first data and first metadata associated with the first data in the first storage device within the first namespace managed by the first central processing unit,

transfer the first metadata to the second storage system, and

wherein the second storage system is configured to;

receive the first metadata transferred from the first storage system,

determine whether there is a collision among the transferred first metadata and at least one second metadata of a plurality of second metadata previously stored in the second storage device within the first namespace, based on whether a first portion of the first metadata and a respectively corresponding first portion of the at least one second metadata are the same,

upon determining that there is no collision among the transferred first metadata and the plurality of second metadata, allow access to the first data associated with the transferred first metadata via the second storage system by a client of the second storage system without receiving the first data associated with the first metadata, and

wherein upon determining that there is a collision among the transferred first metadata and the at least one second metadata, allow access to the first data associated with the transferred first metadata via the second storage system by the client of the second storage system without receiving the first data associated with the first metadata upon determining that (1) second data associated with the at least one second metadata is not the same as the first data associated with the first metadata and upon determining that (2) an update time of the transferred first metadata is more recent than a corresponding update time of the at least one second metadata.

2. The storage system according to claim 1, wherein

the second storage system is configured to:

upon determining that there is a collision and upon determining that at least another portion of the first metadata and at least one second metadata are not the same, the second data and the first data are the same, and the update time of the first metadata is more recent than the corresponding update time of the at least one second metadata, update the update time of the at least one second metadata to that of the update time of the first metadata.

3. The storage system of claim 1, wherein

the first storage system and the second storage system are connected using a replication link over the network,

the metadata and data are transferred in batches using the replication link, and

the metadata includes custom metadata and system metadata.

4. The storage system of claim 1, wherein

the first portion of the first metadata and the respectively corresponding first portions of the previously stored second metadata are path names.

5. The storage system of claim 1, wherein

the second storage system is configured to:

upon determining that there is a collision and upon determining at least another portion of the first metadata and at least one second metadata are the same, send an acknowledgment message to the first storage system including instructions, and the instructions are generated based upon configuration settings of the second storage system.

7. The storage system according to claim 1, wherein

the second storage system sends an acknowledgment message to the first storage system after the first metadata is stored in the second storage device within the first namespace, the acknowledgment message includes instructions for sending the first data associated with the first metadata, and the instructions are generated based upon configuration settings of the second storage system, and

upon receiving the acknowledgment message, the first storage system evaluates the instructions to determine whether to transfer the first data content.

6. A storage system, comprising:

a first storage system and a second storage system connected to a network,

wherein the first storage system includes a first central processing unit, a first memory, and a first storage device coupled to the first central processing unit and first memory, the first central processing unit is configured to manage a first namespace which contains data and metadata associated with the data, the data and associated metadata are stored in the first storage device,

wherein the second storage system includes a second central processing unit, a second memory, and a second storage device coupled to the second memory and the second processing unit, the second central processing unit is configured to manage the first namespace which contains data and metadata associated with the data, the data and associated metadata are stored in the second storage device,

wherein the first storage system is configured to;

receive a write request of first data,

store the first data and first metadata associated with the first data in the first storage device within the first namespace managed by the first central processing unit,

transfer the first metadata to the second storage system, and

wherein the second storage system is configured to;

receive the first metadata transferred from the first storage system,

determine whether there is a collision among the transferred first metadata and at least one second metadata of a plurality of second metadata previously stored in the second storage device within the first namespace, based on whether a first portion of the first metadata and a respectively corresponding first portion of the at least one second metadata are the same, and

upon determining that there is no collision among the transferred first metadata and the plurality of second metadata, allow access to the first data associated with the transferred first metadata via the second storage system by a client of the second storage system without receiving the first data associated with the first metadata,

wherein the second storage system is configured to;

upon determining that there is a collision and upon determining that the second data associated with the at least one second metadata is not the same as the first data associated with the transferred first data, and the update time of the transferred first metadata is more recent than a corresponding update time of the at least one second metadata, the transferred first metadata is committed to the second storage system and referred to upon receiving a read request from the client to read the first data associated with the first metadata, and a flag indicating a collision is set in the at least one second metadata, and

upon determining there is a collision and upon determining that the update time of the at least one second metadata is more recent than a corresponding update time of the transferred first metadata, a flag indicating a collision is set in the transferred first metadata.

8. A storage method for a storage system comprising a first storage system and a second storage system connected to a network,

the first storage system includes a first central processing unit, a first memory, and a first storage device coupled to the first memory and the first central processing unit, the first central processing unit is configured to manage a first namespace which contains data and metadata associated with the data, the first data and metadata associated with the data is stored in the first storage device;

the second storage system includes a second central processing unit, a second memory, and a second storage device coupled to the second memory and the second processing unit, the second processing unit is configured to manage the first namespace which contains data and metadata associated with the data, the data and metadata associated with the data is stored in the second storage device, the method comprising the steps of;

receiving, by the first storage system, a write request of first data;

storing the first data and first metadata associated with the first data in the first storage device within the first namespace managed by the first central processing unit;

transferring the first metadata to the second storage system;

receiving, by the second storage system, the first metadata transferred from the first storage system;

determining, by the second storage system, whether there is a collision among the transferred first metadata and at least one second metadata of a plurality of second metadata previously stored in the second storage device within the first namespace, based on whether a first portion of the first metadata and a respectively corresponding first portion of the at least one second metadata are the same

upon determining that there is no collision among the transferred first metadata and the at least one second metadata, allowing access, by the second storage system, to the first data associated with the transferred first metadata via the second storage system by a client of the second storage system without receiving the first data associated with the first metadata; and

upon determining that there is a collision among the transferred first metadata and the at least one second metadata, allow access, by the second storage system, to the first data associated with the transferred first metadata via the second storage system by the client of the second storage system without receiving the first data associated with the first metadata upon determining that (1) second data associated with the at least one second metadata is not the same as the first data associated with the first metadata and upon determining that (2) an update time of the transferred first metadata is more recent than a corresponding update time of the at least one second metadata.

9. The storage method according to claim 8, further comprising the steps of:

the second storage system is configured to;

upon determining that there is a collision and upon determining that at least another portion of the first metadata and the at least one second metadata are not the same, the second data and the first data are the same, and the update time of the first metadata is more recent than the corresponding update time of the at least one second metadata, update the update time of the at least one second metadata to that of the update time of the first metadata.

10. The storage method according to claim 8, further comprising the steps of:

upon determining that there is a collision and upon determining that the update time of the at least one second metadata is more recent than a corresponding update time of the transferred first metadata, a flag indicating a collision is set in the transferred first metadata.

11. The storage method according to claim 8, wherein

the second storage system sends an acknowledgment message to the first storage system after the first metadata is stored in the second storage device within the first namespace, the acknowledgment message includes instructions for sending the first data associated with the first metadata, and the instructions are generated based upon configuration settings of the second storage system, and

upon receiving the acknowledgment message, the first storage system evaluates the instructions to determine whether to transfer the first data content.

12. The storage method according to claim 8, wherein

the first storage system and the second storage system are connected using a replication link over the network, and

the metadata and data are transferred in batches using the replication link.

13. The storage method according to claim 8, wherein

the first portion of the first metadata and the respectively corresponding first portions of the previously stored second metadata are path names.

14. The storage system of claim 8, further comprising the steps of:

upon determining that there is a collision and upon determining at least another portion of the first metadata and at least one second metadata are the same, sending an acknowledgment message to the first storage system including instructions, and the instructions are generated based upon configuration settings of the second storage system.