Coalescing Metadata for Mirroring to a Remote Storage Node in a Cluster Storage System
First Claim
1. A system for storing data and metadata from a local node to a remote node, the system comprising:
- the local node comprising a mirroring engine configured for;
receiving a plurality of data and metadata sets from a client, each received data or metadata set comprising a data or metadata request from the client to store the data or metadata set to the remote node;
producing a plurality of metadata chains, each metadata chain comprising a grouping of at least two received metadata requests;
storing each received metadata request to a metadata request (MR) data structure according to the time order the requests are received, the MR data structure for managing metadata chains, each metadata chain comprising a head metadata request comprising the earliest received metadata request in the metadata chain and a tail metadata request comprising the last received metadata request in the metadata chain; and
transmitting a representative metadata request in the metadata chain to the remote node for storage if the representative metadata request comprises a tail metadata request in the metadata chain.
0 Assignments
0 Petitions
Accused Products
Abstract
Described herein are a system and method for remote mirroring/copying data and metadata sets from a local node to a remote node that reduces the number of metadata sets that are mirrored. In some embodiments, the local node may coalesce metadata sets into metadata chains, each metadata chain comprising a grouping of two or more metadata sets. In some instances, a “representative” metadata set of a metadata chain may be selected for sending to the remote node for storing, wherein the other metadata sets of the metadata chain are not sent to the remote node. In these embodiments, the selected metadata set may represent all the metadata sets in the chain and be the only metadata set in the chain that is transmitted and stored to the remote node. As such, the network congestion between the local and remote nodes may be reduced.
-
Citations
21 Claims
-
1. A system for storing data and metadata from a local node to a remote node, the system comprising:
the local node comprising a mirroring engine configured for; receiving a plurality of data and metadata sets from a client, each received data or metadata set comprising a data or metadata request from the client to store the data or metadata set to the remote node; producing a plurality of metadata chains, each metadata chain comprising a grouping of at least two received metadata requests; storing each received metadata request to a metadata request (MR) data structure according to the time order the requests are received, the MR data structure for managing metadata chains, each metadata chain comprising a head metadata request comprising the earliest received metadata request in the metadata chain and a tail metadata request comprising the last received metadata request in the metadata chain; and transmitting a representative metadata request in the metadata chain to the remote node for storage if the representative metadata request comprises a tail metadata request in the metadata chain. - View Dependent Claims (2, 3, 4, 5, 6, 7)
-
8. A method for storing data and metadata from a local node to a remote node, the method comprising:
providing the local node comprising a mirroring engine configured for; receiving a plurality of data and metadata sets from a client, each received data or metadata set comprising a data or metadata request from the client to store the data or metadata set to the remote node; producing a plurality of metadata chains, each metadata chain comprising a grouping of at least two received metadata requests; storing each received metadata request to a metadata request (MR) data structure according to the time order the requests are received, the MR data structure for managing metadata chains, each metadata chain comprising a head metadata request comprising the earliest received metadata request in the metadata chain and a tail metadata request comprising the last received metadata request in the metadata chain; and transmitting a representative metadata request in the metadata chain to the remote node for storage if the representative metadata request comprises a tail metadata request in the metadata chain. - View Dependent Claims (9, 10, 11, 12, 13, 14)
-
15. A non-transitory computer readable medium having instructions stored thereon when executed by a processor, store data and metadata from a local node to a remote node, the non-transitory computer readable medium comprising instructions for:
on the local node, configuring a mirroring engine for; receiving a plurality of data and metadata sets from a client, each received data or metadata set comprising a data or metadata request from the client to store the data or metadata set to the remote node; producing a plurality of metadata chains, each metadata chain comprising a grouping of at least two received metadata requests; storing each received metadata request to a metadata request (MR) data structure according to the time order the requests are received, the MR data structure for managing metadata chains, each metadata chain comprising a head metadata request comprising the earliest received metadata request in the metadata chain and a tail metadata request comprising the last received metadata request in the metadata chain; and transmitting a representative metadata request in the metadata chain to the remote node for storage if the representative metadata request comprises a tail metadata request in the metadata chain. - View Dependent Claims (16, 17, 18, 19, 20, 21)
Specification