Distributed filesystem atomic flush transactions
First Claim
1. A networked computing system where a file access request has been passed between an upstream site and a file service proxy cache node, the networked computing system comprising:
- an upstream site, the upstream site including;
one or more hardware processors;
system memory coupled to the one or more hardware processors, the system memory storing instructions that are executable by the one or more hardware processors;
the one or more hardware processors executing the instructions stored in the system memory to flush modified file data downstream towards an origin file server, including the following;
acquire a site channel containing modified file data, the site channel bound into a single integrated channel traversing one or more file service proxy cache nodes through which the flush request will pass; and
prepare a flush request, the flush request;
being an atomic operation;
including a file access request, wherein the file access request includes making modifications to a file located on the origin file server; and
including modified file data to save to a stable memory, wherein;
the modified file data is sent in multiple network flush requests; and
each flush request is uniquely identified; and
the last flush request of the atomic operation is identified with a flag referred to as DATA_COMPLETE; and
each flush request includes at least one modified file extent;
the one or more file service proxy cache nodes, each file service proxy cache node;
being a network node located between the upstream site and the origin file server node; and
including;
one or more hardware processors;
system memory coupled to the one or more hardware processors, the system memory storing instructions that are executable by the one or more hardware processors;
the one or more hardware processors executing the instructions stored in the system memory to complete a flush request, including the following;
receive the flush request from the upstream site;
store each received modified file extent in a first shadow extent referenced by the site channel, wherein the shadow extent is a temporary buffer;
dispatch the received flush request containing the modified file extents to a downstream node, wherein the downstream node includes at least one of;
a file service proxy cache node closer to the origin file server node;
or
the origin file server node; and
the origin file server node including;
one or more hardware processors;
system memory coupled to the one or more hardware processors, the system memory storing instructions that are executable by the one or more hardware processors;
the one or more hardware processors executing the instructions stored in the system memory to complete a flush request, including the following;
receive the flush request from the file service proxy cache node located upstream of the origin file server node;
store each received modified file extent in a second shadow extent referenced by the site channel, wherein the shadow extent is a temporary buffer;
if the flush request contains a DATA_COMPLETE flag;
write all shadow extents to the origin filesystem; and
if all shadow extents are successfully written to the origin filesystem;
replace all extents that have shadow extents with their respective shadow extent; and
send a response to the file service proxy cache node located upstream of the origin file server node that includes a flag referred to as ALL_DATA_RECEIVED that indicates all data was successfully received; and
send a first status code, wherein the first status code indicates the successful completion of the flush request;
if all shadow extents are not successfully written to the origin filesystem;
discard all shadow extents; and
send a response to the file service proxy cache node located upstream of the origin file server node that includes a second status code, wherein the second status code indicates the failure of the flush request;
if the flush request does not contain a DATA_COMPLETE flag;
send a response to the file service proxy cache node located upstream of the origin file server node that includes the first status code, wherein the first status code indicates the successful completion of the flush request; and
wherein each of the file service proxy cache nodes and the upstream site;
receive a flush request response from the downstream node, the response including a status code, wherein the status code includes at least one of;
the first status code;
orthe second status code;
replace all extents that have shadow extents with their respective shadow extent when the status code is the first status code;
delete all modified file data saved to the shadow extents when the status code is the second status code; and
wherein each of the file service proxy cache nodes;
send a response to an upstream node, wherein;
the upstream node includes at least one of;
a file service proxy cache node closer to the upstream site;
orthe upstream site; and
the response includes at least one of;
the first status code;
orthe second status code.
0 Assignments
0 Petitions
Accused Products
Abstract
Large scale high performance file proxy caching sites may be configured to coalesce many client write operations into one very large assemblage of modified file data. At some point the caching site will flush all modified file data downstream towards the origin file server. In some instances the amount of modified data being flushed may be more than can be transferred in a single network request. When multiple network requests are required, the consistency guarantee provided by many filesystems requires that the file either be updated with the data contained in all of the network requests or not be modified at all. In addition, once the first flush request is processed no other file read or write requests can be serviced until the last flush request has been processed. This document discloses methods for performing atomic multi-request flush operations within a large geographically distributed filesystem environment.
-
Citations
8 Claims
-
1. A networked computing system where a file access request has been passed between an upstream site and a file service proxy cache node, the networked computing system comprising:
-
an upstream site, the upstream site including; one or more hardware processors; system memory coupled to the one or more hardware processors, the system memory storing instructions that are executable by the one or more hardware processors; the one or more hardware processors executing the instructions stored in the system memory to flush modified file data downstream towards an origin file server, including the following; acquire a site channel containing modified file data, the site channel bound into a single integrated channel traversing one or more file service proxy cache nodes through which the flush request will pass; and prepare a flush request, the flush request; being an atomic operation; including a file access request, wherein the file access request includes making modifications to a file located on the origin file server; and including modified file data to save to a stable memory, wherein;
the modified file data is sent in multiple network flush requests; and
each flush request is uniquely identified; and
the last flush request of the atomic operation is identified with a flag referred to as DATA_COMPLETE; and
each flush request includes at least one modified file extent;the one or more file service proxy cache nodes, each file service proxy cache node; being a network node located between the upstream site and the origin file server node; and including; one or more hardware processors; system memory coupled to the one or more hardware processors, the system memory storing instructions that are executable by the one or more hardware processors; the one or more hardware processors executing the instructions stored in the system memory to complete a flush request, including the following; receive the flush request from the upstream site; store each received modified file extent in a first shadow extent referenced by the site channel, wherein the shadow extent is a temporary buffer; dispatch the received flush request containing the modified file extents to a downstream node, wherein the downstream node includes at least one of;
a file service proxy cache node closer to the origin file server node;
or
the origin file server node; andthe origin file server node including; one or more hardware processors; system memory coupled to the one or more hardware processors, the system memory storing instructions that are executable by the one or more hardware processors; the one or more hardware processors executing the instructions stored in the system memory to complete a flush request, including the following; receive the flush request from the file service proxy cache node located upstream of the origin file server node; store each received modified file extent in a second shadow extent referenced by the site channel, wherein the shadow extent is a temporary buffer; if the flush request contains a DATA_COMPLETE flag;
write all shadow extents to the origin filesystem; and
if all shadow extents are successfully written to the origin filesystem;
replace all extents that have shadow extents with their respective shadow extent; and
send a response to the file service proxy cache node located upstream of the origin file server node that includes a flag referred to as ALL_DATA_RECEIVED that indicates all data was successfully received; and
send a first status code, wherein the first status code indicates the successful completion of the flush request;
if all shadow extents are not successfully written to the origin filesystem;
discard all shadow extents; and
send a response to the file service proxy cache node located upstream of the origin file server node that includes a second status code, wherein the second status code indicates the failure of the flush request;if the flush request does not contain a DATA_COMPLETE flag;
send a response to the file service proxy cache node located upstream of the origin file server node that includes the first status code, wherein the first status code indicates the successful completion of the flush request; andwherein each of the file service proxy cache nodes and the upstream site; receive a flush request response from the downstream node, the response including a status code, wherein the status code includes at least one of; the first status code;
orthe second status code; replace all extents that have shadow extents with their respective shadow extent when the status code is the first status code; delete all modified file data saved to the shadow extents when the status code is the second status code; and wherein each of the file service proxy cache nodes; send a response to an upstream node, wherein; the upstream node includes at least one of; a file service proxy cache node closer to the upstream site;
orthe upstream site; and the response includes at least one of; the first status code;
orthe second status code. - View Dependent Claims (2, 3, 4, 5, 6, 7, 8)
-
Specification