HYBRID SEGMENT-ORIENTED FILE SERVER AND WAN ACCELERATOR

• US 20080281908A1
• Filed: 05/08/2008
• Published: 11/13/2008
• Est. Priority Date: 05/08/2007
• Status: Active Grant

• Current Assignee: Riverbed Technology, LLC (Riverbed Technology Incorporated)
• Original Assignee: Riverbed Technology Incorporated
• Inventors: Martin, John, McCanne, Steven, Parab, Nitin
• Granted Patent: US 9,930,099 B2
• Time in Patent Office: Days
• US Class Current: 709/203
• CPC Class Codes: G06F 11/1471 involving logging of persistent data for recovery
• CPC Class Codes: G06F 16/00 Information retrieval; Database structures therefor; File system structures therefor
• CPC Class Codes: G06F 16/10 File systems; File servers
• CPC Class Codes: G06F 16/13 File access structures, e.g. distributed indices arrangements of input from, or output to, record carriers G06F3/06
• CPC Class Codes: G06F 16/172 Caching, prefetching or hoarding of files
• CPC Class Codes: G06F 16/1748 De-duplication implemented within the file system, e.g. based on file segments de-duplication techniques in storage systems for the management of data blocks G06F3/0641
• CPC Class Codes: G06F 21/6218 to a system of files or objects, e.g. local or distributed file system or database
• CPC Class Codes: H04L 67/06 specially adapted for file transfer, e.g. file transfer protocol [FTP]

Abstract

In a network including WAN accelerators and segment-oriented file servers, a method comprises responding to a client request to manipulate a file via a network file protocol by receiving a first request at a first WAN accelerator, wherein the request is a request to open a file located at a file server that is a segment-oriented file server, sending a local request for the file, corresponding to the first request, from the WAN accelerator to the file server, using a segment-aware network request protocol, returning at least a portion of the requested file in the form of a representation of a data map corresponding to the at least a portion of the requested file stored on the file server and using a data map for reconstruction of the requested file.

Claims

1. In a network including WAN accelerators and segment-oriented file servers, a method of responding to a client request to manipulate a file via a network file protocol comprising:

receiving a first request at a first WAN accelerator, wherein the request is a request to open a file located at a file server that is a segment-oriented file server;

sending a local request for the file, corresponding to the first request, from the WAN accelerator to the file server, using a segment-aware network request protocol;

returning at least a portion of the requested file in the form of a representation of a data map corresponding to the at least a portion of the requested file stored on the file server;

retrieving, at the WAN accelerator, the data map for said file server; and

reconstructing data at the WAN accelerator corresponding to the requested file from segments stored in a segment store of the WAN accelerator.

2. The method of claim 1, further comprising pairing WAN accelerators for request-response sessions.

3. The method of claim 1, wherein the file server is a hybrid segment-oriented file server.

4. A WAN accelerator coupled between one or more LAN clients and one or more file servers separated from the one or more LAN clients by a WAN, comprising:

logic for receiving write requests from a client that are write requests for a file resident on a file server;

logic for converting the write request data into changes to a data map and segment data referenced by the data map;

logic for conveying the changed data map to the file server and receiving an acknowledgement of the commit of the data map to the file server;

logic for conveying the changed segment data to the file server and receiving an acknowledgement of the commit of the changed segment data; and

logic for signaling to the client making the write requests or other LAN clients that the file has been committed to the file server, wherein the signaling can occur after the acknowledgement of the commit of the data map to the file server and before receiving an acknowledgement of the commit of the changed segment data.

5. The WAN accelerator of claim 4, further comprising logic for providing a second client with the file being written by providing the data map from the file server and the segment data from a local segment store.

6. In a network having segment-based WAN accelerators that transform data to be transported over the WAN into a data map comprising segment references and stores of segments, a method of processing files stored in a file store as data maps, comprising:

receiving file data at a storage gateway, wherein the file data is to be stored in a file store coupled to the storage gateway;

identifying a format of the file data;

if the format of the file data is identified as a format that is to be converted into a data map format, passing the file data from the storage gateway to a WAN accelerator in the network in a manner that would cause the WAN accelerator to generate the data map for the file data;

receiving the generated data map at the storage gateway; and

storing that generated data map on the file store.

7. The method of claim 6, wherein the storage gateway has itself a capability of generating data maps for received file data, the method further comprising:

determining whether the storage gateway has a current capacity to generate the data map itself;

if the determination is that the storage gateway has the current capacity, generating the data map from the file data at the storage gateway; and

if the determination is that the storage gateway does not have the current capacity, generating the data map from the file data at the WAN accelerator.

8. The segment-oriented file store of claim 7, wherein the WAN accelerators used by the storage gateway are WAN accelerators at a relative edge of a network.

9. A segment-oriented file store that interfaces to segment-oriented WAN accelerators, wherein segment-oriented devices are devices that handle data as segments of data that can be independent of file type and sequences of segment references referencing those segments of data, and wherein segments independent of data type are independent in that at least one segment reference is present in files of more than one file type, the segment-oriented file store comprising:

storage for data maps, wherein a data map represents a file on a file store as a sequence of segment references;

logic for receiving data maps from network devices including segment-oriented WAN accelerators;

logic for serving data maps to network devices including segment-oriented WAN accelerators; and

logic for obtaining, over the network, segment data corresponding to segment references in data maps stored in the storage for data maps.

10. The segment-oriented file store of claim 9, wherein the storage for data maps includes storage for unreferenced data of at least one file that comprises segment references and unreferenced data.

11. The segment-oriented file store of claim 9, wherein the logic for receiving and logic for serving are configured to use a protocol that is an extension of a non-segment aware protocol.

12. The segment-oriented file store of claim 9, wherein the logic for receiving and logic for serving are configured to use a protocol that includes protocol elements for handling segmented data and reconstructing references to data.

13. The segment-oriented file store of claim 12, wherein the protocol is a wrapper over a non-segment aware protocol.

14. The segment-oriented file store of claim 9, wherein the data maps include segment references that are globally unique over a domain, and wherein at least some of the segment references are represented in fewer bits that the segment data they represent.

15. The segment-oriented file store of claim 9, wherein the file store comprises a segment-aware storage gateway and a non-segment aware storage device.