File service using a shared file access-rest interface

• US 10,536,523 B2
• Filed: 05/10/2015
• Issued: 01/14/2020
• Est. Priority Date: 05/11/2014
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: Calder, Bradley Gene, Ghanem, Jean, Edwards, Andrew James, Aroya, Ishai Ben, Goebel, David Allen, Wu, Jiesheng, Haridas, Jaidev, Ganguly, Shuvabrata, Hendel, Matthew Douglas, Demir, Ozan
• Primary Examiner(s): Vital, Pierre M
• Assistant Examiner(s): Lodhi, Andalib F
• Application Number:
  US14/708,286
• Publication Number:
  US 20150324386A1
• Time in Patent Office: 1,710 Days
• Field of Search: 707649
• CPC Class Codes: G06F 11/1464 for networked environments
• CPC Class Codes: G06F 16/176 Support for shared access to files; File sharing support
• CPC Class Codes: G06F 16/1774 Locking methods, e.g. locking methods for file systems allowing shared and concurrent access to files
• CPC Class Codes: G06F 16/182 Distributed file systems
• CPC Class Codes: G06F 16/955 using information identifiers, e.g. uniform resource locators [URL]
• CPC Class Codes: G06F 2201/84 Using snapshots, i.e. a logical point-in-time copy of the data
• CPC Class Codes: H04L 67/1097 for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]
• CPC Class Codes: H04L 69/18 Multiprotocol handlers, e.g. single devices capable of handling multiple protocols

Abstract

In various embodiments, methods and systems for providing access to file systems are provided. An SFA-based operation having an operation modifier for a file in a file system is received. The SFA-based operation is defined based on a file-representational state transfer (REST) interface. The SFA-REST interface comprises integrated functionality of an SFA protocol and REST protocol. The SFA-based operation is executed using the operation modifier. A REST-based operation for the file in the distributed file storage is received. The REST-based operation is defined based on the SFA-REST interface. The operation modifier is referenced in order to execute the REST-based operation. The REST-based operation based on the SFA-REST interface. A schema can be implemented for accessing the file system. The schema includes tables to store files, the table comprises fields corresponding to elements of a SFA-REST interface. A snapshot component can be implemented to backup file shares in the file system.

Claims

1. A system for executing REST-based operations and SFA-based operations in distributed file systems, the system comprising:

one or more hardware processors and computer storage media storing computer-executable instructions and components that, when executed, by the one or more hardware processors, cause the one or more hardware processors to execute;

a Shared File Access-Representational State Transfer (SFA-REST) interface component configured for;

receiving one or more SFA-based operations for files in a distributed file system;

processing the one or more SFA-based operations based on an SFA-REST interface, wherein the SFA-REST interface comprises integrated functionality of an SFA protocol and REST protocol;

receiving one or more REST-based operations for files in the distributed file system;

processing the one or more REST-based operations based on the SFA-REST interface;

executing an SFA-based operation on a first file;

executing an SFA-based operation on a second file; and

executing a REST-based operation on the second file based in part on referencing the currently executing SFA-based operation accessing the second file.

2. The system of claim 1, wherein the SFA interface component further comprises a data access component configured for:

returning a handle corresponding to the first file for the SFA-based operation; and

not returning a handle corresponding to the second file for the REST-based operation.

3. The system of claim 1, wherein the SFA interface component further comprises a lease access component configured for:

granting a REST lease for the second file, wherein the REST lease supports exclusive write shared read access to the second file; and

granting shared access to files based on REST leases or handles of operation modifiers associated with the files, wherein the one or more SFA-based operations include operation modifiers that define a constraint that modifies the one or more SFA-based operations.

4. The system of claim 1, wherein the SFA interface component further comprises a cache component configured for:

identifying an opportunistic lock associated with operation modifiers of the one or more SFA-based operations, wherein the opportunistic lock indicates that a latest state of files is cached on an SFA client associated with the SFA-based operation;

determining to break the opportunistic lock of the SFA-based operation based on referencing the operation modifiers; and

performing an opportunistic lock break operation such that the one or more REST-based operations are executed.

5. The system of claim 1, wherein the SFA interface component further comprises a time stamp component configured for:

generating a plurality of timestamps for files in the distributed file system for both the one or more SFA-based operations and the one or more REST-based operations, wherein the plurality of time stamps include at least an entity tag (ETAG) timestamp and one or more of;

a creation time, a last modify time, a last write time, and a last access time, wherein the ETAG timestamp is updated for both the one or more REST-based operations and the one or more SFA-based operations to support optimistic concurrency.

6. The system of claim 1, wherein the SPA interface component further, comprises:

a metadata component configured for providing a restricted name set for metadata corresponding to the one or more SFA-based operations and the one or more REST-based operations; and

a name and referencing component configured for implementing case insensitive and case preserving directory names and file names system.

7. The system of claim 1, wherein the SFA interface component further comprises a byte lock component configured for:

rejecting read operations or write operations through the one or more REST-based operations when a conflict exists with a logic of the one or more SFA-based operations byte range lock, wherein an SFA-based byte range lock defines read and write access to regions of tiles.

8. The system of claim 1, wherein the SFA interface component further comprises a locking component configured for executing locking operation modifiers associated with the one or more SFA-based operations, wherein access to files is processed based on locking operation modifiers of the one or more SFA-based operations.

9. The system of claim 1, further comprising a file service schema comprising a plurality of tables, wherein the plurality of tables store states of files in the distributed file system, and wherein the plurality tables comprise fields corresponding to elements of the SFA-REST interface.

10. The system of claim 1, further comprising a snapshot component configured for:

snapshotting a file share to back-up the contents of the file share, wherein the file share comprises changes to the file share written to the file share using a copy-on-write type pattern such that a previous version at each snapshot can be accessed independently, and wherein snapshotting the file share is based one of;

an atomic snapshot operation comprising a two-pass-notification and write instruction operation;

a non-atomic snapshot operation comprising a single-pass write instruction operation;

or

a cross-share atomic snapshot comprising snapshotting multiple file shares.

11. The system of claim 1, further comprising a snapshot component comprising a share table having a schema that represents the files and snapshots of the file share, wherein the share table comprises an entry for the file share and one or more entries for snapshots of the file share.

12. A system for executing REST-based operations and SFA-based operations in distributed file systems, the system comprising:

one or more hardware processors and computer storage media storing computer-executable instructions and components that, when executed, by the one or more hardware processors, cause the one or more hardware processors to execute;

a file service schema comprising a plurality of tables, wherein the plurality of tables store states of files in the distributed file system, and wherein the plurality of tables comprise fields corresponding to elements of an a Shared File Access-Representational State Transfer (SFA-REST) interface,

wherein the SFA-REST interface comprises integrated functionality of an SFA protocol and REST protocol for referencing between one or more REST-based operations and one or more SFA-based operations concurrently executing to access files in the file system.

13. The system of claim 12, wherein the file service schema further comprises hierarchical tables for storing files and directories based on corresponding names of the files and directories, wherein files and directories are retrieved using lookups to a namespace table storing a plurality of file identifiers.

14. The system of claim 12, further comprising a namespace table, wherein the name space table is a primary table of the file service schema.

15. The system of claim 12, further comprising a file information table, wherein the file information table stores information about the content of files, wherein the file information table comprises a plurality of timestamps fields for files.

16. The system of claim 12, further comprising a file page range table, wherein the file page range table comprises data associated with files, wherein a key for the table comprises a file identifier and an offset size of a page range stored for a file.

17. The system of claim 12, further comprising a handle table, wherein the handle table comprises one or more open handles for the one or more SFA-REST interface operations.

18. The system of claim 12, further comprising a lease table, wherein the lease table stores leases that are associated with files.

19. The system of claim 12, further comprising a change notify table and a change notify events table, the change notify table is updated upon a registration to detect changes within a directory and the change notify events table is updated upon the occurring of an event that is stored.

20. The system of claim 12, further comprising a byte range lock table, wherein the byte range lock stores locks for files using at least a lock identifier and a handle identifier.

21. A computer-implemented method for executing REST-based operations and SFA-based operations in a distributed file system, the method comprising:

receiving a Shared File Access (SFA)-based operation for a file in a distributed file system from an SFA client, the SFA-based operation having an operation modifier that indicates an opportunistic lock of the SFA-based operation, wherein the SFA-based operation is defined based on a Shared File Access-Representational State Transfer (SFA-REST) interface;

processing the SFA-based operation on the file based on referencing the operation modifier of the SFA-based operation;

receiving a REST-based operation for the file in the distributed file system from a REST client, wherein the REST-based operation is defined based on the SFA-REST interface, wherein the SFA-REST interface comprises integrated functionality of an SFA protocol and REST protocol;

processing the REST-based operation based on the SFA-REST interface;

executing the SFA-based operation on the file; and

executing the REST-based operation on the file based on referencing the operation modifier of the currently executing SFA-based operation.

22. The method of claim 21, wherein executing the REST-based operation on the file based on referencing the operation modifier further comprises:

identifying the opportunistic lock associated with the operation modifier of the SFA-based operation, wherein the opportunistic lock indicates that a latest state of the file is cached on the SFA client associated with the SFA-based operation;

determining to break the opportunistic lock of the SFA-based operation based on referencing the operation modifier; and

performing an opportunistic lock break operation such that the REST-based operation is executed.

23. The method of claim 21, wherein the SFA client acquires the opportunistic lock comprising one or more of:

a read(R), wherein the SFA client reads from a local cache;

a write(W), wherein the client writes locally;

a handle(H), wherein the SFA client does not immediately send a notification when a handle is closed; and

wherein the read(R) opportunistic lock is broken when the REST-based operation is a write operation;

wherein the write(W) opportunistic lock is broken when the REST-based operation is a read operation; and

wherein the handle(H) opportunistic lock is broken when the REST-based operation is a delete operation.

24. The method of claim 23, wherein the handle(H) opportunistic lock is broken when the REST-based operation initiates a sharing violation with the handle(H) of the SFA-based operation, wherein the access mode and the sharing mode of the SFA-based operation are different from the opportunistic lock.

25. The method of claim 23, wherein the opportunistic lock break operation comprises:

flushing one or more cached SMB client changes; and

communicating an operation failed notification for a client cache flush delayed over a specified request timeout.

26. A computer-implemented method for executing REST-based operations and SFA-based operations in a distributed file system, the method comprising:

receiving a Shared File Access (SFA)-based operation for a file in a distributed file system from an SFA client, the SFA-based operation having an operation modifier that indicates an access-shared mode of the SFA-based operation, wherein the SFA-based operation is defined based on a Shared File Access-Representational State Transfer (SFA-REST) interface;

processing the SFA-based operation based on the SFA-REST interface, wherein the SFA-REST interface comprises integrated functionality of an SFA protocol and REST protocol;

executing the SFA-based operation based on the operation modifier;

receiving a REST-based operation for the file in the distributed file system from a REST client;

processing the REST-based operation based on the SFA-REST interface; and

executing the REST-based operation on the file based on referencing the operation modifier of the currently executing SFA-based operation.

27. The method of claim 26, wherein the REST-based operation is a Create File operation, the Create File operation executed only when the access-share mode modifier comprises one of:

a write sharing mode, or a write/delete sharing mode.

28. The method of claim 26, wherein the REST-based operation is a Get File operation or List Ranges operation, the Get File operation or List Ranges operation executed only when the share mode modifier comprises a read sharing mode.

29. The method of claim 26, wherein the REST-based operation is a Put Range operation, Set Properties operation or a Set File Metadata operation, the Put Range operation, Set Properties operation or he Set File Metadata operation executed only when the share mode modifier comprises a write sharing mode.

30. The method of claim 26, wherein the REST-based operation is a Delete operation, the Delete operation executed only when the share mode modifier comprises a Delete sharing mode.

31. The method of claim 26, wherein the REST-based operation is a List Files operation, a Get File Properties operation, or a Get File Metadata operation, where the List Files operation, the Get File Properties operation, or the Get File Metadata operation are executed while the file is open with an exclusive read access based on the SFA-based operation having share mode modifier.

32. The method of claim 26, further comprising communicating a sharing violation when the share mode modifier precludes access to the file based on the REST-based operation received.

33. A computer-implemented method for executing REST-based operations and SFA-based operations in a distributed file system, the method comprising:

receiving an operation for a file at a Shared File Access-Representational State Transfer (SFA-REST) interface;

accessing a file service to perform the operation corresponding to the file, the file service comprising a plurality of tables, wherein the plurality of tables store states of files in the distributed file system, and wherein the SFA-REST interface comprises integrated functionality of an SFA protocol and REST protocol for accessing files in the file system;

processing the operation based on SFA-REST interface; and

executing the operation based at least in part on identifying a file state for the file that is stored based on a file service schema of the file service.

34. The method of claim 33, wherein the plurality of tables comprise fields defined in the file service schema corresponding to the SFA-REST interface, the plurality of tables related based on operations defined in the SFA-REST interface for coordinating operation interactions between SFA clients and REST clients.

35. The method of claim 33, wherein the file service schema supports bypassing storing files state in-memory states to storing file states directly on a cloud computing platform store such that a plurality SFA clients and REST clients simultaneously access the file states using the file service schema.

36. The method of claim 33, wherein the file service schema includes a hierarchical table for storing files and directories based on corresponding names of the files and directories, wherein files and directories are retrieved using lookups to a namespace table storing a plurality of file identifiers.

37. A computer-implemented method for executing REST-based operations and SFA-based operations in a distributed file system, the method comprising:

receiving an operation for a file share at a Shared File Access Representational State Transfer (SFA-REST) interface, wherein the SFA-REST interface comprises integrated functionality of an SFA protocol and REST protocol for accessing files in the file system;

accessing a share table to perform the operation corresponding the file share, the share table represents files shares and snapshots of the file shares using semantics of the SFA-REST interface, wherein the share table comprises an entry for a file share and an entry for a snapshot;

processing the operation based on the SFA-REST interface; and

executing the operation based at least in part on identifying a file share state or a snapshot state for the file share stored based on the share table.

38. The method of claim 37, further comprising snapshotting the file share to back-up the contents of the file share, wherein the file share comprises changes to the file share written to the file share using a copy-on-write type pattern such that a previous version at each snapshot can be accessed independently, and wherein snapshotting the file share is based on one of:

an atomic snapshot operation comprising a two-pass-notification and write instruction operation;

a non-atomic snapshot operation comprising a single-pass write instruction operation;

or

a cross-share atomic snapshot comprising snapshotting multiple file shares.

39. The method of claim 37, wherein snapshotting the file share is based on one of:

an atomic snapshot operation comprising a two-pass notification and write instruction operation;

a non-atomic snapshot operation comprising a single-pass write instruction operation;

or

a cross-share atomic snapshot comprising snapshotting multiple file shares.

40. The method of claim 37, further comprising:

updating the share table comprising the file share, wherein updating the share table comprises adding a row to the share table, the row associated with a new version of the file share;

communicating to a node associated with the file share a notification directing the node to start writing to the new version of the file share;

determining that the node is writing to the new version of the file share; and

updating the share table to indicate that the snapshot completed successfully.