System and method of using in-memory replicated object to support file services wherein file server converts request to block I/O command of file handle, replicating said block I/O command across plural distributed storage module and performing said block I/O command by local storage module

• US 10,642,783 B2
• Filed: 01/12/2018
• Issued: 05/05/2020
• Est. Priority Date: 01/12/2018
• Status: Active Grant

• Current Assignee: Vmware LLC (Broadcom, Inc.)
• Original Assignee: VMware, Inc. (Broadcom, Inc.)
• Inventors: Wang, Wenguang, Knauft, Eric, Premachandran, Srinath, Guo, Zhaohui, Spillane, Richard P.
• Primary Examiner(s): Partridge, William B
• Assistant Examiner(s): Yew, Chie
• Application Number:
  US15/870,506
• Publication Number:
  US 20190220211A1
• Time in Patent Office: 844 Days
• Field of Search: None
• CPC Class Codes: G06F 11/08 Error detection or correction by redundancy in data representation, e.g. by using checking codes
• CPC Class Codes: G06F 16/10 File systems; File servers
• CPC Class Codes: G06F 2009/45579 I/O management, e.g. providing access to device drivers or storage
• CPC Class Codes: G06F 2009/45583 Memory management, e.g. access or allocation
• CPC Class Codes: G06F 3/0607 by facilitating the process of upgrading existing storage systems, e.g. for improving compatibility between host and storage device
• CPC Class Codes: G06F 3/0661 Format or protocol conversion arrangements
• CPC Class Codes: G06F 3/0665 at area level, e.g. provisioning of virtual or logical volumes
• CPC Class Codes: G06F 3/067 Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
• CPC Class Codes: G06F 9/45558 Hypervisor-specific management and integration aspects

Abstract

Techniques are disclosed for using in-memory replicated object to support file services. Certain embodiments provide a method of storing persistent file handles in a storage system comprising a plurality of computing devices. The method may include requesting to write a persistent file handle corresponding to a file to a file system stored on the plurality of computing devices. The request may be translated to a block input/output (I/O) command to an in-memory object, the in-memory object representing at least a portion of the file system, a copy of the in-memory object being stored at each of the plurality of computing devices in volatile memory. The persistent file handle may then be written to the copy of the in-memory object stored in the volatile memory of each of the plurality of computing devices.

Claims

1. A method of storing persistent file handles in a storage system comprising a plurality of computing devices, comprising:

receiving, at a server, a request to open a file, a copy of the file being stored on each of a plurality of storage devices each connected to or housed in at least one of the plurality of computing devices, wherein the server is running in a virtual computing instance that is running on a first computing device of the plurality of computing devices;

generating, by the server, a request to write a persistent file handle corresponding to the file to a file system stored on the plurality of computing devices;

sending the request from the server to a file server running on the first computing device;

translating, by the file server, the request to a block input/output (I/O) command directed to an in-memory object, the in-memory object representing at least a portion of the file system, a copy of the in-memory object being stored at each of the plurality of computing devices in volatile memory;

sending, by the file server, the block I/O command to a distributed storage module executed by at least one processor of the first computing device;

replicating, by the distributed storage module, the block I/O command;

sending, by the distributed storage module, a copy of the block I/O command to a local storage module on each of the plurality of computing devices, each local storage module executed by at least one processor of the corresponding one of the plurality of computing devices; and

performing, by the local storage module on the first computing device, the block I/O command to write the persistent file handle to a first copy of the in-memory object stored in the volatile memory of the first computing device.

2. The method of claim 1, wherein each copy of the in-memory object represents a first volume of the file system, wherein a copy of a storage object is stored at each of the plurality of storage devices, wherein each copy of the storage object represents a second volume of the file system, and further comprising:

receiving, by the local storage module on the first computing device, a second block I/O command to write data to a first copy of the storage object stored in one of the plurality of storage devices connected to or housed in the first computing device, the one of the plurality of storage devices comprising a non-volatile storage; and

performing, by the local storage module on the first computing device, the second block I/O command to write the data to the first copy of the storage object.

3. The method of claim 2, further comprising:

maintaining, by the local storage module on the first computing device, a first queue for block I/O commands to the first volume and a second queue for block I/O commands to the second volume; and

performing block I/O commands in the first queue at a higher priority than block I/O commands in the second queue.

4. The method of claim 1, wherein the persistent file handle is never written to a non-volatile storage of the first computing device.

5. The method of claim 1, wherein the first computing device becomes unavailable for a period of time and again becomes available after the period of time, and further comprising:

performing, by the distributed storage module, a full resync of the first copy of the in-memory object stored in the volatile memory of the first computing device after the period of time by copying all data corresponding to at least one copy of the in-memory object from at least a second computing device of the plurality of computing devices.

6. The method of claim 5, wherein each copy of the in-memory object represents a first volume of the file system, wherein a copy of a storage object is stored at each of the plurality of storage devices, wherein each copy of the storage object represents a second volume of the file system, and further comprising:

performing, by the distributed storage module, a partial resync of a first copy of the storage object stored in one of the plurality of storage devices connected to or housed in the first computing device after the period of time by copying any data updated during the period of time corresponding to at least one copy of the storage object from at least a second one of the plurality of storage devices, the one of the plurality of storage devices comprising a non-volatile storage.

7. The method of claim 1, wherein the in-memory object is a virtual storage network object of a virtual storage network.

8. A non-transitory computer readable medium comprising instructions to be executed in a computer system, wherein the instructions when executed in the computer system perform a method of storing persistent file handles in a storage system comprising a plurality of computing devices, the method comprising:

receiving, at a server, a request to open a file, a copy of the file being stored on each of a plurality of storage devices each connected to or housed in at least one of the plurality of computing devices, wherein the server is running in a virtual computing instance that is running on a first computing device of the plurality of computing devices;

generating, by the server, a request to write a persistent file handle corresponding to the file to a file system stored on the plurality of computing devices;

sending the request from the server to a file server running on the first computing device;

translating, by the file server, the request to a block input/output (I/O) command directed to an in-memory object, the in-memory object representing at least a portion of the file system, a copy of the in-memory object being stored at each of the plurality of computing devices in volatile memory;

sending, by the file server, the block I/O command to a distributed storage module executed by at least one processor of the first computing device;

replicating, by the distributed storage module, the block I/O command;

sending, by the distributed storage module, a copy of the block I/O command to a local storage module on each of the plurality of computing devices, each local storage module executed by at least one processor of the corresponding one of the plurality of computing devices; and

performing, by the local storage module on the first computing device, the block I/O command to write the persistent file handle to a first copy of the in-memory object stored in the volatile memory of the first computing device.

9. The non-transitory computer readable medium of claim 8, wherein each copy of the in-memory object represents a first volume of the file system, wherein a copy of a storage object is stored at each of the plurality of storage devices, wherein each copy of the storage object represents a second volume of the file system, and wherein the method further comprises:

receiving, by the local storage module on the first computing device, a second block I/O command to write data to a first copy of the storage object stored in one of the plurality of storage devices connected to or housed in the first computing device, the one of the plurality of storage devices comprising a non-volatile storage; and

performing, by the local storage module on the first computing device, the second block I/O command to write the data to the first copy of the storage object.

10. The non-transitory computer readable medium of claim 9, wherein the method further comprises:

maintaining, by the local storage module on the first computing device, a first queue for block I/O commands to the first volume and a second queue for block I/O commands to the second volume; and

performing block I/O commands in the first queue at a higher priority than block I/O commands in the second queue.

11. The non-transitory computer readable medium of claim 8, wherein the persistent file handle is never written to a non-volatile storage of the first computing device.

12. The non-transitory computer readable medium of claim 8, wherein the first computing device becomes unavailable for a period of time and again becomes available after the period of time, and wherein the method further comprises:

performing, by the distributed storage module, a full resync of the first copy of the in-memory object stored in the volatile memory of the first computing device after the period of time by copying all data corresponding to at least one copy of the in-memory object from at least a second computing device of the plurality of computing devices.

13. The non-transitory computer readable medium of claim 12, wherein each copy of the in-memory object represents a first volume of the file system, wherein a copy of a storage object is stored at each of the plurality of storage devices, wherein each copy of the storage object represents a second volume of the file system, and wherein the method further comprises:

performing, by the distributed storage module, a partial resync of a first copy of the storage object stored in one of the plurality of storage devices connected to or housed in the first computing device after the period of time by copying any data updated during the period of time corresponding to at least one copy of the storage object from at least a second one of the plurality of storage devices, the one of the plurality of storage devices comprising a non-volatile storage.

14. The non-transitory computer readable medium of claim 8, wherein the in-memory object is a virtual storage network object of a virtual storage network.

15. A computer system, wherein system software for the computer system is programmed to execute a method of storing persistent file handles in a storage system comprising a plurality of computing devices, the method comprising:

receiving, at a server, a request to open a file, a copy of the file being stored on each of a plurality of storage devices each connected to or housed in at least one of the plurality of computing devices, wherein the server is running in a virtual computing instance that is running on a first computing device of the plurality of computing devices;

generating, by the server, a request to write a persistent file handle corresponding to the file to a file system stored on the plurality of computing devices;

sending the request from the server to a file server running on the first computing device;

translating, by the file server, the request to a block input/output (I/O) command directed to an in-memory object, the in-memory object representing at least a portion of the file system, a copy of the in-memory object being stored at each of the plurality of computing devices in volatile memory;

sending, by the file server, the block I/O command to a distributed storage module executed by at least one processor of the first computing device;

replicating, by the distributed storage module, the block I/O command;

sending, by the distributed storage module, a copy of the block I/O command to a local storage module on each of the plurality of computing devices, each local storage module executed by at least one processor of the corresponding one of the plurality of computing devices; and

performing, by the local storage module on the first computing device, the block I/O command to write the persistent file handle to a first copy of the in-memory object stored in the volatile memory of the first computing device.

16. The computer system of claim 15, wherein each copy of the in-memory object represents a first volume of the file system, wherein a copy of a storage object is stored at each of the plurality storage devices, wherein each copy of the storage object represents a second volume of the file system, and wherein the method further comprises:

receiving, by the local storage module on the first computing device, a second block I/O command to write data to a first copy of the storage object stored in one of the plurality of storage devices connected to or housed in the first computing device, the one of the plurality of storage devices comprising a non-volatile storage; and

performing, by the local storage module on the first computing device, the second block I/O command to write the data to the first copy of the storage object.

17. The computer system of claim 16, wherein the method further comprises:

maintaining, by the local storage module on the first computing device, a first queue for block I/O commands to the first volume and a second queue for block I/O commands to the second volume; and

performing block I/O commands in the first queue at a higher priority than block I/O commands in the second queue.

18. The computer system of claim 15, wherein the persistent file handle is never written to a non-volatile storage of the first computing device.

19. The computer system of claim 15, wherein the first computing device becomes unavailable for a period of time and again becomes available after the period of time, and wherein the method further comprises:

performing, by the distributed storage module, a full resync of the first copy of the in-memory object stored in the volatile memory of the first computing device after the period of time by copying all data corresponding to at least one copy of the in-memory object from at least a second computing device of the plurality of computing devices.

20. The computer system of claim 15, wherein the in-memory object is a virtual storage network object of a virtual storage network.