Scalable distributed storage architecture

• US 9,811,531 B2
• Filed: 08/26/2013
• Issued: 11/07/2017
• Est. Priority Date: 08/26/2013
• Status: Active Grant

• Current Assignee: Vmware LLC (Broadcom, Inc.)
• Original Assignee: VMware, Inc. (Broadcom, Inc.)
• Inventors: Karamanolis, Christos, Vasani, Soam
• Primary Examiner(s): Hashemi, Sana Al
• Application Number:
  US14/010,293
• Publication Number:
  US 20150058384A1
• Time in Patent Office: 1,534 Days
• Field of Search: 707607, 707609, 707687, 707705, 707790, 707813, 707821
• CPC Class Codes: G06F 16/182 Distributed file systems
• CPC Class Codes: G06F 16/188 Virtual file systems
• CPC Class Codes: G06F 16/27 Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor

Abstract

Techniques are disclosed for providing a file system interface for an object store intended to support simultaneous access to objects stored in the object store by multiple clients. In accordance with one method, an abstraction of a root directory to a hierarchical namespace for the object store is exposed to clients. The object store is backed by a plurality of physical storage devices housed in or directly attached to the plurality of host computers and internally tracks its stored objects using a flat namespace that maps unique identifiers to the stored objects. The creation of top-level objects appearing as subdirectories of the root directory is enabled, wherein each top-level object represents a separate abstraction of a storage device having a separate namespace that can be organized in accordance with any designated file system.

Claims

1. A method for providing a file system interface for an object store intended to support simultaneous access to objects stored in the object store by multiple clients, the steps comprising:

exposing an abstraction of a root directory to a hierarchical namespace for the object store, wherein the object store (i) is backed by a plurality of physical storage devices housed in or directly attached to different host computers including a first physical storage device housed in or directly attached to a first host computer and a second physical storage device housed in or directly attached to a second host computer;

(ii) includes at least one stored object that is backed by both the first physical storage device and the second physical storage device; and

(iii) internally tracks its stored objects using a flat namespace that maps unique identifiers to the stored objects; and

enabling creation of top-level objects appearing as subdirectories of the root directory, wherein each top-level object represents a separate abstraction of a storage device having a separate namespace organized in accordance with a designated file system.

4. The method of claim 1, wherein at least one top-level object is a clustered file system designed to support simultaneous access by clients up to a maximum limit.

5. The method of claim 1, wherein the flat namespace is implemented as an in-memory database, a synchronized copy of which is stored in the memory of each of the host computers.

6. The method of claim 1, wherein an administrator specified name for an object is transformed into a unique identifier for the object when the flat namespace reports a conflict with an auto-generated unique identifier for the object.

7. The method of claim 1, wherein the stored object that is backed by both the first physical storage device and the second physical storage device represents a virtual disk file used by a virtual machine running on one of the host computers.

14. The method of claim 4, wherein the top-level object is auto-mounted as a subdirectory at a client'"'"'s local version of the abstraction of the root directory when the client attempts to access a file stored within the clustered file system.

15. The method of claim 4, wherein the top-level object is configured to store metadata for file objects representing files stored in the clustered file system, wherein the file objects are stored in the object store separate from the top-level object.

20. The method of claim 14, wherein the top-level objects is automatically unmounted after a period of time in which no files stored within the clustered file system are accessed by the client.

21. The method of claim 15, wherein a file object is configured to store unique identifiers usable by the flat namespace to find locations of corresponding component objects that represent separate portions of the files and are stored in the object store separate from the file object.

22. The method of claim 15, wherein the metadata stored in the top-level object for a file object includes at least a file descriptor that comprises a unique identifier usable by the flat namespace to find a location of the file object in the object store.

27. The method of claim 22, wherein the file represented by the file object is a virtual disk file used by a virtual machine running on one of the host computers.

30. The method of claim 27, wherein the top-level object is configured to store metadata for file objects that are only utilized by one of a plurality of virtual machines capable of being instantiated on any of the host computers.

2. A non-transitory computer readable storage medium storing instructions, which, when executed on a processor, performs an operation for providing a file system interface for an object store intended to support simultaneous access to objects stored in the object store by multiple clients, the operation comprising:

exposing an abstraction of a storage device, wherein the storage device is backed by an object store comprising, in aggregate, a plurality of physical storage devices housed in or attached to different host computers including a first physical storage device housed in or directly attached to a first host computer and a second physical storage device housed in or directly attached to a second host computer; and

enabling creation of multiple top-level file system objects in the object store that each represent separate file systems, wherein each top-level file system object is configured to (i) respond according to the semantics of its corresponding file system, and (ii) store metadata for file objects representing files stored in the corresponding file system, wherein the file objects are stored in the object store separate from their corresponding top-level file system objects and at least one file object is backed by both the first physical storage device and the second physical storage device.

8. The computer readable storage medium of claim 2, wherein at least one top-level object contains a clustered file system designed to support simultaneous access by clients up to a maximum limit.

9. The computer readable storage medium of claim 2, wherein the flat namespace is implemented as an in-memory database, a synchronized copy of which is stored in the memory of each of the host computers.

10. The method of claim 2, wherein the file object that is backed by both the first physical storage device and the second physical storage device represents a virtual disk file used by a virtual machine running on one of the host computers.

16. The computer readable storage medium of claim 8, wherein the top-level object is auto-mounted as a subdirectory at a client'"'"'s local version of the abstraction of the root directory when the client attempts to access a file stored within the clustered file system.

17. The computer readable storage medium of claim 8, wherein the top-level object is configured to store metadata for file objects representing files stored in the clustered file system, wherein the file objects are stored in the object store separate from the top-level object.

23. The computer readable storage medium of claim 17, wherein a file object is configured to store unique identifiers usable by the flat namespace to find locations of corresponding component objects that represent separate portions of the files and are stored in the object store separate from the file object.

24. The computer readable storage medium of claim 17, wherein the metadata stored in the top-level object for a file object includes at least a file descriptor that comprises a unique identifier usable by the flat namespace to find a location of the file object in the object store.

28. The computer readable storage medium of claim 24, wherein the file represented by the file object is a virtual disk file used by a virtual machine running on one of the host computers.

31. The computer readable storage medium of claim 28, wherein the top-level object is configured to store metadata for file objects that are only utilized by one of a plurality of virtual machines capable of being instantiated on any of the of host computers.

3. A computer system, comprising:

a processor and

a memory hosting a module, which, when executed on the processor, performs an operation for providing a file system interface for an object store intended to support simultaneous access to objects stored in the object store by multiple clients, the operation comprising;

exposing an abstraction of a storage device, wherein the storage device is backed by an object store comprising, in aggregate, a plurality of physical storage devices housed in or directly attached to different host computers including a first physical storage device housed in or directly attached to a first host computer and a second physical storage device housed in or directly attached to a second host computer; and

enabling creation of multiple top-level file system objects in the object store that each represent separate file systems, wherein each top-level file system object is configured to (i) respond according to the semantics of its corresponding file system, and (ii) store metadata for file objects representing files stored in the corresponding file system, wherein the file objects are stored in the object store separate from their corresponding top-level file system objects and at least one file object is backed by both the first physical storage device and the second physical storage device.

11. The computer system of claim 3, wherein at least one top-level object contains a clustered file system designed to support simultaneous access by clients up to a maximum limit.

12. The computer system of claim 3, wherein the flat namespace is implemented as an in-memory database, a synchronized copy of which is stored in the memory of each of the host computers.

13. The computer system of claim 3, wherein the file object that is backed by both the first physical storage device and the second physical storage device represents a virtual disk file used by a virtual machine running on one of the host computers.

18. The computer system of claim 11, wherein the top-level object is auto-mounted as a subdirectory at a client'"'"'s local version of the abstraction of the root directory when the client attempts to access a file stored within the clustered file system.

19. The computer system of claim 11, wherein the top-level object is configured to store metadata for file objects representing files stored in the clustered file system, wherein the file objects are stored in the object store separate from the top-level object.

25. The computer system of claim 19, wherein a file object is configured to store unique identifiers usable by the flat namespace to find locations of corresponding component objects that represent separate portions of the files and are stored in the object store separate from the file object.

26. The computer system of claim 19, wherein the metadata stored in the top-level object for a file object includes at least a file descriptor that comprises a unique identifier usable by the flat namespace to find a location of the file object in the object store.

29. The computer system of claim 26, wherein the file represented by the file object is a virtual disk file used by a virtual machine running on one of the host computers.

32. The computer system of claim 29, wherein the top-level object is configured to store metadata for file objects that are only utilized by one of a plurality of virtual machines capable of being instantiated on any of the host computers.