Unified file and object storage architecture for clustered file systems

• US 10,380,364 B2
• Filed: 03/30/2016
• Issued: 08/13/2019
• Est. Priority Date: 03/30/2016
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Eda, Sasikanth, Hildebrand, Dean, Mate, Ashutosh V., Mittal, Varun, Owen, William W., Patil, Sandeep R., Raut, Smita J.
• Primary Examiner(s): Beausoliel, Jr., Robert W
• Assistant Examiner(s): Ganger, Arthur
• Application Number:
  US15/085,367
• Publication Number:
  US 20170286707A1
• Time in Patent Office: 1,231 Days
• Field of Search: 707785
• CPC Class Codes: G06F 16/13 File access structures, e.g. distributed indices arrangements of input from, or output to, record carriers G06F3/06
• CPC Class Codes: G06F 16/182 Distributed file systems
• CPC Class Codes: G06F 16/1827 Management specifically adapted to NAS management of storage area networks [SAN] G06F3/067
• CPC Class Codes: G06F 21/6218 to a system of files or objects, e.g. local or distributed file system or database
• CPC Class Codes: H04L 63/101 Access control lists [ACL]
• CPC Class Codes: H04L 63/205 involving negotiation or determination of the one or more network security mechanisms to be used, e.g. by negotiation between the client and the server or between peers or by selection according to the capabilities of the entities involved negotiation of communication capabilities H04L69/24

Abstract

A processor may identify a first directory in the UFO storage system. The first directory may include one or more subdirectories in one or more levels under the first directory. The one or more subdirectories may include a second directory that has includes one or more objects. The first directory may be associated with a first inode, and the second directory may be associated with a second inode. The processor may perform a stat call on the second directory to determine metadata attributes for the one or more objects that are stored in the second directory. The metadata attributes for the one or more objects may be stored in the second inode. The processor may add the metadata attributes for the one or more objects to the first inode.

Claims

1. A computer implemented method comprising:

identifying, by a processor, a first directory in a unified file and object (UFO) storage system, the first directory having one or more subdirectories in one or more levels under the first directory, the one or more subdirectories including a second directory, the second directory having one or more objects, the first directory being associated with a first inode and the second directory being associated with a second inode;

performing a stat call on the second directory to determine metadata attributes for the one or more objects, the metadata attributes for the one or more objects being stored in the second inode; and

adding the metadata attributes for the one or more objects to the first inode.

2. The method of claim 1, the method further comprising:

determining metadata attributes for a first object that is in the first directory; and

populating the first inode with at least the metadata attributes for the first object.

3. The method of claim 2, the method further comprising:

identifying one or more metadata parameters for objects in the UFO storage system, the UFO storage system having a UFO namespace where data can be accessed as an object and as a file, the one or more metadata parameters being parameters that are generated for objects stored in the UFO storage system;

determining that a second object has been updated using a file storage interface;

generating, by the processor and in response to determining that the second object has been updated using the file storage interface, metadata for the second object, the generated metadata including values for the one or more metadata parameters; and

appending the metadata to the second object.

4. The method of claim 3, wherein the one or more metadata parameters are configurable by a user during a creation of the UFO namespace.

5. The method of claim 3, the method further comprising:

identifying a third directory, the third directory being a directory in the file storage system that contains the second object;

updating a third inode for the third directory, the third inode storing the metadata attributes for the second object, wherein the updating includes updating the metadata attributes of the second object;

determining that the third directory is not a top-level directory;

passing, in response to determining that the third directory is not the top-level directory, the metadata attributes for the second object to a fourth directory, the fourth directory being a parent directory of the third directory; and

updating a fourth inode using the metadata attributes for the second object, the fourth inode being associated with the fourth directory.

6. The method of claim 3, the method further comprising:

updating a first container database file, the first container database file storing a list of objects in a first container and associated object statistics for the objects in the first container, the objects in the first container including the second object; and

updating a first account database file, the first account database file storing a list of one or more containers and associated container statistics for the one or more containers, the one or more containers including the first container.

7. The method of claim 2, the method further comprising:

identifying, by an Information Lifecycle Management (ILM) policy engine, a first rule;

determining, by analyzing the first inode using the first rule, that a second object in the UFO storage system being an object that needs to be migrated to a new storage device, wherein the second object is not in the first directory; and

migrating the second object to the new storage device.

8. The method of claim 2, the method further comprising:

determining that a user has modified an access control list setting for a second object that is stored in a third directory;

comparing the modified access control list setting for the second object to a container access control list setting for a first container that includes the second object by analyzing an inode associated with the third directory; and

determining whether the modified access control list setting for the second object matches the container access control list setting.

9. The method of claim 8, the method further comprising:

notifying, in response to determining that the modified access control list setting for the second object does not match the container access control list setting, the user of the mismatch;

prompting the user to approve the modified access control list setting for the second object; and

updating, in response to the user approving the modified access control list setting, the inode associated with the third directory.

10. The method of claim 1, wherein the adding the metadata attributes for the one or more objects to the first inode comprises:

determining that the first inode includes one or more entries;

determining that the one or more entries do not include an entry for the one or more objects;

generating a new entry in the first inode for each of the one or more objects; and

populating each new entry with the metadata attributes for one of the one or more objects such that each of the one or more objects has an associated entry in the first inode.

11. The method of claim 1, wherein the metadata attributes include a creation time for each of the one or more objects, a size of each of the one or more objects, and owner and permission data for each of the one or more objects.

12. The method of claim 1, wherein the one or more objects includes a first object, the method further comprising;

determining that the first object has been modified;

determining updated metadata attributes for the first object;

updating the second inode with the updated metadata attributes for the first object;

passing the updated metadata attributes for the first object to the first directory; and

updating the metadata attributes for the first object in the first inode.

13. The method of claim 1, wherein the metadata attributes for the one or more objects include a set of metadata attributes for each of the one or more objects, the set of metadata attributes for a respective object including values associated with the respective object for a plurality of metadata parameters.

14. The method of claim 1, wherein adding the metadata attributes for the one or more objects to the first inode includes storing, in the first inode, a complete copy of the metadata for each of the one or more objects.

15. The method of claim 14, wherein:

the metadata attributes for the one or more objects are stored in the first inode using a table,

the table includes a plurality of rows and columns,

each row corresponds to a different object,

each column corresponds to a particular metadata parameter, and

each data element in a respective row for a respective object includes a value associated with a metadata parameter that corresponds to a column of the data element.

16. A computer implemented method comprising:

identifying a root directory in a unified file and object (UFO) storage system that includes one or more subdirectories organized in one or more levels, the root directory and each of the one or more subdirectories having an associated inode that stores metadata, the one or more subdirectories including a leaf directory;

retrieving, from the inode for the leaf directory, metadata attributes for the leaf directory; and

storing the metadata attributes for the leaf directory in the inodes for each directory in a path between the root directory and the leaf directory.

17. The method of claim 16, wherein storing the metadata attributes for the leaf directory in the inodes for each directory in the path between the root directory and the leaf directory comprises:

passing the metadata attributes for the leaf directory to a first parent directory, the first parent directory being the parent directory of the leaf director;

storing the metadata attributes for the leaf directory in the inode for the first parent directory;

determining that the first parent directory is not the root directory;

determining metadata attributes for the first parent directory, the metadata attributes for the first parent directory including the metadata attributes for the leaf directory and metadata attributes of one or more objects stored in the first parent directory;

passing the metadata attributes for the first parent directory to a second parent directory, the second parent directory being the parent directory of the first parent directory; and

accumulating the metadata attributes for the leaf directory and the first parent directory in the inode for the second parent directory.

18. A system comprising:

a memory;

a processor in communication with the memory, the processor being configured to perform a method comprising;

identifying a first directory in a unified file and object (UFO) storage system, the first directory having one or more subdirectories in one or more levels under the first directory, the one or more subdirectories including a second directory, the second directory having one or more objects, the first directory being associated with a first inode and the second directory being associated with a second inode,

wherein the second inode includes metadata attributes for each of the one or more objects;

determining the metadata attributes for each of the one or more objects using the second inode; and

storing the metadata attributes for each of the one or more objects in the first inode.

19. The system of claim 18, wherein the method performed by the processor further comprises:

identifying one or more metadata parameters for objects in the UFO storage system, the UFO storage system having a UFO namespace where data can be accessed as an object and as a file, the one or more metadata parameters being parameters that are generated for objects stored in the UFO storage system;

determining that a second object has been updated using a file storage interface;

generating, by the processor and in response to determining that the second object has been updated using the file storage interface, metadata for the second object, the generated metadata including values for the one or more metadata parameters; and

appending the metadata to the second object.

20. The system of claim 18, wherein the one or more objects includes a first object, the method performed by the processor further comprising;

determining that the first object has been modified;

determining updated metadata attributes for the first object;

updating the second inode with the updated metadata attributes for the first object;

passing the updated metadata attributes for the first object to the first directory; and

updating the metadata attributes for the first object in the first inode.

21. The system of claim 18, wherein the method performed by the processor further comprises:

determining that a user has modified an access control list setting for a second object that is stored in a third directory;

comparing the modified access control list setting for the second object to a container access control list setting for a first container by analyzing an inode associated with the third directory, the first container including the second object;

determining whether the modified access control list setting for the second object matches the container access control list setting;

notifying, in response to determining that the modified access control list setting for the second object does not match the container access control list setting, the user of the mismatch;

prompting the user to approve the modified access control list setting for the second object; and

updating, in response to the user approving the modified access control list setting, the inode associated with the third directory.

22. A computer program product comprising a computer readable storage medium having program instructions embodied therewith, the program instructions executable by a processor to cause the processor to perform a method comprising:

identifying, by the processor, a first directory in a unified file and object (UFO) storage system having one or more subdirectories in one or more levels under the first directory, the one or more subdirectories including a second directory having one or more objects, the first directory being associated with a first inode and the second directory being associated with a second inode,

wherein the second inode includes a set of metadata attributes for each of the one or more objects, the set of metadata attributes for a respective object including values associated with the respective object for a plurality of metadata parameters;

retrieving the set of metadata attributes for the one or more objects from the second inode; and

adding the set of metadata attributes for the one or more objects to the first inode.

23. The computer program product of claim 22, wherein the method performed by the processor further comprises:

identifying one or more metadata parameters for objects in the UFO storage system, the UFO storage system having a UFO namespace where data can be accessed as an object and as a file, the one or more metadata parameters being parameters that are generated for objects stored in the UFO storage system;

determining that a second object has been updated using a file storage interface;

generating, by the processor and in response to determining that the second object has been updated using the file storage interface, metadata for the second object, the generated metadata including values for the one or more metadata parameters; and

appending the metadata to the second object.

24. The computer program product of claim 22, wherein the one or more objects includes a first object, the method performed by the processor further comprising;

determining that the first object has been modified;

determining updated metadata attributes for the first object;

updating the second inode with the updated metadata attributes for the first object;

passing the updated metadata attributes for the first object to the first directory; and

updating the metadata attributes for the first object in the first inode.

25. The computer program product of claim 22, wherein the method performed by the processor further comprises:

determining that a user has modified an access control list setting for a second object that is stored in a third directory;

comparing the modified access control list setting for the second object to a container access control list setting for a first container by analyzing an inode associated with the third directory, the first container including the second object;

determining whether the modified access control list setting for the second object matches the container access control list setting;

notifying, in response to determining that the modified access control list setting for the second object does not match the container access control list setting, the user of the mismatch;

prompting the user to approve the modified access control list setting for the second object; and

updating, in response to the user approving the modified access control list setting, the inode associated with the third directory.