Recursive Multi-Threaded File System Scanner For Serializing File System Metadata Exoskeleton

• US 20160224581A1
• Filed: 10/26/2015
• Published: 08/04/2016
• Est. Priority Date: 02/02/2015
• Status: Active Grant

• Current Assignee: Quantum Corporation (Chi Ko Investment Co., Ltd.)
• Original Assignee: Quantum Corporation (Chi Ko Investment Co., Ltd.)
• Inventors: Barry, Tim, Lord, Steve
• Granted Patent: US 10,545,915 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 16/13 File access structures, e.g. distributed indices arrangements of input from, or output to, record carriers G06F3/06
• CPC Class Codes: G06F 2212/1041 Resource optimization

Abstract

Methods, apparatus, and other embodiments facilitate discovering and serializing metadata for a file system. After initializing or accessing a serialization data structure, recursive search threads that begin at a selected location in the file system are spawned. The threads analyzes the selected location in the file system, acquire metadata associated with the selected location in the file system, serialize the metadata associated with the selected location in the file system into the serialization data structure, and then selectively spawn additional recursive search threads. Additional recursive search threads may be spawned upon determining that the current recursive search thread has reached a branch point in a hierarchy of the file system. Different threads may perform different types of searches (e.g., depth-first breadth-first) and may operate in parallel. The serialization data structure may be persisted and surfaced in an object (e.g., JSON object) that can be queried or searched.

Claims

1. A computerized method for serializing metadata associated with a file system, comprising:

accessing the file system;

initializing a serialization data structure;

spawning a current recursive search thread that begins at a selected location in the file system, where the current recursive search thread;

analyzes the selected location in the file system,

acquires metadata associated with the selected location in the file system,

serializes the metadata associated with the selected location in the file system into the serialization data structure, and

selectively spawns an additional recursive search thread upon determining that the current recursive search thread has reached a branch point in a hierarchy of the file system, where the additional recursive search thread begins at a selected node in a branch path available at the branch point.

2. The method of claim 1, comprising persisting the serialization data structure in an object.

3. The method of claim 2, where the object is a Javascript object notation (JSON) object.

4. The method of claim 1, where the current recursive search thread is a depth-first search thread or a breadth-first search thread and where the additional recursive search thread is a depth-first search thread or a breadth-first search thread.

5. The method of claim 1, comprising selecting whether the additional recursive search thread is a depth-first search thread or a breadth-first search thread based, at least in part, ort metadata associated with the selected location in the file system.

6. The method of claim 1, comprising selecting whether the additional recursive search thread is a depth-first search thread or a breadth-first search thread based, at least in part, on metadata associated with the selected node in the branch path available at the branch point.

7. The method of claim 1, where the file system is a shared disk file system.

8. The method of claim 1, where the file system is a file locking storage area network file system.

9. The method of claim 1, where the file system is a shared disk file system that provides filing locking and a single names ace for the file system in a storage area network configuration.

10. The method of claim 1, where the metadata includes hierarchy information or file information.

11. The method of claim 2, comprising:

accessing a query concerning the file system, and

satisfying the query from information in the object without accessing the file system.

12. The method of claim 2, comprising:

accessing the object, and

rebuilding a portion of the file system from the object using a recursive, multi-threaded builder.

13. The method of claim 12, where the recursive multithreaded builder:

spawns a current recursive build thread that begins at a first location in the file system,

acquires metadata associated with the first location in the file system from the object,

populates the first location in the file system with the metadata associated with the first location in the file system, and

selectively spawns an additional recursive build thread upon determining that the current recursive build thread has reached a build branch point in the hierarchy of the file system, where the additional recursive build thread begins at a second node in a branch path associated with the build branch point.

14. An apparatus for serializing metadata associated with a file system, comprising:

a processor;

a memory that stores information concerning a multi-threaded traversal of the file system;

a set of logics; and

an interface to connect the processor, the memory, and the set of logics,

the set of logics comprising;

a first logic that spawns a recursive thread that is configured to;

acquire metadata for locations in the file system visited by the recursive thread, and

make a request to the first logic to spawn an additional recursive thread at a branch point in a hierarchy of the file system,

a second logic that serializes metadata acquired by recursive threads spawned by the first logic into an object, and

a third logic that persists and surfaces the object.

15. The apparatus of claim 14, comprising:

a fourth logic that builds a copy of the file system from the object using a recursive tree building thread.

16. The apparatus of claim 14, comprising a fifth logic that satisfies a query concerning the file system metadata associated with the file system from the object.

17. The apparatus of claim 14, where the recursive thread is a depth-first search thread and where the additional recursive thread is a depth-first search thread.

18. The apparatus of claim 14, where the recursive thread is a breadth-first search thread and where the additional recursive thread is a breadth-first search thread.

19. The apparatus of claim 14, where the recursive thread is a depth-first search thread or a breadth-first search thread and where the additional recursive thread is a depth-first search thread or a breadth-first search thread, and where the first logic chooses whether to spawn a depth-first search thread or a breadth-first search thread based, at least in part, on metadata associated with the branch point in the hierarchy of the file system.

20. A non-transitory computer-readable storage medium storing computer executable instructions that when executed by a computer control the computer to perform a method for serializing file system metadata into a persistent object, the method comprising:

establishing a start point for a recursive, multi-threaded, multi-modal search thread;

launching the recursive, multi-threaded, multi-modal search thread at the start point; and

serializing data produced by the recursive, multi-threaded, multi-modal search thread and one or more progeny threads launched by the recursive, multi-threaded, multi-modal search thread into an object.