Pluggable storage system for distributed file systems

• US 10,459,917 B2
• Filed: 05/09/2016
• Issued: 10/29/2019
• Est. Priority Date: 02/25/2013
• Status: Active Grant

• Current Assignee: Emc IP Holding Company LLC (Dell Technologies Inc.)
• Original Assignee: Emc IP Holding Company LLC (Dell Technologies Inc.)
• Inventors: Chang, Lei, Ma, Tao, Wang, Zhanwei, Jian, Lirong, Ma, Lili, Sherry, Gavin
• Primary Examiner(s): Beausoliel, Jr., Robert W
• Assistant Examiner(s): Rayyan, Susan F
• Application Number:
  US15/150,263
• Publication Number:
  US 20160292181A1
• Time in Patent Office: 1,268 Days
• CPC Class Codes: G06F 16/10 File systems; File servers
• CPC Class Codes: G06F 16/11 File system administration, e.g. details of archiving or snapshots file system backup G06F11/14
• CPC Class Codes: G06F 16/113 Details of archiving lifecycle management in storage systems G06F3/0649; backup systems G06F11/1446
• CPC Class Codes: G06F 16/148 File search processing
• CPC Class Codes: G06F 16/182 Distributed file systems
• CPC Class Codes: G06F 16/1858 Parallel file systems, i.e. file systems supporting multiple processors
• CPC Class Codes: G06F 16/217 Database tuning G06F16/2282 takes precedence; database performance monitoring G06F11/3409
• CPC Class Codes: G06F 16/245 Query processing
• CPC Class Codes: G06F 16/24524 Access plan code generation and invalidation; Reuse of access plans
• CPC Class Codes: G06F 16/2453 Query optimisation
• CPC Class Codes: G06F 16/24532 of parallel queries
• CPC Class Codes: G06F 16/24542 Plan optimisation
• CPC Class Codes: G06F 16/2455 Query execution
• CPC Class Codes: G06F 16/2471 Distributed queries
• CPC Class Codes: G06F 16/27 Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
• CPC Class Codes: G06F 16/43 Querying
• CPC Class Codes: G06F 16/907 Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
• CPC Class Codes: H04L 65/60 Network streaming of media packets
• CPC Class Codes: H04L 67/1097 for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]

Abstract

A method, article of manufacture, and apparatus for managing data. In some embodiments, this includes an initial instruction for a file stored in a first storage system, determining that the initial instruction is not supported by the first storage system, identifying a combination of instructions to the first storage system after determining that the initial instruction is not supported by the first storage system, wherein the combination of instructions is based on the initial instruction, performing the identified combination of instructions on the file stored in the first storage system, and storing results of the performed identified combination of instructions.

Claims

1. A method for managing data, comprising:

receiving, by an interfacing module between a segment execution engine and a first storage system, an initial instruction for a file stored in the first storage system;

determining, by the interfacing module, that the initial instruction is not supported by the first storage system;

identifying, by the interfacing module, a combination of instructions to the first storage system after determining that the initial instruction is not supported by the first storage system, wherein the combination of instructions is based on the initial instruction;

performing the identified combination of instructions on the file stored in the first storage system; and

storing results of the performed identified combination of instructions.

2. The method of claim 1, wherein the combination of instructions is determined based at least in part on metadata associated with the first storage system, and the metadata associated with the first storage system is stored in a meta store that includes information respectively associated with one or more storage systems.

3. The method of claim 1, wherein the combination of instructions is determined based at least in part on metadata associated with the first storage system, and the metadata associated with the first storage system indicates a mechanism for accessing a file system corresponding to the first storage system.

4. The method of claim 1, wherein the combination of instructions comprise one or more instructions that are determined based at least in part on metadata associated with the first storage system.

5. The method of claim 1, wherein the combination of instructions comprise one or more instructions that are determined based at least in part on the initial instruction.

6. The method of claim 1, further comprising:

selecting a second storage system to which the file is to be moved;

storing the file on the second storage system; and

performing a series of actions in connection with carrying out the collective result.

7. The method of claim 6, wherein the combination of instructions comprise one or more instructions, the one or more instructions include one or more of the selecting of the second storage system, the storing of the file on the second storage system, and the performing of the series of actions in connection with carrying out the collective result.

8. The method of claim 6, wherein the second storage system is selected based at least in part on the second storage system having a second file system.

9. The method of claim 6, further comprising:

in response to performing the series of actions, moving the file to the first storage system.

10. The method of claim 1, wherein the initial instruction includes a truncate instruction.

11. The method of claim 10, wherein the combination of instructions comprise one or more instructions, the one or more instructions include copy a first portion of the file, delete the file, and rename the first portion of the file.

12. The method of claim 1, wherein the initial instruction includes an update instruction.

13. A system for managing data, comprising a storage device and at least one processor configured to:

receive, by an interfacing module between a segment execution engine and a first storage system, an initial instruction associated with performing a first action in relation to a file stored in the first storage system, wherein the file is stored in the first storage system based at least in part on file system functionality associated with a first file system associated with the first storage system and one or more usage statistics associated with the file;

determine, by the interfacing module, that the initial instruction associated with performing the first action is not supported by the first storage system based at least in part on metadata associated with the first storage system;

in response to determining that the initial instruction associated with performing the first action is not supported by the first storage system, identify, by the interfacing module, one or more instructions that have a collective result that is equivalent to a result of the first action, wherein to identify the combination of instructions includes to store an update in a second file and merging the second file with the file after the update is complete;

perform the identified one or more instructions in connection with the file stored in the first storage system; and

store results of the performed identified one or more instructions.

14. The system of claim 13, wherein the combination of instructions is determined based at least in part on metadata associated with the first storage system, and the metadata associated with the first storage system is stored in a meta store that includes information respectively associated with one or more storage systems.

15. The system of claim 13, wherein the combination of instructions is determined based at least in part on metadata associated with the first storage system, and the metadata associated with the first storage system indicates a mechanism for accessing a file system corresponding to the first storage system.

16. The system of claim 13, wherein the combination of instructions comprise one or more instructions that are determined based at least in part on metadata associated with the first storage system.

17. The system of claim 13, wherein the combination of instructions comprise one or more instructions that are determined based at least in part on the initial instruction.

18. The system of claim 13, wherein the at least one processor is further configured to:

select a second storage system to which the file is to be moved;

store the file on the second storage system; and

perform a series of actions in connection with carrying out the collective result.

19. The system of claim 18, wherein the combination of instructions comprise one or more instructions that cause the processor to perform one or more of the selecting of the second storage system, the storing of the file on the second storage system, and the performing of the series of actions in connection with carrying out the collective result.

20. A computer program product for processing data, comprising a non-transitory computer readable medium having program instructions embodied therein for:

receiving, by an interfacing module between a segment execution engine and a first storage system, an initial instruction associated with performing a first action in relation to a file stored in a first storage system, wherein the file is stored in the first storage system based at least in part on file system functionality associated with a first file system associated with the first storage system and one or more usage statistics associated with the file;

determining, by the interfacing module, that the initial instruction associated with performing the first action is not supported by the first storage system based at least in part on metadata associated with the first storage system;

in response to determining that the initial instruction associated with performing the first action is not supported by the first storage system, identifying, by the interfacing module, one or more instructions that have a collective result that is equivalent to a result of the first action, wherein the identifying of the combination of instructions includes storing an update in a second file and merging the second file with the file after the update is complete;

performing the identified one or more instructions in connection with the file stored in the first storage system; and

storing results of the performed identified one or more instructions.