Burst buffer appliance with storage tiering control functionality based on user specification

• US 9,659,019 B1
• Filed: 06/27/2013
• Issued: 05/23/2017
• Est. Priority Date: 06/27/2013
• Status: Active Grant

• Current Assignee: Emc IP Holding Company LLC (Dell Technologies Inc.)
• Original Assignee: Emc IP Holding Company LLC (Dell Technologies Inc.)
• Inventors: Faibish, Sorin, Gupta, Uday, Peng, Tao, Bent, John M.
• Primary Examiner(s): Beausoliel, Jr., Robert
• Assistant Examiner(s): Ganger, Arthur
• Application Number:
  US13/929,025
• Time in Patent Office: 1,426 Days
• CPC Class Codes: G06F 16/13 File access structures, e.g. distributed indices arrangements of input from, or output to, record carriers G06F3/06
• CPC Class Codes: G06F 16/182 Distributed file systems

Abstract

A cluster file system comprises a burst buffer appliance coupled to a plurality of object storage servers via a network. The burst buffer appliance is configured to implement storage tiering control functionality based on a user specification for at least first and second storage tiers comprising respective disjoint subsets of the plurality of object storage servers. The user specification comprises, for example, a user request to move data from a first storage tier to a second storage tier; for an indication of where a given file is stored; for an indication of a given storage tier where a given file is stored and/or for an indication of an object storage server storing one or more given file portions.

Claims

1. An object storage device of a cluster file system comprising:

a burst buffer appliance configured to communicate with a plurality of object storage servers of the cluster file system over a network; and

a daemon for notifying, in response to a request for at least a portion of a logical file that is striped across said plurality of object storage servers, at least one external object storage server that stores a different portion of said requested logical file to pre-fetch the different portion, wherein said request and said notifying occur substantially simultaneously;

the burst buffer appliance further configured to implement storage tiering control functionality for at least first and second storage tiers comprising respective disjoint subsets of the plurality of object storage servers to control movement of data between the first and second storage tiers, wherein said burst buffer appliance implements said movement of data between the first and second storage tiers based on a user specification, wherein said burst buffer appliance comprises a high-speed memory that provides a buffer between a plurality of clients on one or more compute nodes and said cluster file system for storing data associated with input-output (IO) operations;

the object storage device being implemented utilizing at least one processing device coupled to a memory.

2. The object storage device of claim 1, further comprising a parallel log structured file system daemon for processing said user specification to implement said requested movement of data.

3. The object storage device of claim 1, wherein said user specification comprises a user request to move data from a first storage tier to a second storage tier.

4. The object storage device of claim 1, wherein said user specification comprises a user request for an indication of where a given file is stored.

5. The object storage device of claim 1, wherein said user specification comprises a user request for an indication of a given storage tier where a given file is stored.

6. The object storage device of claim 1, wherein said user specification comprises a user request for an indication of an object storage server storing one or more given file portions.

7. The object storage device of claim 6, wherein the object storage targets of the first type comprise respective non-volatile electronic storage devices including one or more of flash, MRAM and PC-RAM.

8. The object storage device of claim 6, wherein the object storage targets of the second type comprise respective disk storage devices.

9. The object storage device of claim 1, wherein each of the object storage servers is configured to interface with at least one object storage target, and further wherein the object storage servers in the first storage tier are configured to interface with object storage targets of a first type and the object storage servers in the second storage tier are configured to interface with object storage targets of a second type different than the first type.

10. The object storage device of claim 1, wherein the cluster file system comprises a Lustre file system.

11. The object storage device of claim 1, wherein the burst buffer appliance is further configured to communicate with a plurality of clients over the network and to process requests from said clients.

12. The object storage device of claim 1, wherein the burst buffer appliance further comprises a controller to implement said storage tiering control functionality.

13. The object storage device of claim 12, wherein the controller comprises a data placement and migration controller.

14. The object storage device of claim 12, wherein at least a portion of the controller is implemented using data migration software at least a portion of which is stored in said memory and executed by said processor.

15. A processing platform comprising the object storage device of claim 1.

28. The object storage device of claim 1, wherein the daemon communicates with one or more external burst buffer-aware entities.

29. The object storage device of claim 1, wherein the daemon receives one or more commands from one or more of a user and a client by means of a user interface.

16. A method comprising:

configuring a burst buffer appliance of an object storage device of a cluster file system to communicate with a plurality of object storage servers of the cluster file system over a network;

configuring a daemon of said object storage device to notify, in response to a request for at least a portion of a logical file that is striped across said plurality of object storage servers, at least one external object storage server that stores a different portion of said requested logical file to pre-fetch the different portion, wherein said request and said notifying occur substantially simultaneously; and

implementing storage tiering control functionality in the burst buffer appliance for at least first and second storage tiers comprising respective disjoint subsets of the plurality of object storage servers to control movement of data between the first and second storage tiers, wherein said burst buffer appliance implements said movement of data between the first and second storage tiers based on a user specification, wherein said burst buffer appliance comprises a high-speed memory that provides a buffer between a plurality of clients on one or more compute nodes and said cluster file system for storing data associated with input-output (IO) operations.

17. The method of claim 16, further comprising the step of a parallel log structured file system daemon processing said user specification to implement said requested movement of data.

18. The method of claim 16, wherein said user specification comprises one or more of a user request to move data from a first storage tier to a second storage tier, a user request for an indication of where a given file is stored, a user request for an indication of a given storage tier where a given file is stored and a user request for an indication of an object storage server storing one or more given file portions.

19. The method of claim 16, further comprising moving data between the first and second storage tiers under control of the burst buffer appliance.

20. The method of claim 16, further comprising:

configuring the object storage servers in the first storage tier to interface with object storage targets of a first type; and

configuring the object storage servers in the second storage tier to interface with object storage targets of a second type different than the first type;

wherein the first and second different types of object storage targets provide respective different levels of performance.

21. The method of claim 16, wherein the daemon communicates with one or more external burst buffer-aware entities.

22. The method of claim 16, wherein the daemon receives one or more commands from one or more of a user and a client by means of a user interface.

23. A computer program product comprising a non-transitory machine-readable storage medium having encoded therein executable code of one or more software programs, wherein the one or more software programs when executed cause the burst buffer appliance to perform the steps of the method of claim 16.

24. A cluster file system comprising:

a burst buffer appliance; and

a plurality of object storage servers coupled to the burst buffer appliance via a network, each of said object storage servers comprising a daemon for notifying, in response to a request for at least a portion of a logical file that is striped across said plurality of object storage servers, at least one external object storage server that stores a different portion of said requested logical file to pre-fetch the different portion, wherein said request and said notifying occur substantially simultaneously;

the burst buffer appliance configured to implement storage tiering control functionality for at least first and second storage tiers comprising respective disjoint subsets of the plurality of object storage servers;

the burst buffer appliance thereby being configured to control movement of data between the first and second storage tiers, wherein said burst buffer appliance implements said movement of data between the first and second storage tiers based on a user specification, wherein said burst buffer appliance comprises a high-speed memory that provides a buffer between a plurality of clients on one or more compute nodes and said cluster file system for storing data associated with input-output (IO) operations.

25. The cluster file system of claim 24, further comprising a parallel log structured file system daemon for processing said user specification to implement said requested movement of data.

26. The cluster file system of claim 24, wherein said user specification comprises one or more of a user request to move data from a first storage tier to a second storage tier, a user request for an indication of where a given file is stored, a user request for an indication of a given storage tier where a given file is stored and a user request for an indication of an object storage server storing one or more given file portions.

27. The cluster file system of claim 24, wherein the cluster file system comprises a Lustre file system.