Cluster file system with a burst buffer appliance for controlling movement of data among storage tiers

• US 10,454,751 B1
• Filed: 06/27/2013
• Issued: 10/22/2019
• Est. Priority Date: 06/27/2013
• Status: Active Grant

• Current Assignee: Emc IP Holding Company LLC (Dell Technologies Inc.)
• Original Assignee: EMC Corporation (Dell Technologies Inc.)
• Inventors: Bent, John M., Faibish, Sorin, Gupta, Uday, Pedone, James, Peng, Tao
• Primary Examiner(s): Recek, Jason D
• Application Number:
  US13/928,985
• Time in Patent Office: 2,308 Days
• Field of Search: 709217
• CPC Class Codes: H04L 67/1097 for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]

Abstract

A cluster file system comprises a burst buffer appliance coupled to a plurality of object storage servers via a network. The burst buffer appliance comprises a controller configured to implement storage tiering control functionality for at least first and second storage tiers comprising respective disjoint subsets of the plurality of object storage servers. The burst buffer appliance is thereby configured to control movement of data between the first and second storage tiers. The object storage servers in the first storage tier may be configured to interface with object storage targets of a first type and the object storage servers in the second storage tier may be configured to interface with object storage targets of a second type different than the first type. For example, the object storage targets of the first type may comprise non-volatile electronic storage devices such as flash storage devices, and the object storage targets of the second type may comprise disk storage devices.

Claims

1. An object storage device of a cluster file system, comprising:

a burst buffer appliance of said object storage device configured to communicate with a plurality of object storage servers of the cluster file system over a network;

the burst buffer appliance comprising at least first and second storage tiers and a controller to implement storage tiering control functionality for the at least first and second storage tiers of said burst buffer appliance, wherein the at least first and second storage tiers comprise respective disjoint subsets of persistent storage, wherein the first storage tier comprises persistent storage of a first persistent storage type and wherein the second storage tier comprises persistent storage of a second persistent storage type different than the first persistent storage type, wherein, upon a read request for a given requested data object stored in a predefined one of the at least first and second storage tiers of said burst buffer appliance, the controller moves the data from the predefined one of the at least first and second storage tiers to the other of the at least first and second storage tiers, wherein the burst buffer appliance further comprises a daemon for a parallel log-structured file system and wherein, upon a read request for a portion of a logical file striped across a plurality of object storage devices, notifies at least one daemon of at least one additional burst buffer appliance of at least one additional object storage device to pre-fetch at least one additional portion of said logical file that is stored by said at least one additional object storage device;

the burst buffer appliance thereby being configured to control movement of data between the first and second storage tiers of said burst buffer appliance; and

the burst buffer appliance being implemented utilizing at least one processing device coupled to a memory.

2. The object storage device of claim 1 wherein each of the object storage servers is configured to interface with at least one object storage target, and further wherein the object storage servers in the first storage tier are configured to interface with object storage targets of a first type and the object storage servers in the second storage tier are configured to interface with object storage targets of a second type different than the first type.

3. The object storage device of claim 2 wherein the object storage targets of the first type comprise respective non-volatile electronic storage devices including one or more of flash memory, magneto-resistive random-access memory (MRAM) and phase-change random access memory (PC-RAM).

4. The object storage device of claim 2 wherein the object storage targets of the second type comprise respective disk storage devices.

5. The object storage device of claim 1 wherein the burst buffer appliance is further configured to communicate with a plurality of clients over the network and to process requests from said clients.

6. The object storage device of claim 1 wherein the controller comprises a data placement and migration controller.

7. The object storage device of claim 1 wherein at least a portion of the controller is implemented using data migration software at least a portion of which is stored in said memory and executed by said at least one processing device.

8. The object storage device of claim 1 wherein the burst buffer appliance provides an interface to one or more of a parallel log-structured file system and said clustered file system.

9. A processing platform comprising the object storage device of claim 1.

10. The object storage device of claim 1 wherein said movement of data between the first and second storage tiers of said object storage device is based upon an amount of storage utilized in at least one of said first and second storage tiers.

11. A method comprising:

configuring a burst buffer appliance of an object storage device in a cluster file system to communicate with a plurality of object storage servers of the cluster file system over a network, wherein the burst buffer appliance comprises at least first and second storage tiers and a controller, wherein the burst buffer appliance further comprises a daemon for a parallel log-structured file system and wherein, upon a read request for a portion of a logical file striped across a plurality of object storage devices, notifies at least one daemon of at least one additional burst buffer appliance of at least one additional object storage device to pre-fetch at least one additional portion of said logical file that is stored by said at least one additional object storage device; and

implementing storage tiering control functionality in the burst buffer appliance for the at least first and second storage tiers of the burst buffer appliance, wherein the at least first and second storage tiers comprise respective disjoint subsets of persistent storage, wherein the first storage tier comprises persistent storage of a first persistent storage type and wherein the second storage tier comprises persistent storage of a second persistent storage type different than the first persistent storage type, wherein, upon a read request for a given requested data object stored in a predefined one of the at least first and second storage tiers of said burst buffer appliance, the controller moves the data from the predefined one of the at least first and second storage tiers to the other of the at least first and second storage tiers;

the burst buffer appliance thereby being configured to control movement of data between the first and second storage tiers of the burst buffer appliance.

12. The method of claim 11 further comprising moving data between the first and second storage tiers under control of the burst buffer appliance.

13. The method of claim 11 further comprising:

configuring the object storage servers in the first storage tier to interface with object storage targets of a first type; and

configuring the object storage servers in the second storage tier to interface with object storage targets of a second type different than the first type;

wherein the first and second different types of object storage targets provide respective different levels of performance.

14. The method of claim 11 wherein the burst buffer appliance provides an interface to a parallel log-structured file system.

15. The method of claim 11 wherein the burst buffer appliance provides an interface to said clustered file system.

16. A computer program product comprising a non-transitory machine-readable storage medium having encoded therein executable code of one or more software programs, wherein the one or more software programs when executed cause the burst buffer appliance to perform the steps of the method of claim 11.

17. The method of claim 11 wherein said movement of data between the first and second storage tiers of said object storage device is based upon an amount of storage utilized in at least one of said first and second storage tiers.

18. A cluster file system comprising:

a burst buffer appliance of an object storage device, wherein the burst buffer appliance comprises at least first and second storage tiers and a controller; and

a plurality of object storage servers coupled to the burst buffer appliance via a network;

the burst buffer appliance configured to implement storage tiering control functionality for the at least first and second storage tiers of the burst buffer appliance, wherein the at least first and second storage tiers comprise respective disjoint subsets of persistent storage, wherein the first storage tier comprises persistent storage of a first persistent storage type and wherein the second storage tier comprises persistent storage of a second persistent storage type different than the first persistent storage type, wherein, upon a read request for a given requested data object stored in a predefined one of the at least first and second storage tiers of said burst buffer appliance, the controller moves the data from the predefined one of the at least first and second storage tiers to the other of the at least first and second storage tiers, wherein the burst buffer appliance further comprises a daemon for a parallel log-structured file system and wherein, upon a read request for a portion of a logical file striped across a plurality of object storage devices, notifies at least one daemon of at least one additional burst buffer appliance of at least one additional object storage device to pre-fetch at least one additional portion of said logical file that is stored by said at least one additional object storage device;

the burst buffer appliance thereby being configured to control movement of data between the first and second storage tiers of the burst buffer appliance.

19. The system of claim 18 further comprising:

a plurality of object storage targets distributed over the first and second storage tiers;

wherein each of the object storage servers is configured to interface with a corresponding subset of one or more of the object storage targets; and

wherein the object storage servers in the first storage tier are configured to interface with object storage targets of a first type and the object storage servers in the second storage tier are configured to interface with object storage targets of a second type different than the first type.

20. The system of claim 18 wherein said movement of data between the first and second storage tiers of said object storage device is based upon an amount of storage utilized in at least one of said first and second storage tiers.