Burst buffer appliance comprising multiple virtual machines

• US 9,069,594 B1
• Filed: 12/27/2012
• Issued: 06/30/2015
• Est. Priority Date: 12/27/2012
• Status: Active Grant

• Current Assignee: Emc IP Holding Company LLC (Dell Technologies Inc.)
• Original Assignee: EMC Corporation (Dell Technologies Inc.)
• Inventors: Faibish, Sorin, Bent, John M.
• Primary Examiner(s): Zaman, Faisal M
• Application Number:
  US13/728,421
• Time in Patent Office: 915 Days
• Field of Search: 710/104, 710/306, 710/313, 710/314, 718/1, 718/103, 718/102, 718/100
• US Class Current: 1/1
• CPC Class Codes: G06F 12/0246 in block erasable memory, e.g. flash memory
• CPC Class Codes: G06F 13/1673 using buffers
• CPC Class Codes: G06F 13/34 with priority control
• CPC Class Codes: G06F 13/40 Bus structure for computer networks G06F15/163; for optical bus networks H04B10/25
• CPC Class Codes: G06F 16/1858 Parallel file systems, i.e. file systems supporting multiple processors
• CPC Class Codes: G06F 2009/4557 Distribution of virtual machine instances; Migration and load balancing
• CPC Class Codes: G06F 2009/45579 I/O management, e.g. providing access to device drivers or storage
• CPC Class Codes: G06F 2212/7201 Logical to physical mapping or translation of blocks or pages
• CPC Class Codes: G06F 9/45558 Hypervisor-specific management and integration aspects

Abstract

A burst buffer appliance is adapted for coupling between a computer system and a file system. The burst buffer appliance comprises a flash memory or other high-speed memory having a substantially lower access time than the file system, and is configured to include a plurality of virtual machines for processing respective different types of input-output operations that involve utilization of the high-speed memory, with each of the virtual machines providing a different performance level for its associated type of input-output operations. The performance levels provided by the plurality of virtual machines may comprise respective different quality of service (QoS) levels for the respective different types of input-output operations, specified in terms of parameters such as latency and throughput rate. A highest QoS level may be provided by a particular virtual machine for operations involving writing checkpoints from the computer system to the high-speed memory.

Claims

1. An apparatus comprising:

a burst buffer appliance adapted for coupling between a computer system and a file system;

the burst buffer appliance comprising a high-speed memory having a substantially lower access time than the file system;

the burst buffer appliance being configured to include a plurality of virtual machines for processing respective different types of input-output operations that involve utilization of the high-speed memory;

wherein each of the virtual machines provides a different performance level for its associated type of input-output operations;

wherein the plurality of virtual machines comprises;

a first virtual machine for processing a first type of input-output operations that involve utilization of the high-speed memory; and

at least a second virtual machine for processing at least a second type of input-output operations that involve utilization of the high-speed memory;

wherein the first type of input-output operations is associated with a first priority and the second type of input-output operations is associated with a second priority lower than the first priority;

wherein the first type of input-output operations comprises operations involving writing checkpoints from the computer system to the high-speed memory; and

wherein the second type of input-output operations comprises operations involving at least one of;

writing sensor data from a sensor network to the high-speed memory;

writing or reading analytic engine data to or from the high-speed memory; and

writing from the high-speed memory to the file system.

2. The apparatus of claim 1 wherein the computer system comprises a super computer having a plurality of compute nodes and the burst buffer appliance is adapted for coupling between at least one of the compute nodes and the file system.

3. The apparatus of claim 1 wherein the file system comprises a cluster file system comprising one of a parallel file system and a distributed file system.

4. The apparatus of claim 1 wherein the high-speed memory comprises a flash memory.

5. The apparatus of claim 1 wherein the performance levels provided by the plurality of virtual machines comprise respective different quality of service (QoS) levels for the respective different types of input-output operations.

6. The apparatus of claim 1 wherein the performance level provided by a given one of the virtual machines is characterized by at least one of a specified latency and a specified throughput rate for processing the associated type of input-output operations utilizing the high-speed memory.

7. The apparatus of claim 1 wherein the first virtual machine provides a first performance level for operations involving writing checkpoints from the computer system to the high-speed memory.

8. The apparatus of claim 7 wherein the first performance level comprises a highest QoS level of a plurality of QoS levels corresponding to the respective performance levels provided by the respective virtual machines of the plurality of virtual machines.

9. The apparatus of claim 7 wherein the second virtual machine provides a different performance level than the first performance level for operations involving writing sensor data from a sensor network to the high-speed memory.

10. The apparatus of claim 7 wherein the second virtual machine provides a different performance level than the first performance level for operations involving writing or reading analytic engine data to or from the high-speed memory.

11. The apparatus of claim 7 wherein the second virtual machine provides a different performance level than the first performance level for operations involving writing from the high-speed memory to the file system.

12. A processing platform that incorporates the apparatus of claim 1.

13. A method comprising:

configuring a burst buffer appliance adapted for coupling between a computer system and a file system to include a high-speed memory having a substantially lower access time than the file system; and

configuring the burst buffer appliance to include a plurality of virtual machines for processing respective different types of input-output operations that involve utilization of the high-speed memory;

wherein each of the virtual machines provides a different performance level for its associated type of input-output operations;

wherein the plurality of virtual machines comprises;

a first virtual machine for processing a first type of input-output operations that involve utilization of the high-speed memory; and

at least a second virtual machine for processing at least a second type of input-output operations that involve utilization of the high-speed memory;

wherein the first type of input-output operations is associated with a first priority and the second type of input-output operations is associated with a second priority lower than the first priority;

wherein the first type of input-output operations comprises operations involving writing checkpoints from the computer system to the high-speed memory; and

wherein the second type of input-output operations comprises operations involving at least one of;

writing sensor data from a sensor network to the high-speed memory;

writing or reading analytic engine data to or from the high-speed memory; and

writing from the high-speed memory to the file system.

14. The method of claim 13 wherein configuring the burst buffer appliance to include a plurality of virtual machines comprises configuring the first virtual machine to provide a first performance level for operations involving writing checkpoints from the computer system to the high-speed memory.

15. The method of claim 14 wherein configuring the burst buffer appliance to include a plurality of virtual machines comprises configuring the second virtual machine to provide a different performance level than the first performance level for operations involving writing sensor data from a sensor network to the high-speed memory.

16. The method of claim 14 wherein configuring the burst buffer appliance to include a plurality of virtual machines comprises configuring the second virtual machine to provide a different performance level than the first performance level for operations involving writing or reading analytic engine data to or from the high-speed memory.

17. The method of claim 14 wherein configuring the burst buffer appliance to include a plurality of virtual machines comprises configuring the second virtual machine to provide a different performance level than the first performance level for operations involving writing from the high-speed memory to the file system.

18. A computer program product comprising a processor-readable storage medium having encoded therein executable code of one or more software programs, wherein the one or more software programs when executed cause the server to perform the steps of the method of claim 13.

19. An information processing system comprising:

a computer system;

a file system; and

a burst buffer appliance coupled between the computer system and the file system, the burst buffer appliance comprising;

a high-speed memory having a substantially lower access time than the file system; and

a plurality of virtual machines for processing respective different types of input-output operations that involve utilization of the high-speed memory;

wherein each of the virtual machines provides a different performance level for its associated type of input-output operations;

wherein the plurality of virtual machines comprises;

a first virtual machine for processing a first type of input-output operations that involve utilization of the high-speed memory; and

at least a second virtual machine for processing at least a second type of input-output operations that involve utilization of the high-speed memory;

wherein the first type of input-output operations is associated with a first priority and the second type of input-output operations is associated with a second priority lower than the first priority;

wherein the first type of input-output operations comprises operations involving writing checkpoints from the computer system to the high-speed memory; and

wherein the second type of input-output operations comprises operations involving at least one of;

writing sensor data from a sensor network to the high-speed memory;

writing or reading analytic engine data to or from the high-speed memory; and

writing from the high-speed memory to the file system.

20. The system of claim 19 wherein the computer system comprises a super computer having a plurality of compute nodes and the burst buffer appliance is coupled between at least one of the compute nodes and the file system.

21. The system of claim 19 wherein the file system comprises a cluster file system comprising one of a parallel file system and a distributed file system.