Managing I/O operations in a shared file system

• US 9,772,877 B2
• Filed: 12/29/2015
• Issued: 09/26/2017
• Est. Priority Date: 06/27/2013
• Status: Active Grant

• Current Assignee: Lenovo Global Technologies International Ltd. (Lenovo Group Ltd.)
• Original Assignee: Lenovo Singapore Pte Limited (Lenovo Group Ltd.)
• Inventors: Enault, Ludovic, Michel, Eric, Tallet, Nicolas, Vezolle, Pascal
• Primary Examiner(s): Cao, Phuong Thao
• Application Number:
  US14/983,097
• Publication Number:
  US 20160110219A1
• Time in Patent Office: 637 Days
• Field of Search: 707827
• CPC Class Codes: G06F 11/1464 for networked environments
• CPC Class Codes: G06F 13/1642 with request queuing
• CPC Class Codes: G06F 16/18 File system types
• CPC Class Codes: G06F 16/182 Distributed file systems
• CPC Class Codes: G06F 16/1858 Parallel file systems, i.e. file systems supporting multiple processors
• CPC Class Codes: G06F 2209/486 Scheduler internals
• CPC Class Codes: G06F 2209/5021 Priority
• CPC Class Codes: G06F 9/4881 Scheduling strategies for dispatcher, e.g. round robin, multi-level priority queues
• CPC Class Codes: G06F 9/5044 considering hardware capabilities
• CPC Class Codes: H04L 63/102 Entity profiles
• CPC Class Codes: H04L 67/303 Terminal profiles

Abstract

A method for managing I/O operations in a shared file system environment. The method includes receiving for each of a plurality of compute nodes, information associated with I/O accesses to a shared file system, and applications for executing the I/O accesses. The method includes creating application profiles, based, at least in part, on the received information. The method then includes determining execution priorities for the application, based, at least in part, on the created application profiles.

Claims

1. A method for managing I/O operations in a shared file system environment, the method comprising:

receiving, by a plurality of I/O masters executed by a computer processor, from an I/O agent running on each of a plurality of compute nodes organized into a plurality of subsets, information associated with I/O accesses to a shared file system and applications for executing the I/O accesses, wherein each I/O master is responsible for and receives the information from a corresponding subset of the plurality of compute nodes;

creating, by an analytics engine executed by the computer processor, application profiles, based, at least in part, on the received information;

transferring the application profiles from the analytics engine to a job scheduler executed by the computer processor;

determining, by the job scheduler, application execution priorities for the applications, based, at least in part, on the created application profiles;

determining, by the job scheduler, resource allocations for the application execution priorities based, at least in part, on the created application profiles,

wherein the application execution priorities and the resource allocations are used to manage the I/O operations in the shared file system environment.

2. The method of claim 1, wherein the received information includes:

application names, types of applications running on each of the plurality of compute nodes, number of compute nodes, application execution types, I/O resource requirements, I/O access frequency, I/O access patterns, and I/O real time operations information.

3. The method of claim 1, further comprising:

storing the received information for each of the plurality of compute nodes and the created application profiles in a database for the shared file system environment.

4. The method of claim 1, further comprising:

determining, by the job scheduler, a job schedule, based, at least in part, on the determined application execution priorities for the applications;

communicating the job schedule from the job scheduler to the I/O collector through the analytics engine;

communicating from the I/O collector to each I/O master a portion of the job schedule pertaining to the compute nodes of the subset of compute nodes for which the I/O master is responsible; and

communicating by each I/O master the portion of the job schedule received by the I/O master to the compute nodes of the subset of compute nodes for which the I/O master is responsible.

5. A computer program product for managing I/O operations in a shared file system environment, the computer program product comprising:

one or more non-transitory computer-readable storage media and program instructions stored on the one or more non-transitory computer-readable tangible storage media, the program instructions when executing by a processor of an I/O subsystem manager causes the processor to perform a method comprising;

receiving, by a plurality of I/O masters executed by a computer processor, from an I/O agent running on each of a plurality of compute nodes organized into a plurality of subsets, information associated with I/O accesses to a shared file system and applications for executing the I/O accesses, wherein each I/O master is responsible for and receives the information from a corresponding subset of the plurality of compute nodes;

creating, by an analytics engine executed by the computer processor, application profiles, based, at least in part, on the received information;

transferring the application profiles from the analytics engine to a job scheduler executed by the computer processor;

determining, by the job scheduler, application execution priorities for the applications, based, at least in part, on the created application profiles;

determining, by the job scheduler, resource allocations for the application execution priorities based, at least in part, on the created application profiles,

wherein the application execution priorities and the resource allocations are used to manage the I/O operations in the shared file system environment.

6. The computer program product of claim 5, wherein the received information includes:

application names, types of applications running on each of the plurality of compute nodes, number of compute nodes, application execution types, I/O resource requirements, I/O access frequency, I/O patterns, and I/O real time operations information.

7. The computer program product of claim 5, wherein the method further comprises storing the received information for each of the plurality of compute nodes and the created application profiles in a database for the shared file system environment.

8. The computer program product of claim 5, wherein the method further comprises:

determining, by the job scheduler, a job schedule, based, at least in part, on the determined execution priorities for the applications;

communicating the job schedule from the job scheduler to the I/O collector through the analytics engine;

communicating from the I/O collector to each I/O master a portion of the job schedule pertaining to the compute nodes of the subset of compute nodes for which the I/O master is responsible; and

communicating, by each I/O master the portion of the job schedule received by the I/O master to the compute nodes of the subset of compute nodes for which the I/O master is responsible.

9. A computer system for managing I/O operations in a shared file system environment, the computer system comprising:

one or more computer processors;

one or more non-transitory computer-readable storage media;

program instructions stored on the one or more non-transitory computer-readable storage media that the computer processors execute to realize a plurality of I/O masters, an I/O collector, an analytics engine, and a job scheduler so that;

the I/O masters receive, from an I/O agent running on each of a plurality of compute nodes organized into a plurality of subsets, information associated with I/O accesses to a shared file system and applications for executing the I/O accesses, wherein each I/O master is responsible for and receives the information from a corresponding subset of the plurality of compute nodes;

the analytics engine creates application profiles, based, at least in part, on the received information;

the analytics engine transfers the application profiles to the job scheduler;

the job scheduler determines application execution priorities for the applications, based, at least in part, on the created application profiles;

the job scheduler determines resource allocations for the application execution priorities based, at least in part, on the created application profiles,

wherein the application execution priorities and the resource allocations are used to manage the I/O operations in the shared file system environment.

10. The computer system of claim 9, wherein the received information includes:

application names, types of applications running on each of the plurality of compute nodes, number of compute nodes, application execution types, I/O resource requirements, I/O access frequency, I/O patterns, and I/O real time operations information.

11. The computer system of claim 9, further comprising a database within which the received information for each of the plurality of compute nodes and the created application profiles are stored.

12. The computer system of claim 9, wherein:

the job scheduler determines a job schedule, based, at least in part, on the determined application execution priorities for the applications;

the job scheduler communicates the job schedule to the I/O collector through the analytics engine;

the I/O collector communicates to each I/O master a portion of the job schedule pertaining to the compute nodes of the subset of compute nodes for which the I/O master is responsible; and

each I/O master communicates the portion of the job schedule received by the I/O master to the compute nodes of the subset of compute nodes for which the I/O master is responsible.