Real-time analytics based monitoring and classification of jobs for a data processing platform
First Claim
1. A method comprising:
- selecting, by a computing device, a new job to schedule for execution on a data processing system, the new job including a classification in a plurality of classifications, wherein the classification is determined by;
using a process to analyze a set of operations to determine which operation is to be used to classify the current job, wherein a first operation in the set of operations is selected; and
classifying the first operation in a first classification based on resource usage for the first operation, wherein the first classification is determined based on a resource being used by the first operation in a highest percentage usage in the data processing platform compared to other resources used by the first operation;
retrieving, by the computing device, performance information for a set of current jobs that are being executed in the data processing system, wherein the set of jobs are assigned to a plurality of queues and currently classified with a current classification in the plurality of classifications;
analyzing, by the computing device, the performance information to determine when one or more current jobs in the set of current jobs should be re-classified due to resource usage of a respective current job when being executed in the data processing system, wherein analyzing comprises;
determining a second operation in the set of operations; and
determining that the first classification should be changed to the second classification when a resource being used by the second operation has a higher percentage usage in the data processing platform compared to the highest percentage usage for the first operation;
re-classifying, by the computing device, the classifications for the one or more current jobs in the plurality of queues, wherein the first classification is re-classified to the second classification for the first operation; and
assigning, by the computing device, the new job to one of the queues based on the classification of the new job and the classifications of jobs in the plurality of queues including the re-classified classifications for the one or more current jobs.
1 Assignment
0 Petitions
Accused Products
Abstract
In one embodiment, a method selects a new job to schedule for execution on a data processing system. The new job includes a. Performance information for a set of current jobs that are being executed in the data processing system is retrieved where the set of jobs are assigned to queues and currently classified with a current classification. The method analyzes the performance information to determine when one or more current jobs in the set of current jobs should be re-classified due to resource usage of a respective current job when being executed in the data processing system and re-classifies the classifications for the one or more current jobs in the queues. Then, the new job is assigned to one of the queues based on the classification of the new job and the classifications of jobs in the queues including the re-classified classifications for the one or more current jobs.
-
Citations
20 Claims
-
1. A method comprising:
-
selecting, by a computing device, a new job to schedule for execution on a data processing system, the new job including a classification in a plurality of classifications, wherein the classification is determined by; using a process to analyze a set of operations to determine which operation is to be used to classify the current job, wherein a first operation in the set of operations is selected; and classifying the first operation in a first classification based on resource usage for the first operation, wherein the first classification is determined based on a resource being used by the first operation in a highest percentage usage in the data processing platform compared to other resources used by the first operation; retrieving, by the computing device, performance information for a set of current jobs that are being executed in the data processing system, wherein the set of jobs are assigned to a plurality of queues and currently classified with a current classification in the plurality of classifications; analyzing, by the computing device, the performance information to determine when one or more current jobs in the set of current jobs should be re-classified due to resource usage of a respective current job when being executed in the data processing system, wherein analyzing comprises; determining a second operation in the set of operations; and determining that the first classification should be changed to the second classification when a resource being used by the second operation has a higher percentage usage in the data processing platform compared to the highest percentage usage for the first operation; re-classifying, by the computing device, the classifications for the one or more current jobs in the plurality of queues, wherein the first classification is re-classified to the second classification for the first operation; and assigning, by the computing device, the new job to one of the queues based on the classification of the new job and the classifications of jobs in the plurality of queues including the re-classified classifications for the one or more current jobs. - View Dependent Claims (2, 3, 4, 5, 6, 7, 8, 9, 10)
-
-
11. A non-transitory computer-readable storage medium containing instructions, that when executed, control a computer system to be configured for:
-
selecting a new job to schedule for execution on a data processing system, the new job including a classification in a plurality of classifications, wherein the classification is determined by; using a process to analyze a set of operations to determine which operation is to be used to classify the current job, wherein a first operation in the set of operations is selected; and classifying the first operation in a first classification based on resource usage for the first operation, wherein the first classification is determined based on a resource being used by the first operation in a highest percentage usage in the data processing platform compared to other resources used by the first operation; retrieving performance information for a set of current jobs that are being executed in the data processing system, wherein the set of jobs are assigned to a plurality of queues and currently classified with a current classification in the plurality of classifications; analyzing the performance information to determine when one or more current jobs in the set of current jobs should be re-classified due to resource usage of a respective current job when being executed in the data processing system, wherein analyzing comprises; determining a second operation in the set of operations; and determining that the first classification should be changed to the second classification when a resource being used by the second operation has a higher percentage usage in the data processing platform compared to the highest percentage usage for the first operation; re-classifying the classifications for the one or more current jobs in the plurality of queues, wherein the first classification is re-classified to the second classification for the first operation; and assigning the new job to one of the queues based on the classification of the new job and the classifications of jobs in the plurality of queues including the re-classified classifications for the one or more current jobs. - View Dependent Claims (12, 13, 14, 15, 16, 17, 18, 19)
-
-
20. An apparatus comprising:
-
one or more computer processors; and a non-transitory computer-readable storage medium comprising instructions, that when executed, control the one or more computer processors to be configured for; selecting a new job to schedule for execution on a data processing system, the new job including a classification in a plurality of classifications, wherein the classification is determined by; using a process to analyze a set of operations to determine which operation is to be used to classify the current job, wherein a first operation in the set of operations is selected; and classifying the first operation in a first classification based on resource usage for the first operation, wherein the first classification is determined based on a resource being used by the first operation in a highest percentage usage in the data processing platform compared to other resources used by the first operation; retrieving performance information for a set of current jobs that are being executed in the data processing system, wherein the set of jobs are assigned to a plurality of queues and currently classified with a current classification in the plurality of classifications; analyzing the performance information to determine when one or more current jobs in the set of current jobs should be re-classified due to resource usage of a respective current job when being executed in the data processing system, wherein analyzing comprises; determining a second operation in the set of operations; and determining that the first classification should be changed to the second classification when a resource being used by the second operation has a higher percentage usage in the data processing platform compared to the highest percentage usage for the first operation; re-classifying the classifications for the one or more current jobs in the plurality of queues, wherein the first classification is re-classified to the second classification for the first operation; and assigning the new job to one of the queues based on the classification of the new job and the classifications of jobs in the plurality of queues including the re-classified classifications for the one or more current jobs.
-
Specification