Job scheduling and monitoring in a distributed computing environment

• US 10,606,665 B2
• Filed: 05/16/2017
• Issued: 03/31/2020
• Est. Priority Date: 12/16/2014
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: Peacock, Andrew J., Couris, Cheryl, Storm, Christina, Netz, Amir, Cheung, Chiu Ying, Flasko, Michael J., Grealish, Kevin, Della-Libera, Giovanni M., Carlson, Sonia P., Heninger, Mark W., Bach, Paula M., Nettleton, David J.
• Primary Examiner(s): Khatri, Anil
• Application Number:
  US15/596,709
• Publication Number:
  US 20170286185A1
• Time in Patent Office: 1,050 Days
• Field of Search: 718 1, 718100-108
• CPC Class Codes: G06F 3/0482 Interaction with lists of selectable items, e.g. menus
• CPC Class Codes: G06F 3/04842 Selection of displayed objects or displayed text elements G06F3/0482 takes precedence
• CPC Class Codes: G06F 9/52 Program synchronisation; Mutual exclusion, e.g. by means of semaphores
• CPC Class Codes: G06Q 10/06311 Scheduling, planning or task assignment for a person or group

Abstract

Job execution can be scheduled and monitored. Execution of a job results in generation of one or more data slices, which can be consumed as input by one or more related jobs. Data slices can be presented in an interactive view. Selection of a data slice in the view can trigger automatic identification of related data slices based on dependencies between data slices. Each data slice can also include and be presented with a status indicating the availability of corresponding data, such as pending, ready, or failed. Furthermore, selection of a data slice that failed to generate successfully can trigger identification of related data slices that failed or are predicted to fail to generate successfully.

Claims

1. A method performed by a system comprising at least one processor coupled to a memory storing machine-executable instructions, which, when executed by the processor, control the system to perform acts comprising:

conveying, for display on a display device, a timeline view of a data-transformation job schedule in terms of one or more data slices comprising pieces of data of a continuum comprising a data set, wherein each data slice is generated or consumed by a data-transformation job and includes a length, and start and end times;

receiving a first signal from a user selecting a data slice from the one or more data slices in the view;

determining one or more related data slices to the user selected data slice automatically based on data dependencies, wherein the one or more related data slices comprise at least one of one or more data slices that depend from the user selected data slice or one or more data slices from which the user selected data slice depends; and

conveying, for display on the display device, a visualization of the user selected data slice and the one or more related data slices visually distinct from one or more other data slices in the timeline view.

2. The method of claim 1, wherein the acts further comprise conveying, for display on the display device, one or more data slices scheduled for generation by a job.

3. The method of claim 1, wherein the acts further comprise conveying, for display on the display device, one or more data slices previously generated by a job.

4. The method of claim 1, wherein the acts further comprise conveying, for display on the display device, previously generated data slices visually distinct from data slices scheduled for generation by a job.

5. The method of claim 1, wherein the acts further comprise:

determining compute load during generation of one or more data slices; and

conveying, for display on the display device, the compute load aligned with the one or more data slices.

6. The method of claim 1, wherein the acts further comprise conveying, for display on the display device, an indication that at least one data sliced failed to generate successfully.

7. The method of claim 6, wherein the acts further comprise:

receiving a second signal selecting one of at least one data slice that failed to generate successfully;

determining automatically one or more related data slices related to a selected data slice, wherein the related data slices include at least one of one or more data slices from which the selected data slice depends that failed to generate successfully or one or more data slices predicted to fail to generate successfully based on dependency from the selected data slice; and

conveying, for display on the display device, a highlighted version of the selected data slice and the one or more related data slices.

8. The method of claim 6, wherein the acts further comprise:

receiving a second signal selecting one of the at least one data slice that failed to generate successfully; and

rescheduling generation of the at least one data slice in response to the second signal.

9. The method of claim 1, wherein the acts further comprise conveying, for display on the display device, the selected data slice and the one or more related data slices in a job authoring diagram comprising representations of jobs and data sets as well as connections between the jobs and data sets.

10. A computer system, comprising:

a processor coupled to a memory, the processor configured to execute machine-executable instructions stored in the memory that when executed perform acts comprising;

conveying, for display on a display device, a timeline view of a data-transformation job schedule in terms of one or more data slices comprising pieces of data of a continuum comprising a data set, wherein each data slice is generated or consumed by a data-transformation job and includes a length, and start and end times;

detecting a first signal from a user selecting a data slice from the one or more data slices in the view;

determining one or more related data slices to the user selected data slice automatically based on data dependencies, wherein the one or more related data slices comprise at least one of one or more data slices that depend from the user selected data slice or one or more data slices from which the user selected data slice depends; and

conveying, for display on the display device, a visualization of the user selected data slice and the one or more related data slices visually distinct from one or more other data slices in the view.

11. The system of claim 10, wherein the acts further comprise:

monitoring computer resource utilization in conjunction with data slice production; and

conveying, for display on the display device, a visualization of the resource utilization aligned with one or more respective data slices.

12. The system of claim 10, wherein the acts further comprise:

determining success or failure of data slice generation; and

presenting failed data slice generation different from successful data slice generation.

13. The system of claim 10, wherein the acts further comprise rescheduling generation of a data slice based on a signal received with respect to a failed data slice generation.

14. The system of claim 10, wherein the acts further comprise:

detecting a second signal selecting one of at least one data slice where generation failed;

determining automatically one or more related data slices that are related to a selected data slice and predicted to fail based on dependency on the selected data slice; and

conveying, for display on the display device, a highlighted version of the selected data slice and the one or more related data slices.

15. The system of claim 10, wherein the acts further comprise conveying, for display on the display device, the selected data slice and the one or more related data slices in a job authoring diagram comprising representations of jobs and data sets as well as connections between the jobs and data sets.

16. A computer-readable storage medium having instructions stored thereon that enable at least one processor to perform a method upon execution of the instructions, the method comprising:

conveying, for display on a display device, a timeline view of a data-transformation job schedule in terms of one or more data slices comprising pieces of data of a continuum comprising a data set, wherein each data slice is generated or consumed by a data-transformation job and includes a length, and start and end times;

receiving a first signal from a user selecting a data slice from the one or more data slices in the view;

determining one or more related data slices to the user selected data slice automatically based on data dependencies, wherein the one or more related data slices comprise at least one of one or more data slices that depend from the user selected data slice or one or more data slices from which the user selected data slice depends; and

conveying, for display on the display device, a visualization of the user selected data slice and the one or more related data slices visually distinct from one or more other data slices in the view.

17. The computer-readable storage medium of claim 16, wherein the method further comprises conveying, for display on the display device, previously produced data slices visually distinct from data slices scheduled for generation.

18. The computer-readable storage medium of claim 16, wherein the method further comprises:

determining compute load during production of one or more data slices; and

conveying, for display on the display device, the compute load aligned with the one or more data slices.

19. The computer-readable storage medium of claim 16, wherein the method further comprises conveying, for display on the display device, an indication that at least one data sliced failed to generate successfully.

20. The computer-readable storage medium of claim 16, wherein the method further comprises:

detecting a second signal selecting one of at least one data slice that failed to generate successfully;

determining automatically one or more related data slices related to a selected data slice, wherein the related data slices include at least one of one or more data slices from which the selected data slice depends that failed to generate successfully or one more data slices predicted to fail to generate successfully based on dependency from the selected data slice; and

conveying, for display on the display device, a highlighted version of the selected data slice and the one or more related data slices.