Batch processing of oversubscribed system based on subscriber usage patterns

• US 9,311,139 B1
• Filed: 05/29/2015
• Issued: 04/12/2016
• Est. Priority Date: 05/29/2015
• Status: Active Grant

• Current Assignee: Flexera Software LLC
• Original Assignee: Flexera Software LLC
• Inventors: Lowery, Robert
• Primary Examiner(s): Coyer, Ryan
• Application Number:
  US14/726,269
• Time in Patent Office: 319 Days
• Field of Search: 718/101
• US Class Current: 1/1
• CPC Class Codes: G06F 16/254 Extract, transform and load [ETL] procedures, e.g. ETL data flows in data warehouses
• CPC Class Codes: G06F 9/4843 by program, e.g. task dispatcher, supervisor, operating system
• CPC Class Codes: G06F 9/4887 involving deadlines, e.g. rate based, periodic

Abstract

Some embodiments include a method of scheduling batch processing of a batch processing system based on subscriber usage patterns. The method includes steps of recording a last job commencement event for a subscriber when the batch processing system starts processing a batch process for the subscriber; recording a last usage event for the subscriber when the subscriber uses the batch processing system; in an event that a time period elapsed since the last usage event for the subscriber is less than a time period elapsed since the last job commencement event for the subscriber, placing a next batch process of the subscriber into a recently used queue; identifying the next batch process of the subscriber as the oldest batch process from the recently used queue; and start processing the identified batch process for the subscriber.

Claims

1. A computer-implemented method of scheduling batch processing of a batch processing system, comprising:

automatically recording subscriber usage events based on subscriber usage patterns collected in response to subscriber background usage, wherein said subscriber usage events are identified from any of subscriber interactions with the batch processing system via a user interface provided by the batch processing system, and when background usage for the subscriber occurs;

recording a last job commencement event for a subscriber when the batch processing system starts processing a batch process for the subscriber;

recording a last usage event for the subscriber when the subscriber uses the batch processing system;

in an event that a time period elapsed since the last usage event for the subscriber is less than a time period elapsed since the last job commencement event for the subscriber, placing a next batch process of the subscriber into a recently used queue;

identifying the next batch process of the subscriber as the oldest batch process from the recently used queue; and

starting processing of the identified batch process for the subscriber.

2. The computer-implemented method of claim 1, further comprising:

recording the last job commencement of the subscriber when the batch processing system starts processing the identified batch process for the subscriber; and

recording a time duration of a last batch process of the subscriber when the batch processing system finishes processing the identified batch process for the subscriber.

3. The computer-implemented method of claim 1, further comprising:

determining that there is no batch process left in the recently used queue;

identifying the oldest batch process from a low priority queue, the low priority queue having a lower priority than the recently used queue; and

start processing the identified oldest batch process from the low priority queue.

4. The computer-implemented method of claim 1, further comprising:

recording an expedite request event for the subscriber when the subscriber requests the batch processing system to expedite the next batch process of the subscriber;

in response to the expedite request event for the subscriber, placing the next batch process of the subscriber into an expedite queue, the expedite queue having a higher priority than the recently used queue;

identifying the next batch process of the subscriber as the oldest batch process from the expedite queue; and

start processing the identified batch process for the subscriber.

5. The computer-implemented method of claim 4, wherein the subscriber requests the batch processing system to expedite the next batch process of the subscriber through a web browser user interface provided by the batch processing system.

6. The computer-implemented method of claim 1, further comprising:

in an event that the time period elapsed since the last job commencement event for the subscriber is more than an adjusted overdue threshold, placing the next batch process of the subscriber into an overdue queue, the overdue queue having a higher priority than the recently used queue;

identifying the next batch process of the subscriber as the oldest batch process from the overdue queue; and

start processing the identified batch process for the subscriber.

7. The computer-implemented method of claim 6, wherein the adjusted overdue threshold equals a total cycle processing time times a limit factor, the total cycle processing time is the total time taken by the batch processing system to complete all batch processes of a last cycle, and the limit factor determines how far batch processes in the overdue queue are allowed to lapse before being processed.

8. The computer-implemented method of claim 7, further comprising:

dynamically adjusting the limit factor based on the load situation of the batch processing system to change a time window for allowing running batch processes from the recently used queue.

9. The computer-implemented method of claim 7, further comprising:

receiving a request from the subscriber for adjusting a level of service for the subscriber; and

adjusting the limit factor specifically for one or more batch processes of the subscriber in response to the request for adjusting the level of service for the subscriber.

10. The computer-implemented method of claim 1, wherein the subscriber uses the batch processing system by interacting with the batch processing system through a user interface provided by the batch processing system, or by executing a background usage task, or by interacting with the batch processing system through an application programming interface (API) provided by the batch processing system.

11. The computer-implemented method of claim 1, wherein the identified batch process for the subscriber is an extract, transform and load (ETL) process for data movement and transformation.

12. The computer-implemented method of claim 1, wherein the oldest batch process from the recently used queue is a batch process that has an earliest submission time among batch processes in the recently used queue.

13. A computer-implemented method of prioritizing batch processing based on user usage patterns, comprising:

automatically recording user usage events based on user usage patterns collected in response to user background usage, wherein said user usage events are identified from any of user interactions with the batch processing system via a user interface provided by the batch processing system, and when background usage for the user occurs;

determining, by a batch scheduler of a batch processing system, whether there are any overdue batch processes left in an overdue queue;

in response to the determination that there are overdue batch processes left in the overdue queue, instructing a batch processor of the batch processing system to start processing the oldest overdue batch process from the overdue queue;

determining, by the batch scheduler, whether there are any batch processes of users who recently used the batch processing system left in a recently used queue, the recently used queue having a lower priority than the overdue queue; and

in response to the determination that that there are batch processes left in the recently used queue, instructing the batch processor to start processing the oldest batch process from the recently used queue.

14. The computer-implemented method of claim 13, further comprising:

identifying a batch process for a user wherein a time period elapsed since the user'"'"'s last usage of the batch processing system is less than a time period elapsed since a last batch process commencement event for the user; and

moving the identified batch process into the recently used queue by the batch scheduler.

15. The computer-implemented method of claim 13, further comprising:

identifying an overdue batch process for a user wherein a time period elapsed since a last batch process commencement event for the user is more than an overdue threshold.

16. The computer-implemented method of claim 15, wherein the overdue threshold depends on a total time taken by the batch processing system to complete all batch processes of a last cycle, and a limit factor determining how far batch processes in the overdue queue are allowed to lapse before being processed.

17. The computer-implemented method of claim 13, further comprising:

moving a batch process for a user into an expedite queue in response to a user request to expedite, the expedite queue having a higher priority than the overdue queue;

determining, by the batch scheduler, whether there are any expedite batch processes left in the expedite queue; and

in response to the determination that there are expedite batch processes left in the expedite queue, instructing a batch processor to start processing the oldest expedite batch process from the expedite queue.

18. The computer-implemented method of claim 13, further comprising:

determining, by the batch scheduler, that there are no batch process left in the overdue queue and the recently used queue; and

instructing the batch processor to start processing the oldest batch process from a low priority queue, the low priority queue having a lower priority than the overdue queue and the recently used queue.

20. The computing device of claim 17, wherein the process further includes steps of:

moving a third batch process of a third user into an expedite queue, in response to a request from the third user to expedite the third batch process; and

start processing batch processes in the expedite queue before handling batch processes from the overdue queue and the recently used queue.

21. The computing device of claim 20, wherein the process further includes steps of:

moving batch processes that do not belong to the expedite queue, the overdue queue and the recently used queue into a low priority queue; and

start processing batch processes from the low priority queue when the expedite queue, the overdue queue and the recently used queue are empty.

22. The computing device of claim 17, wherein the adjusted overdue threshold depends on a total time taken by the computing device to complete all batch processes of a last cycle, and a limit factor determining how far batch processes in the overdue queue are allowed to lapse before being processed.

19. A computing device, comprising:

a processor;

a user interface configured to interact with users of the computing device;

a batch scheduler module;

a memory storing executable instructions implementing the batch scheduler module which, when executed by the processor, is configured to perform the process of;

automatically recording user usage events based on user usage patterns collected in response to user background usage, wherein said user usage events are identified from any of user interactions with the batch processing system via a user interface provided by the batch processing system, and when background usage for the user occurs;

monitoring user usage events for the users of the computing device,

recording batch process commencement events for the users and time durations of last batch processes for the users,

determining that a time period elapsed since the last batch process commencement event for a first user is more than an adjusted overdue threshold, and moving a first batch process of the first user into an overdue queue,

determining that a time period elapsed since the last usage event for a second user is less than a time period elapsed since the last batch process commencement event for the second user, and moving a second batch process of the second user into a recently used queue, and

starting executing batch processes in the recently used queue when the overdue queue is empty.