Method and apparatus for processing computational task having multiple subflows

• US 10,146,588 B2
• Filed: 06/17/2016
• Issued: 12/04/2018
• Est. Priority Date: 01/14/2014
• Status: Active Grant

• Current Assignee: Tencent Technology Company Limited (Tencent Holdings Limited)
• Original Assignee: Tencent Technology Company Limited (Tencent Holdings Limited)
• Inventors: Zhou, Dongxiang
• Primary Examiner(s): Teets, Bradley A
• Application Number:
  US15/185,722
• Publication Number:
  US 20160299791A1
• Time in Patent Office: 900 Days
• Field of Search: None
• CPC Class Codes: G06F 2209/548 Queue
• CPC Class Codes: G06F 9/5055 considering software capabilities, i.e. software resources associated or available to the machine
• CPC Class Codes: G06F 9/52 Program synchronisation; Mutual exclusion, e.g. by means of semaphores
• CPC Class Codes: G06F 9/54 Interprogram communication

Abstract

A method and an apparatus for processing a computational task is described. The method may include obtaining a computational task that includes multiple subflows. The method includes storing input data of a current subflow of the computational task in a queue and obtaining first input data from the queue and performing data processing according to logic corresponding to the current subflow. The method includes determining whether to enter a next subflow according to a result of the data processing, and, if a result enters a next subflow, storing the result of the data processing in another queue as input data of a next subflow of the current subflow.

Claims

1. A method for processing a computational task, performed at a computing apparatus having one or more processors and a memory for storing executable instructions to be executed by the one or more the processors, the method comprising:

obtaining the computational task that comprises multiple subflows;

storing input data of a current subflow of the computational task in a queue;

creating multiple task processing threads for each subflow;

obtaining first input data from the queue and performing data processing according to logic corresponding to the current subflow;

wherein different pieces of the first input data are obtained from the queue and the data processing of the different pieces of the first input data are performed according to the logic corresponding to the current subflow separately in the multiple task processing threads in parallel;

determining whether to enter a next subflow according to a result of the data processing;

when determining to enter the next subflow, storing the result of the data processing in another queue as input data of the next subflow of the current subflow, and entering the next subflow; and

decreasing a difference between processing speeds of the current subflow and the next subflow such that a congested state or an idle time of at least one of the multiple task processing threads for the current subflow or the multiple task processing threads for the next subflow is reduced, by adjusting one or a combination of a number of the multiple task processing threads for the current subflow and a number of the multiple task processing threads for the next subflow.

2. The method according to claim 1, wherein the adjusting comprises:

adjusting the one or the combination of the number of the multiple task processing threads for the current subflow and the number of the multiple task processing threads for the next subflow according to at least one number input by a user via a user interface.

3. The method according to claim 1, wherein the adjusting comprises:

adjusting dynamically the one or the combination of the number of the multiple task processing threads for the current subflow and the number of the multiple task processing threads for the next subflow according to the length of the another queue.

4. The method according to claim 1, before the determining whether to enter the next subflow according to the result of the data processing, further comprising:

determining whether an exception occurs in the data processing, and when the exception occurs in the data processing, caching the first input data.

5. The method according to claim 1, wherein

the first input data is computed by a previous subflow of the multiple subfloors, and

the method further includes

determining whether an exception occurs in the data processing, and

when the exception occurs in the data processing, storing the first input data back into the queue.

6. The method according to claim 1, wherein the adjusting comprises:

adjusting the one or the combination of the number of the multiple task processing threads for the current subflow and the number of the multiple task processing threads for the next subflow to minimize a difference between processing speeds of the current subflow and the next subfiow.

7. An apparatus for processing a computational task, the apparatus comprising:

one or more processors;

a memory; and

a plurality of processor-implemented instructions stored in the memory and to be executed by the one or more processors, wherein the one or more processors execute the processor-implemented instructions to;

receive the computational task that comprises multiple subflows;

store input data of a current subflow of the computational task in a queue;

create multiple task processing threads for each subflow;

obtain first input data from the queue and perform data processing according to logic corresponding to the current subflow;

wherein different pieces of the first input data are obtained from the queue and the data processing of the different pieces of the first input data are performed according to the logic corresponding to the current subflow separately in the multiple task processing threads in parallel;

determine whether to enter a next subflow according to a result of the data processing;

when determining to enter the next subflow, store the result of the data processing in another queue as input data of the next subflow of the current subflow, and enter the next subflow; and

decrease a difference between processing speeds of the current subflow and the next subflow such that a congested state or an idle time of at least one of the multiple task processing threads for the current subflow or the multiple task processing threads for the next subflow is reduced, by adjusting one or a combination of a number of the multiple task processing threads for the current subflow and a number of the multiple task processing threads for the next subflow.

8. The apparatus for processing a computational task according to claim 7, wherein the one or more processors execute the processor-implemented instructions to adjust the one or the combination of the number of the multiple task processing threads for the current subflow and the number of the multiple task processing threads for the next subflow according to at least one number input by a user via a user interface.

9. The apparatus for processing a computational task according to claim 7, wherein the one or more processors execute the processor-implemented instructions to adjust dynamically the one or the combination of the number of the multiple task processing threads for the current subflow and the number of the multiple task processing threads for the next subflow according to the length of the another queue.

10. The apparatus for processing a computational task according to claim 7, wherein the one or more processors execute the processor-implemented instructions to further

determine whether an exception occurs in the data processing;

and

when the exception occurs, cache the first input data.

11. A non-transitory computer-readable storage medium comprising executable instructions which when executed by one or more processors of a computing apparatus, cause the computing apparatus to perform a method for processing a computational task, the method comprising:

obtaining the computational task that comprises multiple subflows;

storing input data of a current subflow of the computational task in a queue;

creating multiple task processing threads for each subflow;

obtaining first input data from the queue and performing data processing according to logic corresponding to the current subflow;

wherein different pieces of the first input data are obtained from the queue and the data processing of the different pieces of the first input data are performed according to the logic corresponding to the current subflow separately in the multiple task processing threads in parallel;

determining whether to enter a next subflow according to a result of the data processing;

when determining to enter the next subflow, storing the result of the data processing in another queue as input data of the next subflow of the current subflow, and entering the next subflow; and

decreasing a difference between processing speeds of the current subflow and the next subflow such that a congested state or an idle time of at least one of the multiple task processing threads for the current subflow or the multiple task processing threads for the next subflow is reduced, by adjusting one or a combination of a number of the multiple task processing threads for the current subflow and a number of the multiple task processing threads for the next subflow.

12. The non-transitory computer-readable storage medium according to claim 11, before the determining whether to enter the next subflow according to the result of the data processing, further comprising:

determining whether an exception occurs in the data processing, and when the exception occurs in the data processing, caching the first input data.

13. The non-transitory computer-readable storage medium according to claim 11, wherein the adjusting comprises:

adjusting the one or the combination of the number of the multiple task processing threads for the current subflow and the number of the multiple task processing threads for the next subflow according to at least one number input by a user via a user interface.

14. The non-transitory computer-readable storage medium according to claim 11, wherein the adjusting comprises:

adjusting dynamically the one or the combination of the number of the multiple task processing threads for the current subflow and the number of the multiple task processing threads for the next subflow according to the length of the another queue.