DATA PROCESSING SYSTEM, DATA PROCESSING METHOD, AND PROGRAM

• US 20150254105A1
• Filed: 10/29/2013
• Published: 09/10/2015
• Est. Priority Date: 10/31/2012
• Status: Active Grant

• Current Assignee: NEC Corporation
• Original Assignee: NEC Corporation
• Inventors: Nishimura, Shoji
• Granted Patent: US 9,430,285 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 16/00 Information retrieval; Database structures therefor; File system structures therefor
• CPC Class Codes: G06F 2209/5017 Task decomposition
• CPC Class Codes: G06F 9/4881 Scheduling strategies for dispatcher, e.g. round robin, multi-level priority queues
• CPC Class Codes: G06F 9/4887 involving deadlines, e.g. rate based, periodic
• CPC Class Codes: G06F 9/5066 Algorithms for mapping a plurality of inter-dependent sub-tasks onto a plurality of physical CPUs mappping at compile time, see G06F8/451
• CPC Class Codes: G06F 9/5088 involving task migration

Abstract

Disclosed are a data processing system, etc., whereby total processing time of a data process including a parallel process and a serial process is optimized. This data processing system comprises: a task segmenting means for receiving a plurality of tasks which execute, upon a record set which is a set of a plurality of records, a data process which includes parallel processes which a plurality of computers are capable of executing in parallel and serial processes which are post-processes of the parallel processes and is executed by one computer, and, for a parallel process which is the pre-process of a serial process, among the serial processes which are included in the task, which is envisioned as requiring more time to process than another serial process, by subdividing the record set which is to be processed in the parallel process, segmenting the parallel process into a plurality of sub-tasks; and an assignment means for assigning the plurality of sub-tasks to the plurality of computers, such that the plurality of sub-tasks are executed prior to a task other than the task which includes the parallel processes being executed.

Claims

1. A data processing system comprising:

task dividing unit receiving a plurality of tasks in which a data processing including a parallel processing that can be performed by a plurality of computers in parallel and a serial processing that is a post-processing of the parallel processing and performed by one computer is performed to a record set that is a set of a plurality of records and dividing the parallel processing that is a preprocessing of the serial processing to which it is expected that the processing time thereof is greater than that of the serial processing included in another task into a plurality of sub tasks by subdividing the record set that is a processing target of the parallel processing and

assignment unit assigning a plurality of the sub tasks to a plurality of the computers so that a plurality of the sub tasks are performed before the task other than the task including the parallel processing is performed.

2. The data processing system described in claim 1 further comprising job reception unit receiving a job in which the data processing is performed to a data set that is a set of a plurality of records and

job division unit dividing the job into a plurality of tasks by subdividing the data set that is a processing target of the job into a plurality of record sets, wherein the task dividing unit receives a plurality of the divided tasks.

3. The data processing system described in claim 1 wherein when the processing time of the task is shorter than the processing time of the serial processing with the longest processing time, a parallel processing included in the task is not divided.

4. The data processing system described in claim 1 wherein the task dividing unit divides the parallel processing so as to differentiate the number of the records processed by the sub task and the assignment unit assigns the sub task of which the number of the records processed therein is greater than the number of the records processed in another sub task to a computer to which the serial processing that is a post-processing of the parallel processing is assigned.

5. The data processing system described in claim 1 wherein the assignment unit assigns the sub task that includes a first record processed in the serial processing as the processing target to the computer to which the serial processing that is the post-processing of the parallel processing is assigned.

6. The data processing system described in claim 1 wherein

the assignment unit outputs schedule data in which the computer that is an assignment destination is associated with a priority to each of the task, the sub task, and the serial processing that is the post-processing of the sub task from a plurality of the computers in a referable form;

the computer includes

schedule data acquisition unit acquiring the schedule data and

task processing unit performing the assigned task in a decreasing order of the priority included in the schedule data; and

when the assigned task has already been performed and the task assigned to another computer is not performed, the task processing unit performs the task that is not performed.

7. The data processing system described in claim 1 wherein

the assignment unit outputs schedule data in which the computer that is the assignment destination is associated with the record that is the processing target to each of the task, the sub task, and the serial processing that is the post-processing of the sub task from a plurality of the computers in a referable form and

the computer includes

record acquisition unit acquiring the record,

schedule data acquisition unit acquiring the schedule data and

record transmission unit for transmitting the acquired record to the computer that is the assignment destination of the processing in which the record is used as the processing target on the basis of the acquired schedule data.

8. A data processing method including a division step in which a plurality of tasks in which a data processing including a parallel processing that can be performed by a plurality of computers in parallel and a serial processing that is a post-processing of the parallel processing and performed by one computer is performed to a record set that is a set of a plurality of records are received and the parallel processing that is a preprocessing of the serial processing to which it is expected that the processing time thereof is greater than that of the serial processing included in another task is divided into a plurality of sub tasks by subdividing the record set that is a processing target of the parallel processing and

an assignment step in which a plurality of the sub tasks are assigned to a plurality of the computers so that a plurality of the sub tasks are performed before the task other than the task including the parallel processing is performed.

9. A non-transitory computer readable medium program that causes a computer serving as an data processing system storing a program for causing a computer to execute a process to:

perform a division process in which a plurality of tasks in which a data processing including a parallel processing that can be performed by a plurality of computers in parallel and a serial processing that is a post-processing of the parallel processing and performed by one computer is performed to a record set that is a set of a plurality of records are received and the parallel processing that is a preprocessing of the serial processing to which it is expected that the processing time thereof is greater than that of the serial processing included in another task is divided into a plurality of sub tasks by subdividing the record set that is a processing target of the parallel processing and

an assignment process in which a plurality of the sub tasks are assigned to a plurality of the computers so that a plurality of the sub tasks are performed before the task other than the task including the parallel processing is performed.