METHOD OF PROCESSING BIG DATA, APPARATUS PERFORMING THE SAME AND STORAGE MEDIA STORING THE SAME

• US 20150089415A1
• Filed: 09/24/2014
• Published: 03/26/2015
• Est. Priority Date: 09/24/2013
• Status: Active Grant

• Current Assignee: LG CNS Company Limited (LBHunet Co., Ltd.)
• Original Assignee: LG CNS Company Limited (LBHunet Co., Ltd.)
• Inventors: PARK, Jong Won, LEE, Joo Youl, KANG, Hyun Sang, KIM, Ki Do, OH, Seok Keun, CHOI, Don Jung
• Granted Patent: US 10,175,954 B2
• Time in Patent Office: Days
• US Class Current: 715/765
• CPC Class Codes: G06F 8/34 Graphical or visual programming
• CPC Class Codes: G06Q 10/0633 Workflow analysis

Abstract

Provided is a method of processing a big data which may include arranging a plurality of action icons from a palette and a plurality of flow icons representing a non-cyclic order for the plurality of the action icons to generate a workflow, checking a syntax for the plurality of the arranged action icons and the plurality of the flow icons, the syntax being dependent on a corresponding action icon and graphically generating an alarm badge indicating a syntax error for a specific icon when the syntax error for the specific icon is found. Therefore, the method and apparatus may use a Hadoop framework to effectively manage a workflow for processing irregular big data.

Claims

1. A method of processing big data, the method comprising:

displaying, on a display, a plurality of action icons and a plurality of flow icons on a display, the plurality of action icons and the plurality of flow icons being graphically arranged to represent a non-cyclic order for the plurality of the action icons to generate a workflow;

checking, by the controller, a syntax for the plurality of the arranged action icons and the plurality of the flow icons, the syntax being dependent on a corresponding action icon; and

generating, by the controller, an alarm badge for display on a corresponding action icon, the alarm badge indicating a syntax error for a specific icon when the syntax error for the specific icon is found.

2. The method of claim 1, wherein displaying the plurality of the action icons and the plurality of the flow icons on the display includes checking a sequential process availability for a pair of action icons when a specific flow icon between the pair of the action icons is arranged.

3. The method of claim 1, wherein displaying the plurality of the action icons and the plurality of the flow icons further includes

providing a metadata for a specific action icon from an icon window when the specific action icon in the workflow is selected;

and

storing a sequential execution procedure for the metadata.

4. The method of claim 3, wherein displaying the plurality of the action icons and the plurality of the flow icons further includes receiving a Hadoop setting parameter associated with a map-reduce operation for the specific action icon.

5. The method of claim 4, wherein displaying the plurality of the action icons and the plurality of the flow icons further includes setting an input-output location of Hadoop distribution storage associated with the map-reduce work for the specific action icon.

6. The method of claim 1, wherein at least one of the plurality of the action icons is associated with a big data process and at least one of the plurality of the action icons is associated with a flow process.

7. The method of claim 1, wherein checking the syntax includes checking whether an execution procedure being used in the plurality of the arranged action icons and dependent flow icons thereof are defined or available.

8. The method of claim 1, wherein generating the alarm badge further includes determining a type of the syntax error for a specific icon to determine the alarm badge corresponding to a type of the syntax error.

9. The method of claim 1, further comprising:

adjusting a transparency for the icon window according to a movement speed in which a pop-up icon window is popped up for the specific icon.

10. The method of claim 1, further comprising:

providing process work information for the generated workflow through a work monitoring menu included in a workflow work environment.

11. The method of claim 1, further comprising:

defining metadata for generating the workflow for the plurality of the action icons through a file metadata menu included in the workflow work environment.

12. The method of claim 11, further comprising:

checking whether the metadata defined on the file metadata menu is changed; and

providing a workflow list related to the changed metadata through an effect analysis screen for the metadata included in the workflow work environment when a change of the metadata is checked.

13. The method of claim 1, further comprising:

converting the workflow into a workflow definition document when a syntax check for the generated workflow is completed and storing the workflow definition document in a workflow database.

14. The method of claim 13, further comprising:

converting the workflow definition document into an execution code and analyzing the execution code to determine an action icon; and

transferring the execution code for the determined action icon to a job tracker module to receive whether an execution for the determined action icon is completed.

15. The method of claim 14, further comprising:

storing a result for a corresponding map-reduce process in a map-reduce database when a map-reduce process for the action icon is completed.

16. An apparatus of processing big data, the apparatus comprising:

a workflow writing unit configured to arrange a plurality of action icons and a plurality of flow icons representing a non-cyclic order for the plurality of the action icons to generate a workflow;

an icon syntax checking unit configured to check a syntax for the plurality of the arranged action icons and the plurality of the flow icons, the syntax being dependent on a corresponding action icon; and

an alarm badge generating unit configured to graphically generate an alarm badge that indicates a syntax error for a specific icon when the syntax error for the specific icon is found.

17. The apparatus of claim 16, wherein at least one of the plurality of the action icons is associated with a big data process and at least one of the plurality of the action icons is associated with a flow process.

18. The apparatus of claim 16, further comprising:

a window transparency adjusting unit configured to adjust a transparency for an icon window according to a movement speed of the icon window being popped up for the specific icon.

19. The apparatus of claim 16, further comprising:

a workflow database storing unit configured to convert the workflow into a workflow definition document when a syntax check for the generated workflow is completed and store the workflow definition document in a workflow database.

20. The apparatus of claim 19, further comprising:

an execution completion transferring unit configured to convert the workflow definition document into an execution code, to analyze the execution code to determine an action icon and to transfer the execution code for the determined action icon to a job tracker module to determine whether an execution for the determined action icon is completed.

21. The apparatus of claim 20, further comprising:

a map-reduce database storing unit configured to store a result for a corresponding map-reduce process in a map-reduce database when a map-reduce process for the action icon is completed.

22. A machine-readable non-transitory medium storing a computer program for a method of processing a big data being performed in an apparatus of processing the big data, the machine-readable non-transitory medium having stored thereon machine-executable instructions for:

displaying, on a display, a plurality of action icons and a plurality of flow icons on a display, the plurality of action icons and the plurality of flow icons being graphically arranged to represent a non-cyclic order for the plurality of the action icons to generate a workflow;

checking, by the controller, a syntax for the plurality of the arranged action icons and the plurality of the flow icons, the syntax being dependent on a corresponding action icon; and

generating, by the controller, an alarm badge for display on a corresponding action icon, the alarm badge indicating a syntax error for a specific icon when the syntax error for the specific icon is found.