×

Method of processing big data, including arranging icons in a workflow GUI by a user, checking process availability and syntax, converting the workflow into execution code, monitoring the workflow, and displaying associated information

  • US 10,175,954 B2
  • Filed: 09/24/2014
  • Issued: 01/08/2019
  • Est. Priority Date: 09/24/2013
  • Status: Active Grant
First Claim
Patent Images

1. A method of processing big data, the method comprising:

  • displaying, on a display, a plurality of action icons and a plurality of flow icons, the plurality of action icons and the plurality of flow icons being graphically arranged to represent a non-cyclic order for the plurality of the action icons to generate a workflow, the plurality of action icons being associated with executable functions for a framework for distributed storage and processing of big data including at least one of a distributed file system (DSF) input function, a DSF output function, an expression function, or an aggregator function, and checking a sequential process availability for a pair of action icons when a specific flow icon between the pair of action icons is arranged,wherein checking a sequential process availability further includes sequentially processing the workflow when all requisite action icons are arranged in the workflow between the pair of action icons, and refusing sequential process availability when all requisite action icons are not arranged in the workflow between the pair of action icons;

    checking, by a controller, a syntax for each of the plurality of the arranged action icons and the plurality of the flow icons, the syntax being dependent on an action icon for which the syntax is checked or an action icon corresponding to a flow icon for which the syntax is checked;

    generating, by the controller, an alarm badge for display on a corresponding action icon for which the alarm badge is generated, the alarm badge indicating a syntax error for the corresponding action icon for which the alarm badge is generated when the syntax error is found;

    converting the workflow into a workflow definition document when a syntax check for the generated workflow is completed and storing the workflow definition document in a workflow database;

    converting the workflow definition document into an execution code and analyzing the execution code to determine an action icon;

    transferring the execution code for the determined action icon to a job tracker module to automatically receive a processing progress status including a completion time of the determined action icon; and

    providing process work information for a plurality of the generated workflow through a work monitoring menu included in a workflow work environment, wherein the process work information includes an ID, a status whether a process succeeded or failed, a start time, an end time, a generation time, and a latest change time for the generated workflow,wherein displaying the plurality of the action icons and the plurality of the flow icons further includes receiving a setting parameter for the framework for distributed storage and processing of big data associated with a map-reduce process, andwherein at least one of the plurality of the action icons is associated with a big data process and at least one of the plurality of the action icons is associated with a flow process,wherein displaying the plurality of the action icons and the plurality of the flow icons further includes providing metadata for a specific action icon from an icon window when the specific action icon in the workflow is selected, and storing a sequential execution procedure for the metadata,wherein transparency for the icon window is adjusted according to a movement speed in which an icon window is initially popped up for display based on selection of the specific action icon, wherein when the movement speed is slow the transparency is increased and when the movement speed is fast the transparency is decreased, andwherein the icon window includesa first window having an input metadata region and an output metadata region, the input metadata region displaying a plurality of map-reduce input items for the specific action icon associated with input metadata and the output metadata region displays a plurality of items associated with output metadata for an operation, expression and execution sequence,a second window that displays a configuration region for the framework including at least one key and value associated with an operation mode of the map-reduce process, anda third window that displays an input-output location of distribution storage, wherein displaying the plurality of the action icons and the plurality of the flow icons further includes setting the input-output location of the distribution storage associated with the map-reduce process in the third window.

View all claims
  • 1 Assignment
Timeline View
Assignment View
    ×
    ×