System and method for analyzing big data activities
First Claim
1. A system, comprising:
- a processor and a memory;
an application manager running on the processor that identifies resources used by an application for big data running on a cluster and provides MapReduce attempt success and failure totals for a workload;
a workflow manager running on the processor that identifies problems with the application by providing a status of how each workflow is performing against a service level agreement;
a deep events application running on the processor that provides root-cause analysis and solutions for application problems and system-level problems;
a live alerts application running on the processor that provides alerts on the application, each workflow, users, tables, and queues;
an information tag application running on the processor that tags the application, each workflow, the tables, the users, and entities;
an entity search application running on the processor that indexes and associates cluster activity with the application, each workflow, the tables, the users, the queues, machines, and the entities; and
a planner running on the processor that simulates execution of the workload to identify a cheapest and a fastest infrastructure for the workload.
1 Assignment
0 Petitions
Accused Products
Abstract
A system and method for analyzing big data activities are disclosed. According to one embodiment, a system comprises a distributed file system for the entities and applications, wherein the applications include one or more of script applications, structured query language (SQL) applications, Not Only (NO) SQL applications, stream applications, search applications, and in-memory applications. The system further comprises a data processing platform that gathers, analyzes, and stores data relating to entities and applications. The data processing platform includes an application manager having one or more of a MapReduce Manage, a script applications manager, a structured query language (SQL) applications manager, a Not Only (NO) SQL applications manager, a stream applications manager, a search applications manager, and an in-memory applications manager. The application manager identifies if the applications are one or more of slow-running, failed, killed, unpredictable, and malfunctioning.
10 Citations
14 Claims
-
1. A system, comprising:
-
a processor and a memory; an application manager running on the processor that identifies resources used by an application for big data running on a cluster and provides MapReduce attempt success and failure totals for a workload; a workflow manager running on the processor that identifies problems with the application by providing a status of how each workflow is performing against a service level agreement; a deep events application running on the processor that provides root-cause analysis and solutions for application problems and system-level problems; a live alerts application running on the processor that provides alerts on the application, each workflow, users, tables, and queues; an information tag application running on the processor that tags the application, each workflow, the tables, the users, and entities; an entity search application running on the processor that indexes and associates cluster activity with the application, each workflow, the tables, the users, the queues, machines, and the entities; and a planner running on the processor that simulates execution of the workload to identify a cheapest and a fastest infrastructure for the workload. - View Dependent Claims (2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14)
-
Specification