Querying operating system state on multiple machines declaratively

• US 9,747,333 B2
• Filed: 10/08/2014
• Issued: 08/29/2017
• Est. Priority Date: 10/08/2014
• Status: Active Grant

• Current Assignee: Cloudera Incorporated
• Original Assignee: Cloudera Incorporated
• Inventors: Zeyliger, Philip
• Primary Examiner(s): GORTAYO, DANGELINO N
• Application Number:
  US14/510,006
• Publication Number:
  US 20160103874A1
• Time in Patent Office: 1,056 Days
• Field of Search: 707722, 707737, 707758, 707769, 707602, 707607, 707770, 707713
• CPC Class Codes: G06F 16/2282 Tablespace storage structures; Management thereof
• CPC Class Codes: G06F 16/245 Query processing
• CPC Class Codes: G06F 16/2453 Query optimisation
• CPC Class Codes: G06F 16/24532 of parallel queries
• CPC Class Codes: G06F 16/2455 Query execution
• CPC Class Codes: G06F 16/2465 Query processing support for facilitating data mining operations in structured databases
• CPC Class Codes: G06F 16/2471 Distributed queries
• CPC Class Codes: G06F 16/248 Presentation of query results
• CPC Class Codes: G06F 16/252 between a Database Management System and a front-end application
• CPC Class Codes: G06F 16/951 Indexing; Web crawling techniques
• CPC Class Codes: G06F 3/0482 Interaction with lists of selectable items, e.g. menus
• CPC Class Codes: G06F 3/04842 Selection of displayed objects or displayed text elements G06F3/0482 takes precedence

Abstract

A sysSQL technology for querying operating system states of multiple hosts in a cluster using a Structured Query Language (SQL) query is disclosed. An administrator of a cluster can use a graphical or text-based user interface to submit an SQL query to determine the operating system states of multiple hosts in parallel. The technology parses the SQL query to determine the datasets needed to execute the SQL query and aggregates those datasets from the multiple hosts. The technology then creates a temporary database to execute the SQL query and provides the results from the SQL query for display on the user interface.

Claims

1. A method of querying operating system states of multiple hosts in a cluster, comprising

receiving a Structure Query Language (SQL) query to determine the operating system states of the multiple hosts in the cluster;

aggregating, from each of the multiple hosts, operating system state information in response to receiving the SQL query;

wherein aggregating includes, for each of the multiple hosts;

identifying a dataset applicable to the SQL query by parsing the SQL query;

requesting the identified dataset; and

receiving the dataset including the operating system state information in response to the request;

creating a temporary database for storing the operating system state information by converting the datasets received from the multiple hosts into a queriable form; and

executing the SQL query on the temporary database to obtain as results the operating system states of the multiple hosts.

2. The method of claim 1, wherein the SQL query is received from a user via one of a graphical user interface, a graphical web-based user interface or a text-based user interface accessible to an administrator of the cluster.

3. The method of claim 1, further comprising displaying the results in a table on a web-based user interface, wherein the operating system state information includes a table including data fields defined by a schema.

4. The method of claim 1, wherein

the datasets received from each of the multiple hosts are tables.

5. The method of claim 4, wherein each of the multiple hosts creates the table using system information in a /proc file system according to a table schema.

6. A method, comprising:

receiving a system Structure Query Language (sysSQL) query directed to multiple machines in a cluster;

wherein the sysSQL query is submitted by a user using a user interface; and

wherein the sysSQL query is a request for system information associated with the multiple machines; and

creating a database, in response to receiving the sysSQL query, by aggregating system information from some or all of the multiple machines, wherein aggregating includes, for some or all of the multiple machines;

identifying a dataset applicable to the sysSQL query by parsing the sysSQL query;

requesting the identified dataset;

receiving the dataset including the system information in response to the request; and

converting the received dataset into a queriable form; and

executing the sysSQL query on the database to obtain as results system information associated with some or all of the multiple machines.

7. The method of claim 6, further comprising:

discarding the database after obtaining the results.

8. The method of claim 7,

wherein the datasets received from some or all of the multiple machines are tables.

9. The method of claim 6, wherein each of the multiple machines uses an application programming interface (API) call to obtain an initial dataset and formats the initial dataset to provide the dataset identified based on the sysSQL query.

10. The method of claim 6, wherein the system information is associated with at least one of:

processes, threads, connections, hosts, mounts or network interfaces.

11. The method of claim 6, further comprising displaying the results in a table on the graphical user interface that is web-based.

12. The method of claim 6, wherein the user interface is one of:

a text-based user interface or a graphical user interface.

13. The method of claim 6, further comprising:

detecting that a request to at least one of the multiple machines to provide a dataset identified based on the sysSQL query has timed out;

wherein the database is created without system information from the at least one of the multiple machines.

14. A system comprising:

a server; and

multiple hosts in a cluster;

the server being configured to;

receive a Structure Query Language (SQL) query requesting operating system state information associated with the multiple hosts in the cluster, the SQL query being submitted via a user interface;

aggregate, from each of the multiple hosts, operating system state information in response to receiving the SQL query, wherein aggregating includes;

identifying datasets applicable to the SQL query by parsing the SQL query;

requesting the identified datasets from agents of the multiple hosts; and

receiving the requested datasets including the operating state information in response to the requests;

create a temporary database by converting the received datasets into a queriable form; and

execute the SQL query on the temporary database to obtain the operating system state information associated with the multiple hosts in the cluster.

15. The system of claim 14, wherein the server is further configured to generate a graphical diagram to visualize the operating system state information associated with the multiple hosts.

16. The system of claim 14, wherein the server is further configured to display the operating system state information in a tabular format on the user interface.

17. The system of claim 14, wherein the server is configured to identify at least one dataset needed for the query and request the at least one dataset from each of the multiple hosts.

18. The system of claim 17, wherein each host is configured to query a virtual file system to obtain runtime system information and format the runtime system information using a schema to obtain the at least one dataset.

19. The system of claim 17, wherein the at least one dataset includes processes, threads, connections, hosts, mounts or network interfaces dataset.

20. The system of claim 14, wherein the user interface is one of a text-based user interface or a graphical user interface.