Pluggable architecture for embedding analytics in clustered in-memory databases

• US 9,659,108 B2
• Filed: 12/02/2014
• Issued: 05/23/2017
• Est. Priority Date: 12/02/2013
• Status: Active Grant

• Current Assignee: Finch Computing LLC (Qbase, LLC)
• Original Assignee: Qbase, LLC
• Inventors: Lightner, Scott, Weckesser, Franz
• Primary Examiner(s): Waldron, Scott A
• Application Number:
  US14/558,055
• Publication Number:
  US 20150154283A1
• Time in Patent Office: 903 Days
• CPC Class Codes: G06F 11/3055 Monitoring arrangements for monitoring the status of the computing system or of the computing system component, e.g. monitoring if the computing system is on, off, available, not available error or fault processing without redundancy G06F11/0703; error detection or correction by redundancy in data representation G06F11/08; error detection or correction by redundancy in operation G06F11/14; error detection or correction by redundancy in hardware G06F11/16
• CPC Class Codes: G06F 11/3409 for performance assessment
• CPC Class Codes: G06F 16/2365 Ensuring data consistency and integrity
• CPC Class Codes: G06F 16/2471 Distributed queries
• CPC Class Codes: G06F 16/903 Querying for retrieval from the web G06F16/953
• CPC Class Codes: G06F 16/9035 Filtering based on additional data, e.g. user or group profiles
• CPC Class Codes: G06F 16/9535 Search customisation based on user profiles and personalisation
• CPC Class Codes: G06F 16/9538 Presentation of query results
• CPC Class Codes: G06F 2201/80 Database-specific techniques

Abstract

Disclosed are pluggable, distributed computing-system architectures allowing for embedding analytics to be added or removed from nodes of a system hosting an in-memory database. The disclosed system includes an API that may be used to create customized, application specific analytics modules. The newly created analytics modules may be easily plugged into the in-memory database. Each user query submitted to the in-memory database may specify different analytics be applied with differing parameters. All analytics modules operate on the in-memory image of the data, inside the in-memory database platform. All the analytics modules, may be capable of performing on-the-fly analytics, which may allow a dynamic and comprehensive processing of search results.

Claims

1. An in-memory database system comprising:

one or more storage nodes comprising a non-transitory machine-readable storage media storing one or more records of an in-memory database, wherein the storage media of each respective storage node is a main memory of the respective storage node;

one or more analytics agent nodes comprising a processor executing an analytics module using a set of query results as input parameters responsive to receiving a request for analytics indicating the analytics module, wherein the set of query results contains binary data representing one or more records retrieved from the one or more storage nodes storing the one or more records; and

an analytics module datastore comprising a non-transitory machine-readable storage media storing one or more analytics modules, and a processor transmitting a new analytics module to the one or more analytics agent nodes,

wherein the one or more analytics agent nodes are in communication with one or more supervisor nodes, and wherein, upon the one or more analytics agent nodes receiving the new analytics module from a client computer, the one or more analytics agent nodes transmit a heartbeat signal to the one or more supervisor nodes automatically, the heartbeat signal indicating the one or more analytics agent nodes received the new analytics module, and

wherein each of the one or more storage nodes, the one or more analytics agent nodes, and the one or more supervisor nodes is a distinct node.

2. The system according to claim 1, further comprising a search manager node comprising a processor transmitting the set of query results to the one or more analytics agent nodes responsive to determining if a search query contains the request for analytics.

3. The system according to claim 2, wherein the search manager node identifies the one or more analytics agent nodes prior to transmitting the search query to a search conductor node.

4. The system according to claim 3, wherein the search manager node determines one or more analytics parameters to be applied prior to transmitting the search query to the search conductor node.

5. The system according to claim 2, wherein, responsive to the search manager determining if the search query contains the request for analytics:

identifying the one or more analytics agent nodes executing the analytics module indicated by the request for analytics.

6. The system according to claim 5, further comprising a search conductor node comprising a processor executing a search conductor query and transmitting a subset of query results containing one or more records of a storage node satisfying the search conductor query, in response to receiving the search conductor query from the search manager node, wherein the search manager node generates one or more search conductor queries according to the search query, and wherein the search manager transmits each respective search conductor query to each respective search conductor node.

7. The system according to claim 6, wherein the search manager node aggregates each of the respective subsets of query results received from the respective search conductors, thereby generating the set of query results.

8. The system according to claim 1, wherein the search manager node transmits to the analytics agent at least one record in the set of query results is compressed according to a binary data compression algorithm.

9. The system according to claim 1, wherein the analytics module is selected from the group consisting of:

a disambiguation module, a linking module, a scoring module, and a link on-the-fly module.

10. The system according to claim 1, further comprising a system interface node comprising a processor transmitting a new analytics module to the analytics module store responsive to receiving the new analytics module from a client computer.

11. The system according to claim 1, further comprising a system interface node comprising a processor validating data of the search query in accordance with one or more application programming interfaces.

12. The system according to claim 1, further comprising a system interface node storing the new analytics module in one or more analytics agents and generating one or more new analytics modules based on an application programming interface.

13. The system according to claim 1, further comprising a client computer comprising a processor transmitting one or more search queries to the search manager.

14. The system according to claim 1, further comprising the one or more supervisor nodes comprising a processor transmitting a new analytics module from the analytics module store to a new analytics agent according to a dependency tree associated with the new analytics agent stored in a dependency manager node.