Real-time distributed in memory search architecture
First Claim
1. A system comprising one or more nodes hosting an in-memory database, the system comprising:
- a plurality of storage nodes comprising non-transitory machine-readable storage medium storing one or more partitions of a collection, wherein the collection stored by each respective storage node contains one or more records of a database, and wherein the storage medium of each respective storage node comprises main memory;
one or more partitioner nodes partitioning one or more collections of database records and distributing the one or more collections to the one or more storage nodes according to a schema file responsive to receiving the one or more collections;
a search manager node comprising a processor generating one or more search conductor queries using a search query received from a user node, transmitting the one or more search conductor queries to one or more search conductor nodes according to the search query, and forward one or more sets of search results to one or more analytics agent nodes according to the search query responsive to receive the one or more sets of search results;
an analytics agent node comprising a processor executing one or more analytics algorithms responsive to receiving a set of search results from the search manager node;
a search conductor node comprising a processor querying the collection of the database records of a storage node according to a search conductor query in response to receiving the search conductor query from the search manager, and transmitting the set of one or more search results to the search manager node in response to identifying the one or more search results of the set, wherein each respective search result corresponds to a database record containing data satisfying the search conductor query;
a plurality of network segments comprising one or more connections between one or more nodes communicating over each respective network segment, wherein a first network segment comprises the search manager, the search conductor, and the analytics agent;
a dependency manager comprising non-transitory machine-readable storage medium storing one or more configuration package files and a dependency tree, wherein the one or more configuration package files attempts to minimize cost based on actual need, and a processor configured to determine a configuration package file for a failed node and transmit the configuration package file to a supervisor node based on the dependency tree in response to a request for the configuration package file from the supervisor node;
wherein a third network segment further comprises the one or more connections communicating the request for the configuration package file; and
the one or more supervisor nodes comprising the processor continuously monitoring one or more statuses associated with the one or more nodes of the system, and configured to transmit the configuration package file to the failed node of the one or more nodes in response to receiving a status indicating a failed resource of the failed node.
2 Assignments
0 Petitions
Accused Products
Abstract
Disclosed here are distributed computing system connection configurations having multiple connection bandwidth and latency tiers. Also disclosed are connection configurations including a suitable number of network segments, where network segments may be connected to external servers and clusters including search managers, analytics agents, search conductors, dependency managers, supervisors, and partitioners, amongst others. In one or more embodiments, modules may be connected to the network segments using a desired bandwidth and latency tier. Disclosed here are hardware components suitable for running one or more types of modules on one or more suitable nodes. One or more suitable hardware components included in said clusters include CPUs, Memory, and Hard Disk, amongst others.
-
Citations
10 Claims
-
1. A system comprising one or more nodes hosting an in-memory database, the system comprising:
-
a plurality of storage nodes comprising non-transitory machine-readable storage medium storing one or more partitions of a collection, wherein the collection stored by each respective storage node contains one or more records of a database, and wherein the storage medium of each respective storage node comprises main memory; one or more partitioner nodes partitioning one or more collections of database records and distributing the one or more collections to the one or more storage nodes according to a schema file responsive to receiving the one or more collections; a search manager node comprising a processor generating one or more search conductor queries using a search query received from a user node, transmitting the one or more search conductor queries to one or more search conductor nodes according to the search query, and forward one or more sets of search results to one or more analytics agent nodes according to the search query responsive to receive the one or more sets of search results; an analytics agent node comprising a processor executing one or more analytics algorithms responsive to receiving a set of search results from the search manager node; a search conductor node comprising a processor querying the collection of the database records of a storage node according to a search conductor query in response to receiving the search conductor query from the search manager, and transmitting the set of one or more search results to the search manager node in response to identifying the one or more search results of the set, wherein each respective search result corresponds to a database record containing data satisfying the search conductor query; a plurality of network segments comprising one or more connections between one or more nodes communicating over each respective network segment, wherein a first network segment comprises the search manager, the search conductor, and the analytics agent; a dependency manager comprising non-transitory machine-readable storage medium storing one or more configuration package files and a dependency tree, wherein the one or more configuration package files attempts to minimize cost based on actual need, and a processor configured to determine a configuration package file for a failed node and transmit the configuration package file to a supervisor node based on the dependency tree in response to a request for the configuration package file from the supervisor node;
wherein a third network segment further comprises the one or more connections communicating the request for the configuration package file; and
the one or more supervisor nodes comprising the processor continuously monitoring one or more statuses associated with the one or more nodes of the system, and configured to transmit the configuration package file to the failed node of the one or more nodes in response to receiving a status indicating a failed resource of the failed node. - View Dependent Claims (2, 3, 4, 5, 6, 7, 8, 9, 10)
-
Specification