Hybrid cluster-based data intake and query
First Claim
Patent Images
1. A computer-implemented method comprising:
- receiving, at a first cluster, a search query, the first cluster being a first data intake and query system;
transmitting, through a firewall of either the first cluster or a cloud-based cluster, a request for information identifying a plurality of indexers of the cloud-based cluster, the cloud-based cluster being a second data intake and query system;
in response to the request, obtaining, from the cloud-based cluster, the information identifying the plurality of indexers, wherein the first cluster and the cloud-based cluster each include at least one master node that is knowledgeable about active indexers within its respective cluster, and the information identifies the plurality of indexers based on the at least one master node of the cloud-based cluster identifying the active indexers;
distributing the search query to the plurality of indexers of the cloud-based cluster and one or more indexers of the first cluster, said distributing using the obtained information identifying the plurality of indexers and being through the firewall; and
receiving, at the first cluster, a response to the distributed search query from at least one of the plurality of indexers of the cloud-based cluster wherein each response from a respective indexer is produced by the respective indexer based on an evaluation, by the respective indexer, of the distributed search query.
1 Assignment
0 Petitions
Accused Products
Abstract
Various embodiments describe multi-site cluster-based data intake and query systems, including cloud-based data intake and query systems. Using a hybrid search system that includes cloud-based data intake and query systems working in concert with so-called “on-premises” data intake and query systems can promote the scalability of search functionality. In addition, the hybrid search system can enable data isolation in a manner in which sensitive data is maintained “on premises” and information or data that is not sensitive can be moved to the cloud-based system. Further, the cloud-based system can enable efficient leveraging of data that may already exist in the cloud.
-
Citations
47 Claims
-
1. A computer-implemented method comprising:
-
receiving, at a first cluster, a search query, the first cluster being a first data intake and query system; transmitting, through a firewall of either the first cluster or a cloud-based cluster, a request for information identifying a plurality of indexers of the cloud-based cluster, the cloud-based cluster being a second data intake and query system; in response to the request, obtaining, from the cloud-based cluster, the information identifying the plurality of indexers, wherein the first cluster and the cloud-based cluster each include at least one master node that is knowledgeable about active indexers within its respective cluster, and the information identifies the plurality of indexers based on the at least one master node of the cloud-based cluster identifying the active indexers; distributing the search query to the plurality of indexers of the cloud-based cluster and one or more indexers of the first cluster, said distributing using the obtained information identifying the plurality of indexers and being through the firewall; and receiving, at the first cluster, a response to the distributed search query from at least one of the plurality of indexers of the cloud-based cluster wherein each response from a respective indexer is produced by the respective indexer based on an evaluation, by the respective indexer, of the distributed search query. - View Dependent Claims (2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16)
-
-
17. A non-transitory computer readable storage medium, storing software instructions, which when executed by one or more processors, perform operations comprising:
-
receiving, at a first cluster, a search query, the first cluster being a first data intake and query system; transmitting, through a firewall of either the first cluster or a cloud-based cluster, a request for information identifying a plurality of indexers of the cloud-based cluster, the cloud-based cluster being a second data intake and query system; in response to the request, obtaining, from the cloud-based cluster, the information identifying the plurality of indexers, wherein the first cluster and the cloud-based cluster each include at least one master node that is knowledgeable about active indexers within its respective cluster, and the information identifies the plurality of indexers based on the at least one master node of the cloud-based cluster identifying the active indexers; distributing the search query to the plurality of indexers of the cloud-based cluster and one or more indexers of the first cluster, said distributing using the obtained information identifying the plurality of indexers and being through the firewall; and receiving, at the first cluster, a response to the distributed search query from at least one of the plurality of indexers of the cloud-based cluster wherein each response from a respective indexer is produced by the respective indexer based on an evaluation, by the respective indexer, of the distributed search query. - View Dependent Claims (18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29)
-
-
30. A computer-implemented system, comprising:
-
one or more processors; one or more non-transitory computer readable storage media; computer readable instructions stored on the one or more non-transitory computer readable storage media which, when executed by the one or more processors, implement a first cluster configured to perform operations comprising; receiving, at a first cluster, a search query, the first cluster being a first data intake and query system; transmitting, through a firewall of either the first cluster or a cloud-based cluster, a request for information identifying a plurality of indexers of the cloud-based cluster, the cloud-based cluster being a second data intake and query system; in response to the request, obtaining, from the cloud-based cluster, the information identifying the plurality of indexers, wherein the first cluster and the cloud-based cluster each include at least one master node that is knowledgeable about active indexers within its respective cluster, and the information identifies the plurality of indexers based on the at least one master node of the cloud-based cluster identifying the active indexers; distributing the search query to the plurality of indexers of the cloud-based cluster and one or more indexers of the first cluster, said distributing using the obtained information identifying the plurality of indexers and being through the firewall; and receiving, at the first cluster, a response to the distributed search query from at least one of the plurality of indexers of the cloud-based cluster wherein each response from a respective indexer is produced by the respective indexer based on an evaluation, by the respective indexer, of the distributed search query. - View Dependent Claims (31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47)
-
Specification