Periodic database search manager for multiple data sources

• US 10,572,487 B1
• Filed: 01/13/2017
• Issued: 02/25/2020
• Est. Priority Date: 10/30/2015
• Status: Active Grant

• Current Assignee: Palantir Technologies Incorporated
• Original Assignee: Palantir Technologies Incorporated
• Inventors: Sheasby, Glenn, Boreiko, John, Goenka, Aakash, Glazer, Michael, Tank, Spencer, Spiro, Ezra, Ricafort, Juan
• Primary Examiner(s): Stevens, Robert
• Application Number:
  US15/406,195
• Time in Patent Office: 1,138 Days
• Field of Search: 707768
• CPC Class Codes: G06F 16/2358 Change logging, detection, and notification replication G06F16/27
• CPC Class Codes: G06F 16/24539 using cached or materialised query results
• CPC Class Codes: G06F 16/24566 Recursive queries
• CPC Class Codes: G06F 16/9535 Search customisation based on user profiles and personalisation

Abstract

Systems and techniques for searching multiple data sources are described herein. Users may specify searches of multiple data sources to occur on a periodic basis. The searches may be configured to search time or date ranges that have not previously been searched. A user may select the data sources of interest and specify search terms, review and edit previously created searches, and review results of searches. The system automatically performs the specified searches, and notifies the user and/or a team of the user each time new results are found. The system may efficiently search the data sources by storing previous search results and comparing the previous results to current search results to identify new search results.

Claims

1. A method comprising:

receiving user input from a first user indicating a recurrent search and an execution condition for the recurrent search, the user input further comprising a first indication of a first data source and a permissions group;

storing a search object comprising (i) information regarding the recurrent search, (ii) the first indication of the first data source, and (iii) the permissions group, the search object further indicating a recipient;

querying at least the first data source to determine a first plurality of search results based at least in part on the search object, the first data source comprising an electronic log comprising network data;

storing the first plurality of search results;

determining to initiate the recurrent search of at least the first data source based at least in part on the execution condition being satisfied;

querying at least the first data source to determine a second plurality of search results;

comparing the first plurality of search results to the second plurality of search results;

determining a new search result, wherein the new search result is present in the second plurality of search results and not present in the first plurality of search results;

transmitting, to the recipient, an electronic communication associated with the new search result;

determining that a second user belongs to the permissions group; and

presenting, in a user interface, at least some of the second plurality of search results to the second user.

2. The method of claim 1, wherein determining to initiate the recurrent search of at least the first data source based at least in part on the execution condition being satisfied further comprises:

determining a first query time associated with the first plurality of search results; and

determining that a difference between a current time and the first query time satisfies the execution condition based at least in part on a search frequency.

3. The method of claim 1, wherein querying at least the first data source to determine the second plurality of search results further comprises:

accessing the search object, the search object further comprising one or more search terms, wherein the one or more search terms is a parameter for querying at least the first data source.

4. The method of claim 1, wherein querying at least the first data source to determine the second plurality of search results further comprises:

querying a second data source to determine at least some of the second plurality of search results.

5. The method of claim 4, further comprising:

retrieving the search object, the search object further comprising a second data source identifier corresponding to the second data source.

6. The method of claim 1, further comprising:

serializing the first plurality of search results to first data, wherein storing the first plurality of search results comprises storing the first data in a non-transitory computer storage medium;

retrieving the first data from the non-transitory computer storage medium; and

deserializing the first data to a first plurality of objects, wherein comparing the first plurality of search results to the second plurality of search results further comprises comparing the first plurality of objects to a second plurality of objects corresponding to the second plurality of search results.

7. A non-transitory computer storage medium storing computer executable instructions that when executed by a computer hardware processor perform operations comprising:

receiving user input from a first user indicating a recurrent search and an execution condition for the recurrent search, the user input further comprising a first indication of a first data source and a permissions group;

storing a search object comprising (i) information regarding the recurrent search, (ii) the first indication of the first data source, and (iii) the permissions group, the search object further indicating a recipient;

determining to initiate the recurrent search of at least the first data source based at least in part on the execution condition being satisfied;

retrieving a first plurality of search results;

querying at least the first data source to determine a second plurality of search results based at least in part on the search object, the first data source comprising an electronic log comprising network data;

comparing the first plurality of search results to the second plurality of search results;

determining a new search result, wherein the new search result is present in the second plurality of search results and not present in the first plurality of search results;

transmitting, to the recipient, an electronic communication associated with the new search result;

determining that a second user belongs to the permissions group; and

presenting, in a user interface, at least some of the second plurality of search results to the second user.

8. The non-transitory computer storage medium of claim 7, wherein determining to initiate the recurrent search of at least the first data source based at least in part on the execution condition being satisfied further comprises:

determining a first query time associated with the first plurality of search results; and

determining that a difference between a current time and the first query time satisfies the execution condition based at least in part on a search frequency.

9. The non-transitory computer storage medium of claim 7, wherein querying at least the first data source to determine the second plurality of search results further comprises:

retrieving the search object, the search object comprising one or more search terms, wherein the one or more search terms is a parameter for querying at least the first data source.

10. The non-transitory computer storage medium of claim 7, wherein querying at least the first data source to determine the second plurality of search results further comprises:

querying a second data source to determine at least some of the second plurality of search results.

11. The non-transitory computer storage medium of claim 7, wherein the operations further comprise:

serializing the first plurality of search results to first data; and

storing the first data, wherein retrieving the first plurality of search results further comprises retrieving the first data.

12. The non-transitory computer storage medium of claim 11, wherein the operations further comprise:

deserializing the first data to a first plurality of objects, wherein comparing the first plurality of search results to the second plurality of search results further comprises comparing the first plurality of objects to a second plurality of objects corresponding to the second plurality of search results.

13. A system comprising:

a non-transitory computer storage medium configured to store a first plurality of search results; and

one or more computer hardware processors configured to execute code instructions to;

receive user input from a first user indicating a recurrent search and an execution condition for the recurrent search, the user input further comprising a first indication of a first data source and a permissions group;

store a search object comprising (i) information regarding the recurrent search, (ii) the first indication of the first data source, and (iii) the permissions group, the search object further indicating a recipient;

determine to initiate the recurrent search of at least the first data source based at least in part on the execution condition being satisfied;

retrieve a first plurality of search results;

query at least the first data source to determine a second plurality of search results based at least in part on the search object, the first data source comprising an electronic log comprising network data;

compare the first plurality of search results to the second plurality of search results;

determine a new search result, wherein the new search result is present in the second plurality of search results and not present in the first plurality of search results;

transmit, to the recipient, an electronic communication associated with the new search result;

determine that a second user belongs to the permissions group; and

present, in a user interface, at least some of the second plurality of search results to the second user.

14. The system of claim 13, wherein determining to initiate the recurrent search of at least the first data source based at least in part on the execution condition being satisfied further comprises:

determining a first query time associated with the first plurality of search results; and

determining that a difference between a current time and the first query time satisfies the execution condition based at least in part on a search frequency.

15. The system of claim 13, wherein querying at least the first data source to determine the second plurality of search results further comprises:

retrieving the search object, the search object comprising one or more search terms, wherein the one or more search terms is a parameter for querying at least the first data source.

16. The system of claim 13, wherein querying at least the first data source to determine the second plurality of search results further comprises:

querying a second data source to determine at least some of the second plurality of search results.

17. The system of claim 13, wherein the one or more computer hardware processors are further configured to execute code instructions to:

serialize the second plurality of search results to second data; and

store, in the non-transitory computer storage medium, the second data.

18. The system of claim 13, wherein the one or more computer hardware processors are further configured to execute code instructions to:

replace, in the non-transitory computer storage medium, the first plurality of search results with the second plurality of search results.