Hierarchical diff files

• US 10,997,145 B2
• Filed: 07/12/2018
• Issued: 05/04/2021
• Est. Priority Date: 06/19/2009
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Dole, Bryn R., Lindahl, Gregory B., Markson, Michael, Peters, Keith, Saliba, Robert Michael, Skrenta, Rich, Truel, Robert N.
• Primary Examiner(s): Yen, Syling
• Application Number:
  US16/034,193
• Publication Number:
  US 20190012338A1
• Time in Patent Office: 1,027 Days
• Field of Search: 707812
• CPC Class Codes: G06F 16/2228 Indexing structures
• CPC Class Codes: G06F 16/2282 Tablespace storage structures; Management thereof
• CPC Class Codes: G06F 16/24554 Unary operations; Data partitioning operations
• CPC Class Codes: G06F 16/2462 Approximate or statistical queries
• CPC Class Codes: G06F 16/273 Asynchronous replication or reconciliation
• CPC Class Codes: G06F 16/285 Clustering or classification
• CPC Class Codes: G06F 16/951 Indexing; Web crawling techniques

Abstract

A method, according to one embodiment, includes receiving, at a first node of a distributed database system from one or more caller nodes of the distributed database system, a predetermined number of service tasks to add to a value in a row stored in the distributed database system at a write daemon. Additionally, a time of receipt of the predetermined number of service tasks is recorded. Further, the predetermined number of service tasks are combined to a single row in the distributed database system at the write daemon. Further still, a number of operations to attend to the predetermined number of service tasks is reduced by sending the predetermined number of service tasks to at least two bucket daemons on at least two other nodes of the distributed database system.

Claims

1. A computer-implemented method, comprising:

at a first node of a distributed database system, receiving, at a write daemon from one or more caller nodes of the distributed database system, a predetermined number of service tasks, including a first service task to add to a value in a row stored in the distributed database system, and a second service task to set the value in the row to a predetermined value;

recording a time of receipt of the first service task and the second service task;

combining the predetermined number of service tasks to a single row in the distributed database system at the write daemon, including setting the value in the row to the predetermined value without first adding to the value in the row in response to determining that the time of receipt of the first service task to add to the value in a row precedes the time of receipt of the second service task to set the value in the row to the predetermined value; and

sending the predetermined number of service tasks to at least two bucket daemons on at least two other nodes of the distributed database system.

2. The computer-implemented method of claim 1, wherein the service tasks further include add requests.

3. The computer-implemented method of claim 1, wherein receiving the predetermined number of service tasks include receiving a search query from a search engine user.

4. The computer-implemented method of claim 1, wherein the distributed database system includes a search engine.

5. The computer-implemented method of claim 4, wherein the search engine includes a decentralized swarm system that utilizes one or more swarm algorithms.

6. The computer-implemented method of claim 1, wherein the predetermined number of service tasks include a task of maintaining one or more log files by one or more combinators, including collecting data on number of webpage hits and generating web page summary information.

7. The computer-implemented method of claim 6, wherein:

the one or more combinators include one or more append combinators that append lines of text to an existing set of lines of text, and

the one or more combinators include one or more comb add combinators, that are configured to count a number of items and combine the count to existing data;

wherein the predetermined number of service tasks include detecting email spam using one or more spam combinators that compute a set of signatures based on email headers and content and check whether the same signatures are present in a plurality of emails to a plurality of people.

8. The computer-implemented method of claim 1, wherein the distributed database system includes a search engine, and further comprising synchronizing a clock of the first node with clocks of the at least two other nodes of the distributed database system using a network time protocol (NTP), where the synchronized clock of the first node is used to record the time of receipt of the predetermined number of service tasks.

9. The computer-implemented method of claim 1, wherein the at least two bucket daemons on the at least two other nodes of the distributed database system combine the predetermined number of service tasks and execute a reduced number of operations in accordance with an order associated with the time of receipt.

10. The computer-implemented method of claim 1, further comprising storing programs to be executed by the distributed database system;

wherein the at least two bucket daemons on the at least two other nodes of the distributed database system combine the predetermined number of service tasks and execute a reduced number of operations in accordance with an order associated with the time of receipt.

11. A computer program product for managing a distributed database system, the computer program product comprising a computer readable storage medium having program instructions embodied therewith, wherein the computer readable storage medium is not a transitory signal per se, the program instructions executable by a processor to cause the processor to perform a method comprising:

at a first node of a distributed database system, receiving, by the processor at a write daemon from one or more caller nodes of the distributed database system, a predetermined number of service tasks, including a first service task to add to a value in a row stored in the distributed database system, and a second service task to set the value in the row to a predetermined value;

recording, by the processor, a time of receipt of the first service task and the second service task;

combining, by the processor, the predetermined number of service tasks to a single row in the distributed database system at the write daemon, including setting, by the processor, the value in the row to the predetermined value without first adding to the value in the row in response to determining that the time of receipt of the first service task to add to the value in a row precedes the time of receipt of the second service task to set the value in the row to the predetermined value; and

sending the predetermined number of service tasks to at least two bucket daemons on at least two other nodes of the distributed database system.

12. The computer program product of claim 11, wherein the service tasks further include add requests.

13. The computer program product of claim 11, wherein receiving the predetermined number of service tasks include receiving a search query from a search engine user.

14. The computer program product of claim 11, wherein the distributed database system includes a search engine.

15. The computer program product of claim 14, wherein the search engine includes a decentralized swarm system that utilizes one or more swarm algorithms.

16. The computer program product of claim 11, wherein the predetermined number of service tasks include a task of maintaining one or more log files by one or more combinators, including collecting data on number of webpage hits and generating web page summary information.

17. The computer program product of claim 16, wherein the one or more combinators include one or more append combinators that append lines of text to an existing set of lines of text.

18. The computer program product of claim 16, wherein the one or more combinators include one or more comb add combinators, that are configured to count a number of items and combine the count to existing data.

19. The computer program product of claim 16, wherein the predetermined number of service tasks include detecting email spam using one or more spam combinators that compute a set of signatures based on email headers and content and check whether the same signatures are present in a plurality of emails to a plurality of people.

20. A computer-implemented method, comprising:

at a first node of a distributed database system, receiving from one or more caller nodes of the distributed database system a predetermined number of service tasks to add to a value in a row stored in the distributed database system at a write daemon, where;

the predetermined number of service tasks include a task of maintaining one or more log files by one or more combinators, including collecting data on a number of webpage hits and generating web page summary information, and detecting email spam using one or more spam combinators that compute a set of signatures based on email headers and content and check whether the same signatures are present in a plurality of emails to a plurality of people,

the one or more combinators include one or more append combinators that append lines of text to an existing set of lines of text, and one or more comb add combinators configured to count a number of items and combine the count to existing data;

recording a time of receipt of the predetermined number of service tasks;

combining the predetermined number of service tasks to a single row in the distributed database system at the write daemon; and

reducing a number of operations to attend to the predetermined number of service tasks by sending the predetermined number of service tasks to at least two bucket daemons on at least two other nodes of the distributed database system.