Combinators

• US 10,095,725 B2
• Filed: 03/07/2016
• Issued: 10/09/2018
• Est. Priority Date: 06/19/2009
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Skrenta, Rich, Dole, Bryn Robert, Markson, Michael, Peters, Keith, Saliba, Robert Michael, Truel, Robert N., Lindahl, Gregory B.
• Primary Examiner(s): OBERLY, VAN HONG
• Application Number:
  US15/063,376
• Publication Number:
  US 20160188722A1
• Time in Patent Office: 946 Days
• Field of Search: 707770, 707652
• CPC Class Codes: G06F 16/2228 Indexing structures
• CPC Class Codes: G06F 16/2282 Tablespace storage structures; Management thereof
• CPC Class Codes: G06F 16/24554 Unary operations; Data partitioning operations
• CPC Class Codes: G06F 16/2462 Approximate or statistical queries
• CPC Class Codes: G06F 16/273 Asynchronous replication or reconciliation
• CPC Class Codes: G06F 16/285 Clustering or classification
• CPC Class Codes: G06F 16/951 Indexing; Web crawling techniques

Abstract

A method, according to one embodiment, includes storing programs to be executed by a distributed database system having three or more nodes. At a first node of the distributed database system, an N number of service tasks to add to a value in a row stored in the distributed database system at a write daemon are received from one or more caller nodes. The N number of service tasks are combined to a single row in the distributed database system at the write daemon. A number of service tasks to attend to the N number of requests is reduced by sending the N number of requests to at least two other bucket daemons on at least two other nodes that combine the N number of service tasks and executing the reduced number of service tasks in accordance with an order associated with a time of receipt.

Claims

1. A computer-implemented method, comprising:

storing programs to be executed by a distributed database system having three or more nodes each with a processor and memory;

at a first node of the distributed database system, receiving from one or more caller nodes of the distributed database system an N number of service tasks to add to a value in a row stored in the distributed database system at a write daemon;

recording a time of receipt of the N number of service tasks;

combining the N number of service tasks to a single row in the distributed database system at the write daemon; and

reducing a number of service tasks to attend to the N number of requests by sending the N number of requests to at least two other bucket daemons on at least two other nodes of the distributed database system that combine the N number of service tasks and executing the reduced number of service tasks in accordance with an order associated with the recorded time of receipt.

2. The computer-implemented method of claim 1, wherein the service tasks include add requests.

3. The computer-implemented method of claim 1, wherein receiving the N number of service tasks include receiving a search query from a search engine user.

4. The computer-implemented method of claim 1, wherein the distributed database system includes a search engine.

5. The computer-implemented method of claim 4, wherein the search engine includes a decentralized swarm system that utilizes one or more swarm algorithms.

6. The computer-implemented method of claim 1, wherein the N number of service tasks include a task of maintaining one or more log files by one or more combinators, including collecting data on number of webpage hits and generating web page summary information.

7. The computer-implemented method of claim 6, wherein the one or more combinators include one or more append combinators that append lines of text to an existing set of lines of text.

8. The computer-implemented method of claim 6, wherein the one or more combinators include one or more comb add combinators, that are configured to count a number of items and combine the count to existing data.

9. The computer-implemented method of claim 6, wherein the N number of service tasks include detecting email spam using one or more spam combinators that compute a set of signatures based on email headers and content and check whether the same signatures are present in a plurality of emails to a plurality of people.

10. The computer-implemented method of claim 1, further comprising maintaining one or more log files by one or more combinators, including collecting data on a number of webpage hits and generating summary information of webpages.

11. A computer program product for managing a distributed database system, the computer program product comprising a computer readable storage medium having program instructions embodied therewith, wherein the computer readable storage medium is not a transitory signal per se, the program instructions executable by a processor to cause the processor to perform a method comprising:

storing, by the processor, programs to be executed by a distributed database system having three or more nodes each with processors and memory;

at a first node of the distributed database system, receiving, by the processor, from one or more caller nodes of the distributed database system an N number of service tasks to add to a value in a row stored in the distributed database system at a write daemon;

recording, by the processor, a time of receipt of the N number of service tasks;

combining, by the processor, the N number of service tasks to a single row in the distributed database system at the write daemon; and

reducing, by the processor, a number of service tasks to attend to the N number of requests by sending the N number of requests to at least two other bucket daemons on at least two other nodes of the distributed database system that combine the N number of service tasks and executing the reduced number of service tasks in accordance with an order associated with the recorded time of receipt.

12. The computer program product of claim 11, wherein the service tasks include add requests.

13. The computer program product of claim 11, wherein receiving the N number of service tasks include receiving a search query from a search engine user.

14. The computer program product of claim 11, wherein the distributed database system includes a search engine.

15. The computer program product of claim 14, wherein the search engine includes a decentralized swarm system that utilizes one or more swarm algorithms.

16. The computer program product of claim 11, wherein the N number of service tasks include a task of maintaining one or more log files by one or more combinators, including collecting data on number of webpage hits and generating web page summary information.

17. The computer program product of claim 16, wherein the one or more combinators include one or more append combinators that append lines of text to an existing set of lines of text.

18. The computer program product of claim 16, wherein the one or more combinators include one or more comb add combinators, that are configured to count a number of items and combine the count to existing data.

19. The computer program product of claim 16, wherein the N number of service tasks include detecting email spam using one or more spam combinators that compute a set of signatures based on email headers and content and check whether the same signatures are present in a plurality of emails to a plurality of people.

20. A system, comprising:

a processor and logic integrated with and/or executable by the processor, the logic being configured to;

store programs to be executed by a distributed database system having three or more nodes each with processors and memory;

at a first node of the distributed database system, receive from one or more caller nodes of the distributed database system an N number of service tasks to add to a value in a row stored in the distributed database system at a write daemon;

record a time of receipt of the N number of service tasks;

combine the N number of service tasks to a single row in the distributed database system at the write daemon; and

reduce a number of service tasks to attend to the N number of requests by sending the N number of requests to at least two other bucket daemons on at least two other nodes of the distributed database system that combine the N number of service tasks and executing the reduced number of service tasks in accordance with an order associated with the recorded time of receipt.