Parallel distributed network management

• US 9,729,429 B2
• Filed: 10/31/2014
• Issued: 08/08/2017
• Est. Priority Date: 11/10/2008
• Status: Active Grant

• Current Assignee: Tanium Inc.
• Original Assignee: Tanium Inc.
• Inventors: Hindawi, David, Hindawi, Orion, Lippincott, Lisa, Lincroft, Peter
• Primary Examiner(s): STRANGE, AARON N
• Application Number:
  US14/530,601
• Publication Number:
  US 20150163132A1
• Time in Patent Office: 1,012 Days
• CPC Class Codes: G06F 16/27 Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
• CPC Class Codes: H04L 41/04 Network management architectures or arrangements
• CPC Class Codes: H04L 41/0853 by actively collecting configuration information or by backing up configuration information
• CPC Class Codes: H04L 41/12 Discovery or management of network topologies
• CPC Class Codes: H04L 41/26 using dedicated tools for LAN [Local Area Network] management
• CPC Class Codes: H04L 43/06 Generation of reports
• CPC Class Codes: H04L 43/065 related to network devices
• CPC Class Codes: H04L 43/10 Active monitoring, e.g. heartbeat, ping or trace-route
• CPC Class Codes: H04L 45/16 Multipoint routing
• CPC Class Codes: H04L 45/44 Distributed routing

Abstract

A method of parallel processing by networked devices, in which distributed data in a network is accessed in parallel to provide results. Commands and queries by a user station define a set of such distributed data. The networked devices access the distributed data in portions, individually process each portion, conduct peer-to-peer communication to aggregate results from processing each portion, and individually determine whether and when to report results. Making effective use of network assets (such as processing and storage at each node), making effective use of concurrency of network assets, limiting bandwidth use between network assets and user stations, and limiting use of assets (such as processing and storage) at user stations.

Claims

1. A method, including:

in a computer network having a plurality of nodes, wherein each said node comprises a respective computing device and has a respective defined location in a linearly ordered sequence of the plurality of nodes, and wherein the plurality of nodes further self-organize into a plurality of communication orbits, each communication orbit including a respective subset of the plurality of nodes and organized in an order consistent with the linearly ordered sequence;

at a first node of the plurality of nodes;

receiving a message targeting a first group of nodes among the plurality of nodes;

in response to receiving the message, selecting one or more of the plurality of communication orbits for propagating the message to the first group of nodes; and

sending the message to a respective node in each of the selected one or more communication orbits, wherein the respective node is selected in accordance with the linearly ordered sequence.

2. The method of claim 1, wherein selecting the one or more communication orbits for propagating the message to the first group of nodes further comprises:

determining a respective query type associated with the message; and

selecting the one or more communication orbits in accordance with the determined respective query type of the message.

3. The method of claim 1, wherein selecting the one or more communication orbits further comprises selecting at least a first communication orbit and a second communication orbit of the plurality of communication orbits to initiate parallel processing of information.

4. The method of claim 1, wherein selecting the one or more communication orbits further comprises selecting one of the plurality of communication orbits to skip at least some of the plurality of nodes in the linearly ordered sequence when propagating the message to the first group of nodes.

5. The method of claim 1, wherein the message includes an instruction for collecting information from the first group of nodes or issuing a command to perform an action at the first group of nodes.

6. The method of claim 1, wherein the linearly ordered sequence is in accordance with a sequential order of their respective hardware addresses.

7. The method of claim 1, wherein the plurality of communication orbits include a base orbit that includes substantially all of the plurality of nodes, and one or more express orbits that each include a substantially smaller number of nodes than the base orbit.

8. The method of claim 1, wherein the plurality of communication orbits collectively span the plurality of nodes.

9. The method of claim 1, wherein the selected one or more of the plurality of communication orbits comprise two or more communication orbits in the plurality of communication orbits;

the first node of the plurality of nodes is a node in the two or more communication orbits; and

the method further comprise;

maintaining a record of nearest neighbors of the first node in each of the two or more communication orbits; and

selecting the respective node from each of the two or more communication orbits in accordance with the record of nearest neighbors.

10. A system, comprising:

one or more processors; and

memory having instructions stored thereon, the instructions, when executed by the one or more processors, cause the one or more processors to perform operations comprising;

in a computer network having a plurality of nodes, wherein each said node comprises a respective computing device and has a respective defined location in a linearly ordered sequence of the plurality of nodes, and wherein the plurality of nodes further self-organize into a plurality of communication orbits, each communication orbit including a respective subset of the plurality of nodes and organized in an order consistent with the linearly ordered sequence;

at a first node of the plurality of nodes, the first node served by said system;

receiving a message targeting a first group of nodes among the plurality of nodes;

in response to receiving the message, selecting one or more of the plurality of communication orbits for propagating the message to the first group of nodes; and

sending the message to a respective node in each of the selected one or more communication orbits, wherein the respective node is selected in accordance with the linearly ordered sequence.

11. The system of claim 10, wherein selecting the one or more communication orbits for propagating the message to the first group of nodes further comprises:

determining a respective query type associated with the message; and

selecting the one or more communication orbits in accordance with the determined respective query type of the message.

12. The system of claim 10, wherein selecting the one or more communication orbits further comprises selecting at least a first communication orbit and a second communication orbit of the plurality of communication orbits to initiate parallel processing of information.

13. The system of claim 10, wherein selecting the one or more communication orbits further comprises selecting one of the plurality of communication orbits to skip at least some of the plurality of nodes in the linearly ordered sequence when propagating the message to the first group of nodes.

14. The system of claim 10, wherein the message includes an instruction for collecting information from the first group of nodes or issuing a command to perform an action at the first group of nodes.

15. The system of claim 10, wherein the linearly ordered sequence is in accordance with a sequential order of their respective hardware addresses.

16. The system of claim 10, wherein the plurality of communication orbits include a base orbit that includes substantially all of the plurality of nodes, and one or more express orbits that each include a substantially smaller number of nodes than the base orbit.

17. The system of claim 10, wherein the plurality of communication orbits collectively span the plurality of nodes.

18. The system of claim 10, wherein the selected one or more of the plurality of communication orbits comprise two or more communication orbits in the plurality of communication orbits;

the first node of the plurality of nodes is a node in the two or more communication orbits; and

the operations further comprise;

maintaining a record of nearest neighbors of the first node in each of the two or more communication orbits; and

selecting the respective node from each of the two or more communication orbits in accordance with the record of nearest neighbors.

19. A non-transitory computer-readable storage medium having instructions stored thereon, the instructions, when executed by one or more processors, cause the processors to perform operations comprising:

in a computer network having a plurality of nodes, wherein each said node comprises a respective computing device and has a respective defined location in a linearly ordered sequence of the plurality of nodes, and wherein the plurality of nodes further self-organize into a plurality of communication orbits, each communication orbit including a respective subset of the plurality of nodes and organized in an order consistent with the linearly ordered sequence;

at a first node of the nodes;

receiving a message targeting a first group of nodes among the plurality of nodes;

in response to receiving the message, selecting one or more of the plurality of communication orbits for propagating the message to the first group of nodes; and

sending the message to a respective node in each of the selected one or more communication orbits, wherein the respective node is selected in accordance with the linearly ordered sequence.

20. The computer-readable storage medium of claim 19, wherein selecting the one or more communication orbits for propagating the message to the first group of nodes further comprises:

determining a respective query type associated with the message; and

selecting the one or more communication orbits in accordance with the determined respective query type of the message.

21. The computer-readable storage medium of claim 19, wherein selecting the one or more communication orbits further comprises selecting at least a first communication orbit and a second communication orbit of the plurality of communication orbits to initiate parallel processing of information.

22. The computer-readable storage medium of claim 19, wherein selecting the one or more communication orbits further comprises selecting one of the plurality of communication orbits to skip at least some of the plurality of nodes in the linearly ordered sequence when propagating the message to the first group of nodes.

23. The computer-readable storage medium of claim 19, wherein the message includes an instruction for collecting information from the first group of nodes or issuing a command to perform an action at the first group of nodes.

24. The computer-readable storage medium of claim 19, wherein the linearly ordered sequence is in accordance with a sequential order of their respective hardware addresses.

25. The computer-readable storage medium of claim 19, wherein the plurality of communication orbits include a base orbit that includes substantially all of the plurality of nodes, and one or more express orbits that each include a substantially smaller number of nodes than the base orbit.

26. The computer-readable storage medium of claim 19, wherein the plurality of communication orbits collectively span the plurality of nodes.

27. The computer-readable storage medium of claim 19, wherein the selected one or more of the plurality of communication orbits comprise two or more communication orbits in the plurality of communication orbits;

the first node of the plurality of nodes is a node in the two or more communication orbits; and

the operations further comprise;

maintaining a record of nearest neighbors of the first node in each of the two or more communication orbits; and

selecting the respective node from each of the two or more communication orbits in accordance with the record of nearest neighbors.