Data caching and distribution in a local network

• US 9,769,275 B2
• Filed: 11/26/2014
• Issued: 09/19/2017
• Est. Priority Date: 03/24/2014
• Status: Active Grant

• Current Assignee: Tanium Inc.
• Original Assignee: Tanium Inc.
• Inventors: Hindawi, David, Hindawi, Orion, Lippincott, Lisa, Lincroft, Peter
• Primary Examiner(s): Naji, Younes
• Application Number:
  US14/554,739
• Publication Number:
  US 20150271285A1
• Time in Patent Office: 1,028 Days
• Field of Search: None
• CPC Class Codes: H04L 67/1089 Hierarchical topologies
• CPC Class Codes: H04L 67/568 Storing data temporarily at an intermediate stage, e.g. caching

Abstract

A data caching and distribution method, performed by a plurality of in a linear communication orbit, includes generating a data request by a first machine to request specific data, and passing the data request along a data request path that tracks the linear communication orbit until the request is received at a second machine, in the linear communication orbit, that returns the specific data in response to the data request. The method includes, at a third machine between the second machine and the first machine in the linear communication orbit, conditionally storing the specific data in a local cache of the third machine according to a data caching method.

Claims

1. A method of data caching and distribution, comprising:

at a plurality of machines that are included in a linear communication orbit, wherein the linear communication orbit includes two end nodes, comprising a head node and a tail node, and the linear communication orbit is coupled to a server via one of the two end nodes;

generating a data request by a first machine to request specific data;

passing the data request along a data request path that tracks the linear communication orbit until the data request is received at a second machine, in the linear communication orbit, that returns the specific data in response to the data request, including passing the data request to a third machine between the second machine and the first machine in the linear communication orbit;

returning the specific data from the second machine to the first machine via the third machine; and

conditionally storing the specific data in a local cache of the third machine according to a data caching method, wherein;

the plurality of machines includes the first machine, the second machine, and the third machine; and

the plurality of machines are ordered in an ordered sequence in the linear communication orbit based on a predefined set of rules, with the third machine between the first and second machines in the ordered sequence, and with the first machine between the third machine and one of the two end nodes in the ordered sequence.

2. The method of claim 1, wherein the second machine comprises a machine, in the linear communication orbit, that has a cached copy of the specific data in a local cache of the machine or the head node if no machine in the data request path has a cached copy of the specific data in a local cache of any machine in the data request path.

3. The method of claim 1, wherein the second machine comprises a machine closest, along the data request path, to the first machine that has a cached copy of the specific data in a local cache of said closest machine that has a cached copy of the specific data.

4. The method of claim 1, wherein passing the data request includes passing the data request along the data request path that tracks the linear communication orbit until (A) the data request is received at a machine, in the linear communication orbit, that has a cached copy of the specific data in a local cache of the machine or (B) the data request is received at the head node, which obtains and returns the specific data in response to the data request.

5. The method of claim 1, wherein the plurality of machines that are included in the linear communication orbit are interconnected by a single local area network.

6. The method of claim 1, wherein:

the data caching method comprises a statistically random method of determining whether to cache respective data received from another machine.

7. The method of claim 1, wherein the specific data comprises one of a plurality of shards, in an object, to be cached in the linear communication orbit, and wherein the data caching method is based on at least one of a size of the specific data, a size of the object, a size of the local cache of the third machine, a total size of available caches in the linear communication orbit, and a total size of a data set that includes the object.

8. The method of claim 1, wherein the data request comprises a first data request and the specific data comprises first specific data, the method including:

generating, at the first machine, a second data request to request second specific data;

passing the second data request along the data request path until the second data request is received at a fourth machine, in the linear communication orbit, that returns the second specific data in response to the second data request, wherein the fourth machine is distinct from the second machine; and

combining, at the first machine, a plurality of specific data, including the first specific data and the second specific data, to generate an object for use at the first machine.

9. The method of claim 8, wherein the plurality of specific data comprise shards of said object specified in a manifest.

10. The method of claim 9, wherein generating the first data request by the first machine includes delaying generation of the first data request, with respect to time at which the manifest is received by the first machine, in accordance with a predefined request spreading method.

11. The method of claim 1, wherein the data request comprises a first data request, the method further comprising:

generating at a fourth machine a second data request to request the specific data after generating the first data request by the first machine; and

in accordance with a determination that predefined criteria in a request consolidation method are satisfied, forgoing passing the second data request along at least a part of a respective data request path that is associated with the second data request.

12. The method of claim 11, wherein the specific data are conveyed to the fourth machine by returning the specific data to the first machine, which reconveys the specific data to the fourth machine.

13. The method of claim 1, wherein the specific data comprise one of a plurality of shards of an object, and the object is one of a group consisting of a software application, software application update, database, and database update.

14. The method of claim 1, further comprising:

passing the specific data from the second machine to the first machine through a plurality of intervening machines in the linear communication orbit;

at each machine of the plurality of intervening machines, located between the second machine and the first machine, caching a copy of the specific data in a local cache of the machine if predefined local caching criteria are met.

15. The method of claim 1, further comprising:

passing the specific data from the second machine to the first machine through a plurality of intervening machines in the linear communication orbit and continuing to pass the specific data to additional machines in the linear communication orbit until the specific data are received by one of the end nodes; and

at each machine of a plurality of machines, selected from the plurality of intervening machines and the additional machines, caching a copy of the specific data in a local cache of the machine in accordance with a determination that predefined local caching criteria are met.

16. The method of claim 15, wherein the second machine is a downstream machine between the first machine and the tail node, the data request being passed downstream from the first machine to the second machine, the specific data being returned upstream from the second machine to the first machine.

17. The method of claim 15, wherein the second machine is an upstream machine between the first machine and the head node, the data request being passed from the first machine downstream to the tail node and upstream to the second machine, the specific data being returned downstream from the second machine to the first machine.

18. The method of claim 15, wherein the second machine is the tail node or the head node, the specific data being returned from the second machine to the first machine and conditionally stored in the plurality of machines, including the third machine, that are coupled within the linear communication orbit and do not have the specific data.

19. The method of claim 1, wherein the linear communication orbit is organized from the plurality of machines based on a set of rules at a first time and subject to changes in configuration and membership when one or more machines join or exit the linear communication orbit.

20. A system, comprising:

one or more processors included in a plurality of machines;

memory having instructions stored thereon, which when executed by the one or more processors cause the one or more processors to perform operations comprising;

at the plurality of machines, which are included in a linear communication orbit, wherein the linear communication orbit terminates at two end nodes, comprising a head node and a tail node, and the linear communication orbit is coupled to a server via one of the two end nodes;

generating a data request by a first machine to request specific data;

passing the data request along a data request path that tracks the linear communication orbit until the data request is received at a second machine, in the linear communication orbit, that returns the specific data in response to the data request, including passing the data request to a third machine between the second machine and the first machine in the linear communication orbit;

returning the specific data from the second machine to the first machine via the third machine; and

conditionally storing the specific data in a local cache of the third machine according to a data caching method, wherein;

the plurality of machines includes the first machine, the second machine, and the third machine; and

the plurality of machines are ordered in an ordered sequence in the linear communication orbit based on a predefined set of rules, with the third machine between the first and second machines in the ordered sequence, and with the first machine between the third machine and one of the two end nodes in the ordered sequence.

21. The system of claim 20, wherein the second machine comprises a machine, in the linear communication orbit, that has a cached copy of the specific data in a local cache of the machine or the head node if no machine in the data request path has a cached copy of the specific data in a local cache of any machine in the data request path.

22. The system of claim 20, wherein the second machine comprises a machine closest, along the data request path, to the first machine that has a cached copy of the specific data in a local cache of the machine closest to the first machine.

23. The system of claim 20, wherein passing the data request includes passing the data request along the data request path that tracks the linear communication orbit until (A) the data request is received at a machine, in the linear communication orbit, that has a cached copy of the specific data in a local cache of the machine or (B) the data request is received at the head node, which obtains and returns the specific data in response to the data request.

24. The system of claim 20, wherein the plurality of machines that are included in the linear communication orbit are interconnected by a single local area network.