Systems and methods for peer-to-peer build sharing

• US 10,223,099 B2
• Filed: 04/11/2017
• Issued: 03/05/2019
• Est. Priority Date: 12/21/2016
• Status: Active Grant

• Current Assignee: Palantir Technologies Incorporated
• Original Assignee: Palantir Technologies Incorporated
• Inventors: Avant, Matthew
• Primary Examiner(s): Morshed, Hossain M
• Application Number:
  US15/484,980
• Publication Number:
  US 20180173514A1
• Time in Patent Office: 693 Days
• Field of Search: None
• CPC Class Codes: G06F 16/2246 Trees, e.g. B+trees
• CPC Class Codes: G06F 16/284 Relational databases
• CPC Class Codes: G06F 16/951 Indexing; Web crawling techniques
• CPC Class Codes: G06F 8/36 Software reuse
• CPC Class Codes: G06F 8/65 Updates security arrangements therefor G06F21/57
• CPC Class Codes: G06F 8/71 Version control security arrangements therefor G06F21/57; Configuration management
• CPC Class Codes: G06F 9/5061 Partitioning or combining of resources
• CPC Class Codes: H04L 67/10 in which an application is distributed across nodes in the network software deployment G06F8/60; multiprogramming arrangements G06F9/46

Abstract

Systems and methods are provided for obtaining a build object over a communication network, the build object comprising at least a portion of a first build. The build object is stored locally. A build modification object is obtained over the communication network, the build modification object comprising a change between the first build and a second build. The build object updated using the build modification object obtained over the communication network. At least a portion of the updated build object is provided to a first peer build system over the communication network.

Claims

1. A software build system for generating new software builds based on build objects obtained from a peer build system comprising:

one or more processors; and

memory storing instructions that, when executed by the one or more processors, cause the system to perform;

obtaining a current build object over a communication network, the current build object comprising a portion of a current build, and the current build object comprising any of one or more hashes, data attributes, methods, parameters, tree structures and multiple content items;

storing the current build object;

estimating a change between the current build object and a predicted build object based on a current state of one or more content items associated with the current build;

comparing the estimated change to a threshold condition;

generating, in response to the comparison, one or more new build objects;

determining one or more changes between two or more build objects;

generating one or more build modification objects based on the one or more determined changes between the two or more build objects, wherein at least one of the one or more build modification objects comprise a tree structure storing at least a portion of a build that has changed relative to one or more other builds;

providing the one or more build modification objects to at least one peer build system without providing an entire build, thereby reducing time for the at least one peer system to update one or more local build objects, the one or more local build objects capable of being used for processing build action requests.

2. The system of claim 1, wherein a build object comprises a hash tree structure.

3. The system of claim 2, wherein the build object comprises a plurality of content items, each of the content items comprising any of a file or a binary, each of the content items stored as an element of the hash tree structure.

4. The system of claim 1, wherein the obtaining the current build object over the communication network comprises

obtaining the current build object from a second peer build system over the communication network, and

the obtaining the one or more build modification objects over the communication network comprises obtaining the one or more build modification objects over the communication from an originating build system storing a complete instance of the first build and a complete instance of a second build.

5. The system of claim 1, wherein a build object comprises an immutable build object, and the updating the one or more local build object comprises generating a second build object using the one or more build modification objects.

6. The system of claim 1, wherein the instructions further cause the system to perform:

predicting one or more respective build objects stored by a plurality of respective peer build systems;

receiving a build action request;

selecting a third peer build system from the plurality of respective peer build systems based on the build action request and the prediction;

obtaining a second build object from the third peer build system based on the build action request;

determining whether the second build object satisfies the build action request;

if the second build object satisfies the build action request, processing the build action request using the second build object; and

if the second build object does not satisfy the build action request, iteratively obtaining one or more different build objects from one or more different peer build systems from the plurality of respective build systems until a particular build object of the one or more different build objects satisfies the build action request; and

processing the build action request using the particular build object of the one or more different build objects.

7. The system of claim 6, wherein the selecting a third peer build system from the plurality of respective peer build systems based on the build action request and the prediction comprises selecting a particular peer build system predicted to store a particular build object indicated by the build action request.

8. The system of claim 7, wherein the particular peer build system is selected based on a latency relative to one or more other peers of the plurality of peers predicted to store the particular build object indicated by the build action request.

9. The system of claim 1, wherein the instructions further cause the system to perform:

obtaining one or more additional build objects from one or more additional peer build systems; and

creating a complete instance of a second build from the updated one or more build objects and the one or more additional build objects, the complete instance of the second build comprising a hash tree structure.

10. The system of claim 1, wherein storing a build object comprises storing the build object in a Merkle tree structure, the build object comprising a portion of a software build.

11. A method being implemented by a computing system including one or more physical processors and storage media storing machine-readable instructions, the method comprising:

obtaining a current build object over a communication network, the build object comprising a portion of a current build, and the current build object comprising any of one or more hashes, data attributes, methods, parameters, tree structures and multiple content items;

storing the current build object;

estimating a change between the current build object and a predicted build object based on a current state of one or more content items associated with the current build;

comparing the estimated change to a threshold condition;

generating, in response to the comparison, one or more new build objects;

determining one or more changes between two or more build objects;

generating one or more build modification objects based on the one or more determined changes between the two or more build objects, wherein at least one of the one or more build modification objects comprise a tree structure storing at least a portion of a build that has changed relative to one or more other builds;

providing the one or more build modification objects to at least one peer build system without providing an entire build, thereby reducing time for the at least one peer system to update one or more local build objects, the one or more local build objects capable of being used for processing build action requests.

12. The method of claim 11, wherein a build object comprises a hash tree structure.

13. The method of claim 12, wherein the build object comprises a plurality of content items, each of the content items comprising any of a file or a binary, each of the content items stored as an element of the hash tree structure.

14. The method of claim 11, wherein the obtaining the current build object over the communication network comprises obtaining the current build object from a second peer build system over the communication network, and the obtaining the one or more build modification objects over the communication network comprises obtaining the one or more build modification objects over the communication from an originating build system storing a complete instance of the first build and a complete instance of a second build.

15. The method of claim 11, wherein a build object comprises an immutable build object, and the updating the one or more build objects comprises generating a second build object using the one or more build modification objects.

16. The method of claim 11, further comprising:

predicting one or more respective build objects stored by a plurality of respective peer build systems;

receiving a build action request;

selecting a third peer build system from the plurality of respective peer build systems based on the build action request and the prediction;

obtaining a second build object from the third peer build system based on the build action request;

determining whether the second build object satisfies the build action request;

if the second build object satisfies the build action request, processing the build action request using the second build object; and

if the second build object does not satisfy the build action request, iteratively obtaining one or more different build objects from one or more different peer build systems from the plurality of respective build systems until a particular build object of the one or more different build objects satisfies the build action request; and

processing the build action request using the particular build object of the one or more different build objects.

17. The method of claim 16, wherein the selecting a third peer build system from the plurality of respective peer build systems based on the build action request and the prediction comprises selecting a particular peer build system predicted to store a particular build object indicated by the build action request.

18. The method of claim 17, wherein the particular peer build system is selected based on a latency relative to one or more other peers of the plurality of peers predicted to store the particular build object indicated by the build action request.

19. The method of claim 11, further comprising:

obtaining one or more additional build objects from one or more additional peer build systems; and

creating a complete instance of second build from the updated one or more local build objects and the one or more additional build objects, the complete instance of the second build comprising a hash tree structure.

20. A non-transitory computer readable medium comprising instructions that, when executed, cause one or more processors to perform:

obtaining a current build object over a communication network, the current build object comprising a portion of a current build, and the build current object comprising any of one or more hashes, data attributes, methods, parameters, tree structures and multiple content items;

storing the current build object;

estimating a change between the current build object and a predicted build object based on a current state of one or more content items associated with the current build;

comparing the estimated change to a threshold condition;

generating, in response to the comparison, one or more new build objects;

determining one or more changes between two or more build objects;

generating one or more build modification objects based on the one or more determined changes between the two or more build objects, wherein at least one of the one or more build modification objects comprises a tree structure storing at least a portion of a build that has changed relative to one or more other builds;

providing the one or more build modification objects to at least one peer build system without providing an entire build, thereby reducing time for the at least one peer system to update one or more local build objects, the one or more local build objects capable of being used for processing build action requests.