Method and apparatus for reducing network resource transmission size using delta compression

• US 10,044,826 B2
• Filed: 08/10/2016
• Issued: 08/07/2018
• Est. Priority Date: 08/10/2016
• Status: Active Grant

• Current Assignee: CloudFlare Incorporated
• Original Assignee: CloudFlare Incorporated
• Inventors: Knecht, Dane Orion, Graham-Cumming, John, Prince, Matthew Browning
• Primary Examiner(s): Fan, Hua
• Application Number:
  US15/233,157
• Publication Number:
  US 20180048730A1
• Time in Patent Office: 727 Days
• CPC Class Codes: H04L 67/01 Protocols
• CPC Class Codes: H04L 67/02 based on web technology, e.g. hypertext transfer protocol [HTTP]
• CPC Class Codes: H04L 67/10 in which an application is distributed across nodes in the network software deployment G06F8/60; multiprogramming arrangements G06F9/46
• CPC Class Codes: H04L 67/5651 Reducing the amount or size of exchanged application data
• CPC Class Codes: H04L 67/568 Storing data temporarily at an intermediate stage, e.g. caching
• CPC Class Codes: H04L 69/04 Protocols for data compression, e.g. ROHC

Abstract

A near end point of presence (PoP) of a cloud proxy service receives, from a client device, a request for a network resource. A far end PoP from a plurality of PoPs of the cloud proxy service is identified. Responsive to determining that a version of the network resource is stored in the near end PoP, a request for the network resource is transmitted to the far end PoP with a version identifier that identifies that version. The far end PoP receives, from the near end PoP, a response that includes difference(s) between the version of the network resource stored in the near end PoP with a most current version of the network resource. The response does not include the entire network resource. The near end PoP applies the specified difference(s) to the version that it has stored to generate an updated version of the network resource, and transmits it to the client device.

Claims

1. A method in a near end point of presence (PoP) of a cloud proxy service, of reducing network resource transmission size, the method comprising:

receiving at a near end PoP of a cloud proxy service, a first request for a network resource from a first one of a plurality of client devices, wherein the first client device is physically separate from the near end PoP;

selecting a far end PoP from a plurality of PoPs of the cloud proxy service, wherein the far end PoP is closer to an origin server hosting the requested network resource than the near end PoP is relative to the origin server;

determining whether to perform a delta compression technique for reducing network resource transmission size for the first request, wherein determining whether to perform a delta compression technique includes;

tracking performance measures for network resources and the plurality of PoPs of the cloud proxy service, wherein tracking performance measures includes tracking for each network resource a measure of an average delta compression achieved for a determined geographical area, and

wherein determining whether to perform a delta compression technique for reducing network resource transmission size for the first request is based upon the performance measures; and

responsive to determining that the delta compression technique is to be performed, performing;

responsive to determining that a version of the network resource is not stored in the near end PoP, transmitting a second request for the network resource to the far end PoP,

receiving, from the far end PoP, a first response that includes the requested network resource,

transmitting, to the first client device, a second response that includes the requested network resource,

storing, in the near end PoP, the requested network resource as a first version of the network resource,

receiving, from a second one of the plurality of client devices, a third request for the network resource, wherein the second client device is physically separate from the near end PoP,

responsive to determining that the first version of the network resource is stored in the near end PoP, transmitting a fourth request for the network resource to the far end PoP, the fourth request including a first version identifier that identifies the first version of the network resource stored in the near end PoP,

receiving, from the far end PoP, a third response that includes a differences file that specifies a set of one or more differences between the first version of the network resource stored in the near end PoP with a most current version of the network resource, wherein the third response does not include the entire network resource,

applying the set of differences specified in the differences file to the first version of the network resource stored in the near end PoP to generate an updated version of the network resource, and

transmitting, to the second client device, a fourth response that includes the updated version of the network resource.

2. The method of claim 1, further comprising:

storing the updated version of the network resource in the near end PoP as a second version of the network resource.

3. The method of claim 2, further comprising:

storing a second version identifier associated with the second version of the network resource in the near end PoP.

4. The method of claim 3, wherein the second version identifier is included in the third response received from the far end PoP.

5. The method of claim 3, wherein prior to storing the second version identifier associated with the second version of the network resource, generating the second version identifier by hashing the second version of the network resource.

6. The method of claim 3, wherein the second version of the network resource and the second version identifier are stored in a dynamic dictionary of the near end PoP.

7. The method of claim 6, wherein the dynamic dictionary of the near end PoP is built when resources are received from the far end PoP.

8. The method of claim 1, wherein identifying the far end PoP from the plurality of PoPs of the cloud proxy service is based on respective response latency measures between each one of the plurality of PoPs and an origin server hosting the network resource.

9. The method of claim 1, wherein determining whether to perform the delta compression techniques is based on a measure of a response latency for the network resource.

10. The method of claim 1, wherein selecting the far end PoP from the plurality of PoPs includes automatically selecting the far end PoP based on respective geographic locations of the plurality of far end PoPs.

11. The method of claim 10, wherein selecting the far end PoP from the plurality of PoPs includes receiving through a user interface of the cloud proxy service a selection of the far end PoP from an administrator of the requested resource.

12. A non-transitory machine-readable storage medium that provides instructions that, when executed by a processor of a near end point of presence (PoP) of a cloud proxy service, cause said processor to perform operations comprising:

receiving at a near end PoP of a cloud proxy service, a first request for a network resource from a first one of a plurality of client devices, wherein the first client device is physically separate from the near end PoP;

selecting a far end PoP from a plurality of PoPs of the cloud proxy service, wherein the far end PoP is closer to an origin server hosting the requested network resource than the near end PoP is relative to the origin server;

determining whether to perform a delta compression technique for reducing network resource transmission size for the first request, wherein determining whether to perform a delta compression technique includes;

tracking performance measures for network resources and the plurality of PoPs of the cloud proxy service, wherein tracking performance measures includes tracking for each network resource a measure of an average delta compression achieved for a determined geographical area, and

wherein determining whether to perform a delta compression technique for reducing network resource transmission size for the first request is based upon the performance measures; and

responsive to determining that the delta compression technique is to be performed, performing;

responsive to determining that a version of the network resource is not stored in the near end PoP, transmitting a second request for the network resource to the far end PoP,

receiving, from the far end PoP, a first response that includes the requested network resource,

transmitting, to the first client device, a second response that includes the requested network resource,

storing, in the near end PoP, the requested network resource as a first version of the network resource,

receiving, from a second one of the plurality of client devices, a third request for the network resource, wherein the second client device is physically separate from the near end PoP,

responsive to determining that the first version of the network resource is stored in the near end PoP, transmitting a fourth request for the network resource to the far end PoP, the fourth request including a first version identifier that identifies the first version of the network resource stored in the near end PoP,

receiving, from the far end PoP, a third response that includes a differences file that specifies a set of one or more differences between the first version of the network resource stored in the near end PoP with a most current version of the network resource, wherein the third response does not include the entire network resource,

applying the set of differences specified in the differences file to the first version of the network resource stored in the near end PoP to generate an updated version of the network resource, and

transmitting, to the second client device, a fourth response that includes the updated version of the network resource.

13. The non-transitory machine-readable storage medium of claim 12, further comprising instructions, that when executed by the processor, cause said processor to perform the following operation:

storing the updated version of the network resource in the near end PoP as a second version of the network resource.

14. The non-transitory machine-readable storage medium of claim 13, further comprising instructions, that when executed by the processor, cause said processor to perform the following operation:

storing a second version identifier associated with the second version of the network resource in the near end PoP.

15. The non-transitory machine-readable storage medium of claim 14, wherein the second version identifier is included in the third response received from the far end PoP.

16. The non-transitory machine-readable storage medium of claim 14, wherein prior to storing the second version identifier associated with the second version of the network resource, generating the second version identifier by hashing the second version of the network resource.

17. The non-transitory machine-readable storage medium of claim 14, wherein the second version of the network resource and the second version identifier are stored in a dynamic dictionary of the near end PoP.

18. The non-transitory machine-readable storage medium of claim 17, wherein the dynamic dictionary of the near end PoP is built when resources are received from the far end PoP.

19. The non-transitory machine-readable storage medium of claim 12, wherein identifying the far end PoP from the plurality of PoPs of the cloud proxy service is based on respective response latency measures between each one of the plurality of PoPs and an origin server hosting the network resource.

20. The non-transitory machine-readable storage medium of claim 12, wherein determining whether to perform the delta compression techniques is based on a measure of a response latency for the network resource.

21. The non-transitory machine-readable storage medium that of claim 12, wherein selecting the far end PoP from the plurality of PoPs includes automatically selecting the far end PoP based on respective geographic locations of the plurality of far end PoPs.

22. The non-transitory machine-readable storage medium that of claim 21, wherein selecting the far end PoP from the plurality of PoPs includes receiving through a user interface of the cloud proxy service a selection of the far end PoP from an administrator of the requested resource.