Method and apparatus for reducing network resource transmission size using delta compression

• US 9,729,657 B2
• Filed: 03/17/2015
• Issued: 08/08/2017
• Est. Priority Date: 04/05/2012
• Status: Active Grant

• Current Assignee: CloudFlare Incorporated
• Original Assignee: CloudFlare Incorporated
• Inventors: Graham-Cumming, John
• Primary Examiner(s): Chea, Philip
• Assistant Examiner(s): Khan, Hassan
• Application Number:
  US14/659,909
• Publication Number:
  US 20160014226A1
• Time in Patent Office: 875 Days
• Field of Search: None
• CPC Class Codes: H04L 67/1023 based on a hash applied to IP addresses or costs
• CPC Class Codes: H04L 67/2876 Pairs of inter-processing entities at each side of the network, e.g. split proxies
• CPC Class Codes: H04L 67/5651 Reducing the amount or size of exchanged application data
• CPC Class Codes: H04L 67/568 Storing data temporarily at an intermediate stage, e.g. caching
• CPC Class Codes: H04L 69/02 Protocol performance
• CPC Class Codes: H04L 69/04 Protocols for data compression, e.g. ROHC

Abstract

A near end network optimizer receives, from a client device, a request for a network resource. Responsive to determining that a version of the network resource is stored in the near end network optimizer, a request for the network resource is transmitted to a far end network optimizer along with a version identifier that identifies that version. The near end network optimizer receives, from the far end network optimizer, a response that includes a differences file that specifies the difference(s) between the version of the network resource stored in the near end network optimizer with a most current version of the network resource. The response does not include the entire network resource. The near end network optimizer applies the specified difference(s) to the version that it has stored to generate an updated version of the network resource, and transmits the updated version of the network resource to the client device.

Claims

1. A method in a near end network optimizer of a client device for supporting a delta compression technique for reducing network resource transmission size, wherein the near end network optimizer is included in the client device and coupled with a far end network optimizer included in a server, the method comprising:

generating at the client device, a first request for a network resource;

responsive to determining that a version of the network resource is not stored in the near end network optimizer of the client device, transmitting the first request for the network resource to the far end network optimizer;

receiving, from the far end network optimizer, a first response that includes the requested network resource, wherein the first response also includes a directive that a cached version of the requested network resource is not to be used to respond to a future request for that requested network resource without successful revalidation with an origin server for the requested network resource;

storing, in the near end network optimizer of the client device, the requested network resource as a first version of the network resource regardless that the first response included the directive;

generating, at the client device, a second request for the network resource;

responsive to determining that the first version of the network resource is stored in the near end network optimizer of the client device, transmitting a third request for the network resource to the far end network optimizer, the third request including a first version identifier that identifies the first version of the network resource stored in the near end network optimizer;

receiving, from the far end network optimizer, a second response that includes a differences file that specifies a set of one or more differences between the first version of the network resource stored in the near end network optimizer of the client device with a most current version of the network resource, wherein the second response does not include the entire network resource regardless of the directive associated with the first version of the network resource; and

applying the set of differences specified in the differences file to the version of the network resource stored in the near end network optimizer of the client device to generate an updated version of the requested network resource in response to the second request generated at the client device.

2. The method of claim 1, further comprising:

storing the updated version of the network resource in the near end network optimizer of the client device as a second version of the network resource.

3. The method of claim 2, further comprising:

storing a second version identifier associated with the second version of the network resource in the near end network optimizer of the client device.

4. The method of claim 3, wherein the second version identifier is included in the second response received from the far end network optimizer.

5. The method of claim 3, wherein prior to storing the second version identifier associated with the second version of the network resource, generating the second version identifier by hashing the second version of the network resource.

6. The method of claim 3, wherein the second version of the network resource and the second version identifier are stored in a dynamic dictionary of the near end network optimizer of the client device.

7. The method of claim 6, wherein the dynamic dictionary of the near end network optimizer of the client device is built as resources are received from the far end network optimizer.

8. A method in a far end network optimizer that participates in reducing network resource transmission size through use of delta compression, wherein the far end network optimizer is included in an origin server and is coupled with a near end network optimizer included in a client device, the method comprising:

receiving, from the near end optimizer of the client device, a first request for a network resource;

in response to receiving the first request,

retrieving the requested network resource from the origin server that hosts the requested network resource,

transmitting the retrieved network resource to the near end network optimizer of the client device, and

storing the retrieved network resource as a first version of the requested network resource in a dynamic dictionary of the far end network optimizer regardless of whether the origin server indicates that a cached version of the retrieved network resource is not to be used to respond to a future request for that network resource without successful revalidation with the origin server;

receiving, from the near end network optimizer of the client device, a second request for the network resource, the second request including a version identifier that identifies a second version of the network resource stored in the near end network optimizer;

responsive to determining that the second version of the network resource is not stored in the dynamic dictionary of the far end network optimizer, performing the following;

retrieving a third version that is a most current version of the network resource from the origin server,

transmitting a response to the near end network optimizer that includes the third version of the network resource, and

storing the third version of the requested network resource in the dynamic dictionary of the far end network optimizer regardless of whether the origin server indicates that a cached version of the retrieved network resource is not to be used to respond to a future request for that network resource without successful revalidation with the origin server;

receiving, from the near end network optimizer of the client device, a third request for the network resource, the third request including a version identifier that identifies the first version of the network resource and indicates that the first version of the network resource is stored in the near end network optimizer; and

in response to the third request and responsive to determining that the first version of the network resource is stored in the dynamic dictionary of the far end network optimizer, performing the following;

retrieving a fourth version of the network resource, the fourth version being a most current version of the network resource from the origin server,

determining a set of one or more differences between the first version and the fourth version of the network resource,

transmitting the set of differences to the near end network optimizer regardless that the origin server indicated that a cached version of the network resource is not be used when responding to requests without successful revalidation with the origin server, and

storing the fourth version of the network resource in the dynamic dictionary of the far end network optimizer.

9. The method of claim 8, further comprising:

storing the fourth version of the network resource in the far end network optimizer.

10. The method of claim 9, further comprising:

storing a version identifier associated with the fourth version of the network resource in the far end network optimizer.

11. The method of claim 10, further comprising:

transmitting the version identifier associated with the fourth version of the network resource to the near end network optimizer.

12. The method of claim 10, further comprising:

generating the version identifier with the fourth version of the network resource by hashing the fourth version of the network resource.

13. The method of claim 10, wherein the fourth version of the network resource and the version identifier associated with the fourth version of the network resource are stored in a dynamic dictionary of the far end network optimizer.

14. The method of claim 10, wherein the dynamic dictionary of the far end network optimizer is built as requests are received from the near end network optimizer of the client device and resources are retrieved in response to those requests.

15. A non-transitory machine-readable storage medium that provides instructions that, when executed by a processor of a near end network optimizer, wherein the near end network optimizer is included in a client device and coupled with a far end network optimizer included in a server, cause said processor to perform operations comprising:

generating at the client device, a first request for a network resource;

responsive to determining that a version of the network resource is not stored in the near end network optimizer of the client device, transmitting the first request for the network resource to the far end network optimizer;

receiving, from the far end network optimizer, a first response that includes the requested network resource, wherein the first response also includes a directive that a cached version of the requested network resource is not to be used to respond to a future request for that requested network resource without successful revalidation with an origin server for the requested network resource;

storing, in the near end network optimizer of the client device, the requested network resource as a first version of the network resource regardless that the first response included the directive;

generating, at the client device, a second request for the network resource;

responsive to determining that the first version of the network resource is stored in the near end network optimizer of the client device, transmitting a third request for the network resource to the far end network optimizer, the third request including a first version identifier that identifies the first version of the network resource stored in the near end network optimizer;

receiving, from the far end network optimizer, a second response that includes a differences file that specifies a set of one or more differences between the first version of the network resource stored in the near end network optimizer of the client device with a most current version of the network resource, wherein the second response does not include the entire network resource regardless of the directive associated with the first version of the network resource; and

applying the set of differences specified in the differences file to the version of the network resource stored in the near end network optimizer of the client device to generate an updated version of the requested network resource in response to the second request generated at the client device.

16. The non-transitory machine-readable storage medium of claim 15, further comprising instructions, that when executed by the processor, cause said processor to perform the following operation:

storing the updated version of the network resource in the near end network optimizer of the client device as a second version of the network resource.

17. The non-transitory machine-readable storage medium of claim 16, further comprising instructions, that when executed by the processor, cause said processor to perform the following operation:

storing a second version identifier associated with the second version of the network resource in the near end network optimizer of the client device.

18. The non-transitory machine-readable storage medium of claim 17, wherein the second version identifier is included in the second response received from the far end network optimizer.

19. The non-transitory machine-readable storage medium of claim 17, wherein prior to storing the second version identifier associated with the second version of the network resource, generating the second version identifier by hashing the second version of the network resource.

20. The non-transitory machine-readable storage medium of claim 17, wherein the second version of the network resource and the second version identifier are stored in a dynamic dictionary of the near end network optimizer of the client device.

21. The non-transitory machine-readable storage medium of claim 20, wherein the dynamic dictionary of the near end network optimizer of the client device is built as resources are received from the far end network optimizer.

22. A non-transitory machine-readable storage medium that provides instructions that, when executed by a processor of a far end network optimizer, wherein the far end network optimizer is included in an origin server and is coupled with a near end network optimizer included in a client device, cause said processor to perform operations comprising:

receiving, from the near end optimizer of the client device, a first request for a network resource;

in response to receiving the first request,

retrieving the requested network resource from the origin server that hosts the requested network resource,

transmitting the retrieved network resource to the near end network optimizer of the client device, and

storing the retrieved network resource as a first version of the requested network resource in a dynamic dictionary of the far end network optimizer regardless of whether the origin server indicates that a cached version of the retrieved network resource is not to be used to respond to a future request for that network resource without successful revalidation with the origin server;

receiving, from the near end network optimizer of the client device, a second request for the network resource, the second request including a version identifier that identifies a second version of the network resource stored in the near end network optimizer;

responsive to determining that the second version of the network resource is not stored in the dynamic dictionary of the far end network optimizer, performing the following;

retrieving a third version that is a most current version of the network resource from the origin server,

transmitting a response to the near end network optimizer that includes the third version of the network resource, and

storing the third version of the requested network resource in the dynamic dictionary of the far end network optimizer regardless of whether the origin server indicates that a cached version of the retrieved network resource is not to be used to respond to a future request for that network resource without successful revalidation with the origin server;

receiving, from the near end network optimizer of the client device, a third request for the network resource, the third request including a version identifier that identifies the first version of the network resource and indicates that the first version of the network resource is stored in the near end network optimizer; and

in response to the third request and responsive to determining that the first version of the network resource is stored in the dynamic dictionary of the far end network optimizer, performing the following;

retrieving a fourth version of the network resource, the fourth version being a most current version of the network resource from the origin server,

determining a set of one or more differences between the first version and the fourth version of the network resource,

transmitting the set of differences to the near end network optimizer regardless that the origin server indicated that a cached version of the network resource is not be used when responding to requests without successful revalidation with the origin server, and

storing the fourth version of the network resource in the dynamic dictionary of the far end network optimizer.

23. The non-transitory machine-readable storage medium of claim 22, further comprising instructions, that when executed by the processor, cause said processor to perform the following operation:

storing the fourth version of the network resource in the far end network optimizer.

24. The non-transitory machine-readable storage medium of claim 23, further comprising instructions, that when executed by the processor, cause said processor to perform the following operation:

storing a version identifier associated with the fourth version of the network resource in the far end network optimizer.

25. The non-transitory machine-readable storage medium of claim 24, further comprising instructions, that when executed by the processor, cause said processor to perform the following operation:

transmitting the version identifier associated with the fourth version of the network resource to the near end network optimizer.

26. The non-transitory machine-readable storage medium of claim 24, further comprising instructions, that when executed by the processor, cause said processor to perform the following operation:

generating the version identifier with the fourth version of the network resource by hashing the fourth version of the network resource.

27. The non-transitory machine-readable storage medium of claim 24, wherein the fourth version of the network resource and the version identifier associated with the fourth version of the network resource are stored in a dynamic dictionary of the far end network optimizer.

28. The non-transitory machine-readable storage medium of claim 24, wherein the dynamic dictionary of the far end network optimizer is built as requests are received from the near end network optimizer of the client device and resources are retrieved in response to those requests.