DELTA ENCODING IN STORAGE CLIENTS

• US 20170193028A1
• Filed: 12/31/2015
• Published: 07/06/2017
• Est. Priority Date: 12/31/2015
• Status: Active Application

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Iyengar, Arun K.
• Application Number:
  US14/985,513
• Publication Number:
  US 20170193028A1
• Time in Patent Office: Days
• CPC Class Codes: G06F 16/172 Caching, prefetching or hoarding of files
• CPC Class Codes: G06F 16/2358 Change logging, detection, and notification replication G06F16/27
• CPC Class Codes: G06F 16/2379 Updates performed during online database operations; commit processing
• CPC Class Codes: G06F 16/27 Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
• CPC Class Codes: G06F 21/645 using a third party

Abstract

Embodiments include method, systems and computer program products for delta encoding in storage clients. In some embodiments a storage client for accessing a storage service from a computer program may be provided. A cache may be provided within the storage client for reducing a number of accesses to the storage service. A data object may be stored in the storage service. For each of a plurality of updates to the data object, a delta may be stored in the storage service, wherein the delta encodes a difference between a new version and a previous version of the data object. A complete updated version of the data object may be stored in the storage service based on one of several factors.

Claims

1. A computer-implemented method comprising:

providing a storage client for accessing a storage service from a computer program;

providing a cache within the storage client for reducing a number of accesses to the storage service;

storing a data object in the storage service;

for each of a plurality of updates to the data object, storing a delta in the storage service, wherein the delta encodes a difference between a new version and a previous version of the data object; and

storing a complete updated version of the data object in the storage service in response to at least one of;

(1) a number of deltas from the previous version of the data object exceeds a first threshold;

(2) a total size of deltas from the previous version of the data object exceeds a second threshold;

or (3) the client determines that a cost of determining an updated version of the data object by applying deltas to a previous version exceeds a third threshold.

2. The method of claim 1, further comprising:

in response to storing the complete updated version of the data object in the storage service, deleting an earlier version of the data object and at least one delta on the storage service.

3. The method of claim 1, further comprising:

downloading, by the storage client, the previous version of the data object and at least one delta from the storage service; and

applying, by the storage client, the at least one delta to the previous version of the data object to obtain an updated version of the data object.

4. The method of claim 1, further comprising:

providing a compression method in the storage client to reduce a size of data objects.

5. The method of claim 4, further comprising:

varying a frequency of compressing data from the computer program based on assessing costs and benefits of compressing the data.

6. The method of claim 1, further comprising:

providing an encryption method in the storage client to preserve data privacy.

7. The method of claim 1, further comprising:

monitoring a latency for data accesses from the storage client to the storage service;

increasing an amount of caching in response to the latency increasing; and

decreasing the amount of caching in response to the latency decreasing.

8. The method of claim 1, further comprising:

in response to the storage service being unresponsive or responding too slowly, the application using the cache to allow the application to continue without communicating with the storage service.

9. A computer program product comprising a non-transitory storage medium readable by a processing circuit and storing instructions for execution by the processing circuit for performing a method comprising:

providing a storage client for accessing a storage service from a computer program;

providing a cache within the storage client for reducing a number of accesses to the storage service;

storing a data object in the storage service;

for each of a plurality of updates to the data object, storing a delta in the storage service, wherein the delta encodes a difference between a new version and a previous version of the data object; and

storing a complete updated version of the data object in the storage service in response to at least one of;

(1) a number of deltas from the previous version of the data object exceeds a first threshold;

(2) a total size of deltas from the previous version of the data object exceeds a second threshold;

or (3) the client determines that a cost of determining an updated version of the data object by applying deltas to a previous version exceeds a third threshold.

10. The computer program product of claim 10, the method further comprising:

in response to storing the complete updated version of the data object in the storage service, deleting an earlier version of the data object and at least one delta on the storage service.

11. The computer program product of claim 10, the method further comprising:

downloading, by the storage client, the previous version of the data object and at least one delta from the storage service; and

applying, by the storage client, the at least one delta to the previous version of the data object to obtain an updated version of the data object.

12. The computer program product of claim 10, the method further comprising:

providing a compression method in the storage client to reduce a size of data objects.

13. The computer program product of claim 12, the method further comprising:

varying a frequency of compressing data from the computer program based on assessing costs and benefits of compressing the data.

14. The computer program product of claim 10, the method further comprising:

providing an encryption method in the storage client to preserve data privacy.

15. The computer program product of claim 10, the method further comprising:

monitoring a latency for data accesses from the storage client to the storage service;

increasing an amount of caching in response to the latency increasing; and

decreasing the amount of caching in response to the latency decreasing.

16. The computer program product of claim 10, the method further comprising:

in response to the storage service being unresponsive or responding too slowly, the application using the cache to allow the application to continue without communicating with the storage service.

17. A system, comprising:

a processor in communication with one or more types of memory, the processor configured to;

provide a storage client for accessing a storage service from a computer program;

provide a cache within the storage client for reducing a number of accesses to the storage service;

store a data object in the storage service;

for each of a plurality of updates to the data object, store a delta in the storage service, wherein the delta encodes a difference between a new version and a previous version of the data object; and

store a complete updated version of the data object in the storage service in response to at least one of;

(1) a number of deltas from the previous version of the data object exceeds a first threshold;

(2) a total size of deltas from the previous version of the data object exceeds a second threshold;

or (3) the client determines that a cost of determining an updated version of the data object by applying deltas to a previous version exceeds a third threshold.

18. The system of claim 18, wherein the processor is further configured to:

in response to storing the complete updated version of the data object in the storage service, delete an earlier version of the data object and at least one delta on the storage service.

19. The system of claim 18, wherein the processor is further configured to:

download, by the storage client, the previous version of the data object and at least one delta from the storage service; and

apply, by the storage client, the at least one delta to the previous version of the data object to obtain an updated version of the data object.

20. The system of claim 18, wherein the processor is further configured to:

provide a compression method in the storage client to reduce a size of data objects.