COMPRESSED DATA OBJECTS REFERENCED VIA ADDRESS REFERENCES AND COMPRESSION REFERENCES

• US 20100274772A1
• Filed: 04/23/2009
• Published: 10/28/2010
• Est. Priority Date: 04/23/2009
• Status: Abandoned Application

• Current Assignee: Cirtas Systems, Inc.
• Original Assignee: Cirtas Systems, Inc.
• Inventors: Samuels, Allen
• Application Number:
  US12/429,140
• Publication Number:
  US 20100274772A1
• Time in Patent Office: Days
• US Class Current: 707/693
• CPC Class Codes: G06F 16/1748 De-duplication implemented within the file system, e.g. based on file segments de-duplication techniques in storage systems for the management of data blocks G06F3/0641
• CPC Class Codes: G06F 16/188 Virtual file systems

Abstract

A computing device maintains a mapping of a virtual storage to a physical storage. The mapping includes address references from data included in the virtual storage to one or more compressed data objects included in the physical storage. At least one of the one or more compressed data objects has been compressed at least in part by replacing portions of an uncompressed data object with compression references to matching portions of previously generated compressed data objects.

Claims

1. A method comprising:

maintaining, by a computing device, a mapping of a virtual storage to a physical storage, the mapping including address references from data included in the virtual storage to one or more compressed data objects included in the physical storage, wherein at least one of the one or more compressed data objects having been compressed at least in part by replacing portions of an uncompressed data object with compression references to matching portions of previously generated compressed data objects.

2. The method of claim 1, further comprising:

responding, by the computing device, to a request to access information represented by the data from a client by transferring one or more first compressed data objects referenced by the data via the address references and one or more second compressed data objects referenced by the one or more first compressed data objects via the compression references to the client.

3. The method of claim 2, wherein the responding is performed using a file system protocol, and wherein the compressed data objects are stored using an additional protocol that is not a file system protocol.

4. The method of claim 3, wherein the additional protocol is at least one of HTTP, SOAP and REST protocols.

5. The method of claim 1, wherein the virtual storage is a virtual block device or a virtual file system.

6. The method of claim 1, wherein each of the one or more compressed data objects has a reference count representing usage of the compressed data object by the data and by other compressed data objects

7. The method of claim 6, wherein the reference count includes the compression references to the compressed data object and the address references to the compressed data object.

8. The method of claim 6, further comprising:

generating a new compressed data object at least in part by replacing portions of a new uncompressed data object with references to matching portions of the one or more compressed data objects;

incrementing a reference count for each of the one or more compressed data objects having the matching portions; and

storing the new compressed data object in the physical storage.

9. The method of claim 6, further comprising:

generating a point-in-time copy of the data, wherein the point-in-time copy includes at least one of the address references of the data to the one or more compressed data objects.

10. The method of claim 9, further comprising:

subsequent to generating the point-in-time copy, receiving a request to make a modification to the data;

generating a new compressed data object that includes the modification;

updating the mapping to include a new address reference from the data to the new compressed data object; and

incrementing a reference count for the new compressed data object and for at least one of the one or more compressed data objects previously referenced by the virtual data.

11. The method of claim 6, further comprising:

receiving a command to delete the data;

removing the data from the virtual storage;

removing from the mapping the address references from the data;

decrementing the reference counts for the one or more compressed data objects that had been referenced by the data via the removed address references; and

deleting the compressed data objects for which the reference counts are zero.

12. The method of claim 1, further comprising:

storing the one or more compressed data objects in the physical storage, wherein the physical storage includes a storage cloud.

13. A method comprising:

managing reference counts for a plurality of compressed data objects by a computing device, wherein each of the compressed data objects has a reference count representing a number of address references made to the compressed data object by data included in a virtual storage and a number of compression references made to the compressed data object by other compressed data objects; and

determining, by the computing device, when it is safe to delete a compressed data object based on the reference count for the compressed data object.

14. The method of claim 13, wherein the address references are based on a mapping of the virtual storage, which includes the data, to a physical storage that includes the compressed data objects.

15. The method of claim 13, wherein the compressed data objects are generated at least in part by replacing portions of uncompressed data objects with compression references to matching portions of previously generated compressed data objects.

16. The method of claim 13, further comprising:

in response to generation of a new compressed data object that was generated at least in part by replacing portions of a new uncompressed data object with references to matching portions of the plurality of compressed data objects, incrementing reference counts for the plurality of compressed data objects having the matching portions.

17. The method of claim 13, further comprising:

in response to a request to modify the data after generation of a point-in-time copy of the data, incrementing a reference count for one or more of the plurality of compressed data objects that had been referenced by the data via an address reference.

18. The method of claim 13, further comprising:

in response to a request to delete the data from the virtual storage, decrementing the reference counts for the plurality of compressed data objects that had been referenced by the data via the address references.

19. The method of claim 13, further comprising:

causing those compressed data objects for which the reference count becomes zero to be deleted.

20. A computer readable storage medium including instructions that, when executed by a processing system, cause the processing system to perform a method comprising:

maintaining, by a computing device, a mapping of a virtual storage to a physical storage, the mapping including address references from data included in the virtual storage to one or more compressed data objects included in the physical storage, wherein at least one of the one or more compressed data objects having been compressed at least in part by replacing portions of an uncompressed data object with compression references to matching portions of previously generated compressed data objects.

21. The computer readable storage medium of claim 20, the method further comprising:

responding, by the computing device, to a request to access information represented by the data from a client by transferring one or more first compressed data objects referenced by the data via the address references and one or more second compressed data objects referenced by the one or more first compressed data objects via the compression references to the client.

22. The computer readable storage medium of claim 21, wherein the responding is performed using a file system protocol, and wherein the compressed data objects are stored using an additional protocol that is not a file system protocol.

23. The computer readable storage medium of claim 20, wherein each of the one or more compressed data objects has a reference count representing usage of the compressed data object by the data and by other compressed data objects, wherein the reference count includes the compression references to the compressed data object and the address references to the compressed data object.

24. The computer readable storage medium of claim 23, the method further comprising:

generating a new compressed data object at least in part by replacing portions of a new uncompressed data object with references to matching portions of the one or more compressed data objects;

incrementing a reference count for each of the one or more compressed data objects having the matching portions; and

storing the new compressed data object in the physical storage.

25. The computer readable storage medium of claim 23, the method further comprising:

generating a point-in-time copy of the data, wherein the point-in-time copy includes at least one of the address references of the data to the one or more compressed data objects.

26. The computer readable storage medium of claim 25, the method further comprising:

subsequent to generating the point-in-time copy, receiving a request to make a modification to the data;

generating a new compressed data object that includes the modification;

updating the mapping to include a new address reference from the data to the new compressed data object; and

incrementing a reference count for the new compressed data object and for at least one of the one or more compressed data objects previously referenced by the virtual data.

27. The computer readable storage medium of claim 23, the method further comprising:

receiving a command to delete the data;

removing the data from the virtual storage;

removing from the mapping the address references from the data;

decrementing the reference counts for the one or more compressed data objects that had been referenced by the data via the removed address references; and

deleting the compressed data objects for which the reference counts are zero.

28. A computer readable storage medium including instructions that, when executed by a processing system, cause the processing system to perform a method comprising:

managing reference counts for a plurality of compressed data objects by a computing device, wherein each of the compressed data objects has a reference count representing a number of address references made to the compressed data object by data included in a virtual storage and a number of compression references made to the compressed data object by other compressed data objects; and

determining, by the computing device, when it is safe to delete a compressed data object based on the reference count for the compressed data object.

29. The computer readable storage medium of claim 28, wherein the address references are based on a mapping of the virtual storage, which includes the data, to a physical storage that includes the compressed data objects.

30. The computer readable storage medium of claim 28, wherein the compressed data objects are generated at least in part by replacing portions of uncompressed data objects with compression references to matching portions of previously generated compressed data objects.

31. The computer readable storage medium of claim 28, the method further comprising:

in response to generation of a new compressed data object that was generated at least in part by replacing portions of a new uncompressed data object with references to matching portions of the plurality of compressed data objects, incrementing reference counts for the plurality of compressed data objects having the matching portions.

32. The computer readable storage medium of claim 28, the method further comprising:

in response to a request to modify the data after generation of a point-in-time copy of the data, incrementing a reference count for one or more of the plurality of compressed data objects that had been referenced by the data via an address reference.

33. The computer readable storage medium of claim 28, the method further comprising:

in response to a request to delete the data from the virtual storage, decrementing the reference counts for the plurality of compressed data objects that had been referenced by the data via the address references.

34. The computer readable storage medium of claim 28, the method further comprising:

causing those compressed data objects for which the reference count becomes zero to be deleted.

35. A computing apparatus comprising:

a memory including instructions for a user agent; and

a processor, connected with the memory, to execute the instructions, wherein the instructions cause the processor to;

maintain a mapping of a virtual storage to a physical storage, the mapping including address references from data included in the virtual storage to one or more compressed data objects included in the physical storage, wherein at least one of the one or more compressed data objects having been compressed at least in part by replacing portions of an uncompressed data object with compression references to matching portions of previously generated compressed data objects.

36. The computing apparatus of claim 35, further comprising:

the instructions to cause the processor to respond to a request to access information represented by the data from a client by transferring one or more first compressed data objects referenced by the data via the address references and one or more second compressed data objects referenced by the one or more first compressed data objects via the compression references to the client.

37. The computing apparatus of claim 36, wherein the processor to respond using a file system protocol, and wherein the compressed data objects are stored using an additional protocol that is not a file system protocol.

38. The computing apparatus of claim 35, wherein each of the one or more compressed data objects has a reference count representing usage of the compressed data object by the data and by other compressed data objects, and wherein the reference count includes the compression references to the compressed data object and the address references to the compressed data object.

39. The computing apparatus of claim 38, further comprising the instructions to cause the processor to:

generate a new compressed data object at least in part by replacing portions of a new uncompressed data object with references to matching portions of the one or more compressed data objects;

increment a reference count for each of the one or more compressed data objects having the matching portions; and

store the new compressed data object in the physical storage.

40. The computing apparatus of claim 38, further comprising the instructions to cause the processor to:

generate a point-in-time copy of the data, wherein the point-in-time copy includes at least one of the address references of the data to the one or more compressed data objects.

41. The computing apparatus of claim 40, further comprising the instructions to cause the processor to:

subsequent to generating the point-in-time copy, receive a request to make a modification to the data;

generate a new compressed data object that includes the modification;

update the mapping to include a new address reference from the data to the new compressed data object; and

increment a reference count for the new compressed data object and for at least one of the one or more compressed data objects previously referenced by the virtual data.

42. The computing apparatus of claim 38, further comprising the instructions to cause the processor to:

receive a command to delete the data;

remove the data from the virtual storage;

remove from the mapping the address references from the data;

decrement the reference counts for the one or more compressed data objects that had been referenced by the data via the removed address references; and

delete the compressed data objects for which the reference counts are zero.

43. A computing apparatus comprising:

a memory including instructions for a user agent; and

a processor, connected with the memory, to execute the instructions, wherein the instructions cause the processor to;

manage reference counts for a plurality of compressed data objects, wherein each of the compressed data objects has a reference count representing a number of address references made to the compressed data object by data included in a virtual storage and a number of compression references made to the compressed data object by other compressed data objects; and

determine when it is safe to delete a compressed data object based on the reference count for the compressed data object.

44. The computing apparatus of claim 43, wherein the address references are based on a mapping of the virtual storage, which includes the data, to a physical storage that includes the compressed data objects.

45. The computing apparatus of claim 43, wherein the compressed data objects are generated at least in part by replacing portions of uncompressed data objects with compression references to matching portions of previously generated compressed data objects.

46. The computing apparatus of claim 43, further comprising the instructions to cause the processor to:

in response to generation of a new compressed data object that was generated at least in part by replacing portions of a new uncompressed data object with references to matching portions of the plurality of compressed data objects, increment reference counts for the plurality of compressed data objects having the matching portions.

47. The computing apparatus of claim 43, further comprising the instructions to cause the processor to:

in response to a request to modify the data after generation of a point-in-time copy of the data, increment a reference count for one or more of the plurality of compressed data objects that had been referenced by the data via an address reference.

48. The computing apparatus of claim 43, further comprising the instructions to cause the processor to:

in response to a request to delete the data from the virtual storage, decrement the reference counts for the plurality of compressed data objects that had been referenced by the data via the address references.

49. The computing apparatus of claim 43, further comprising the instructions to cause the processor to:

cause those compressed data objects for which the reference count becomes zero to be deleted.