OBJECT STORAGE IN CLOUD WITH REFERENCE COUNTING USING VERSIONS

• US 20180205791A1
• Filed: 01/15/2017
• Published: 07/19/2018
• Est. Priority Date: 01/15/2017
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Elastifile Ltd. (Alphabet Inc.)
• Inventors: Frank, Shahar, Mosek, Amir
• Granted Patent: US 10,652,330 B2
• Time in Patent Office: Days
• CPC Class Codes: G06F 16/219 Managing data history or versioning querying versioned data G06F16/2474; querying temporal data G06F16/2477
• CPC Class Codes: H04L 67/1097 for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]

Abstract

A data storage apparatus includes an interface and one or more processors. The interface is configured for communicating with a cloud-based object storage system having a built-in versioning mechanism that assigns version numbers to objects stored therein. The one or more processors are configured to receive data for storage from one or more workloads, to store the data as objects in the cloud-based object storage system, and to update and record reference counts for at least some of the objects, by forcing the built-in versioning mechanism of the cloud-based object storage system to update the version numbers so as to match the reference counts.

Claims

1. A data storage apparatus, comprising:

an interface, for communicating with a cloud-based object storage system having a built-in versioning mechanism that assigns version numbers to objects stored therein; and

one or more processors, configured to receive data for storage from one or more workloads, to store the data as objects in the cloud-based object storage system, and to update and record reference counts for at least some of the objects, by forcing the built-in versioning mechanism of the cloud-based object storage system to update the version numbers so as to match the reference counts.

2. The apparatus according to claim 1, wherein the one or more processors are configured to calculate hash values over the data in the objects, and to store the objects in the cloud-based object storage system with the hash values serving as keys.

3. The apparatus according to claim 2, wherein the one or more processors are configured to update and record a reference count of a given object by detecting that an existing object was previously stored using a same hash value as the given object, and in response forcing the built-in versioning mechanism to increment a version number of the existing object.

4. The apparatus according to claim 1, wherein the one or more processors are configured to force the built-in versioning mechanism to update a version number of an object by sending to the cloud-based object storage system an instruction to update a metadata of the object but not a value of the object.

5. The apparatus according to claim 1, wherein the one or more processors are configured to force the built-in versioning mechanism to update a version number of an object by sending to the cloud-based object storage system an instruction to update both a metadata of the object but and a value of the object.

6. The apparatus according to claim 1, wherein, for a given object, the one or more processors are configured to:

define a dummy object associated with the given object; and

update a reference count for the given object by forcing the built-in versioning mechanism of the cloud-based object storage system to update a version number of the dummy object.

7. A method for data storage, comprising:

receiving data for storage from one or more workloads;

storing the data as objects in a cloud-based object storage system having a built-in versioning mechanism that assigns version numbers to the objects stored therein; and

updating and recording reference counts for at least some of the objects, by forcing the built-in versioning mechanism of the cloud-based object storage system to update the version numbers so as to match the reference counts.

8. The method according to claim 7, wherein storing the data comprises calculating hash values over the data in the objects, and storing the objects in the cloud-based object storage system with the hash values serving as keys.

9. The method according to claim 8, wherein updating and recording a reference count of a given object comprises detecting that an existing object was previously stored using a same hash value as the given object, and in response forcing the built-in versioning mechanism to increment a version number of the existing object.

10. The method according to claim 7, wherein forcing the built-in versioning mechanism to update a version number of an object comprises sending to the cloud-based object storage system an instruction to update a metadata of the object but not a value of the object.

11. The method according to claim 7, wherein forcing the built-in versioning mechanism to update a version number of an object comprises sending to the cloud-based object storage system an instruction to update both a metadata of the object but and a value of the object.

12. The method according to claim 7, wherein forcing the built-in versioning mechanism to update a version number of a give object comprises:

defining a dummy object associated with the given object; and

forcing the built-in versioning mechanism of the cloud-based object storage system to update the version number of the dummy object.

13. A computer software product, the product comprising a tangible non-transitory computer-readable medium in which program instructions are stored, which instructions, when read by one or more processors, cause the processors to receive data for storage from one or more workloads, to store the data as objects in a cloud-based object storage system having a built-in versioning mechanism that assigns version numbers to the objects stored therein, and to update and record reference counts for at least some of the objects, by forcing the built-in versioning mechanism of the cloud-based object storage system to update the version numbers so as to match the reference counts.