System and method for logical deletion of stored data objects

• US 10,534,726 B2
• Filed: 05/14/2018
• Issued: 01/14/2020
• Est. Priority Date: 09/21/2010
• Status: Active Grant

• Current Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Original Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Inventors: McHugh, Jason G., Gattu, Praveen Kumar, Ten-Pow, Michael A., Denny-Brown, II, Derek Ernest
• Primary Examiner(s): Li, Zhuo H
• Application Number:
  US15/979,327
• Publication Number:
  US 20180260341A1
• Time in Patent Office: 610 Days
• CPC Class Codes: G06F 11/1435 using file system or storage system metadata
• CPC Class Codes: G06F 11/1453 using de-duplication of the data
• CPC Class Codes: G06F 12/1466 Key-lock mechanism
• CPC Class Codes: G06F 16/162 Delete operations erasing in storage systems G06F3/0652
• CPC Class Codes: G06F 16/1873 Versioning file systems, temporal file systems, e.g. file system supporting different historic versions of files
• CPC Class Codes: G06F 16/2329 using versioning
• CPC Class Codes: G06F 2201/80 Database-specific techniques
• CPC Class Codes: G06F 2201/84 Using snapshots, i.e. a logical point-in-time copy of the data
• CPC Class Codes: G06F 2212/1052 Security improvement
• CPC Class Codes: G06F 3/0608 Saving storage space on storage systems
• CPC Class Codes: G06F 3/0619 in relation to data integrity, e.g. data losses, bit errors
• CPC Class Codes: G06F 3/0622 in relation to access
• CPC Class Codes: G06F 3/0652 Erasing, e.g. deleting, data cleaning, moving of data to a wastebasket
• CPC Class Codes: G06F 3/067 Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
• CPC Class Codes: G06F 3/0671 In-line storage system
• CPC Class Codes: G06F 3/0673 Single storage device

Abstract

Systems and methods for providing object versioning in a storage system may support the logical deletion of stored objects. In response to a delete operation specifying both a user key and a version identifier, the storage system may permanently delete the specified version of an object having the specified key. In response to a delete operation specifying a user key, but not a version identifier, the storage system may create a delete marker object that does not contain object data, and may generate a new version identifier for the delete marker. The delete marker may be stored as the latest object version of the user key, and may be addressable in the storage system using a composite key comprising the user key and the new version identifier. Subsequent attempts to retrieve the user key without specifying a version identifier may return an error, although the object was not actually deleted.

Claims

1. A system, comprising:

a data store that stores a plurality of objects, wherein at least some of the plurality of objects comprise a same object name and different version identifiers, and wherein the objects with the same name and the different version identifiers are stored in a bucket;

one or more processors;

a memory coupled to the one or more processors and storing program instructions that when executed by the one or more processors cause the one or more processors to perform;

receiving a request directed to an object from the data store, wherein the request specifies an object name for the object; and

in response to receiving the request directed to the object;

determining whether the request specifies a particular object version identifier for the object having the specified object name;

in response to determining the request specifies a particular object version identifier for the object, performing the request for the object having the specified object name and the specified particular object version identifier; and

in response to determining the request does not specify a particular object version identifier for the object, performing the request for an object version with the specified object name having a latest object version identifier.

2. The system of claim 1, wherein when executed by the one or more processors, the program instructions further cause the one or more processors to perform:

determining whether the latest object version identifier of the object with the specified object name comprises a delete marker; and

in response to determining the latest object version identifier of the object with the specified object name comprises a delete marker, refraining from returning object data contained in any of a plurality of archived objects stored in a bucket for objects with the specified object name.

3. The system of claim 2, wherein the delete marker does not contain object data.

4. The system of claim 1, wherein when executed by the one or more processors, the program instructions further cause the one or more processors to perform, prior to receiving the request directed to the object:

receiving a request to perform a delete operation, wherein the request to perform the delete operation includes the specified object name for the object but does not specify a version identifier; and

in response to receiving the request to perform the delete operation;

generating a new, unique version identifier;

creating a delete marker, wherein the delete marker comprises the specified object name for the object and the new, unique version identifier, and wherein the delete marker becomes the latest object version that is stored in a bucket for objects with the specified object name;

refraining from deleting any of a plurality of objects previously stored in the bucket for objects with the specified object name in response to receiving the request to perform the delete operation; and

storing the delete marker object in the bucket for objects with the specified object name.

5. The system of claim 1, wherein when executed by the one or more processors, the program instructions further cause the one or more processors to perform:

receiving a request to perform a delete operation, wherein the request to perform the delete operation includes the specified object name and a version identifier of an archived version of the object with the specified object name; and

in response to receiving the request to perform the delete operation, deleting the archived version of the object from the data store.

6. The system of claim 1, wherein when executed by the one or more processors, the program instructions further cause the one or more processors to perform:

receiving a request to perform an operation to list object names of the objects stored in the data store; and

in response to receiving the request to perform the operation to list object names;

returning a list of object names contained in buckets stored in the data store, wherein the list does not include object names for objects stored in one or more buckets wherein a delete marker object is the latest object version stored in the respective bucket.

7. The system of claim 1, wherein when executed by the one or more processors, the program instructions further cause the one or more processors to perform:

receiving a request to perform an operation to list all object versions stored in the data store; and

in response to receiving the request to perform the operation to list all object versions;

returning a list of all object versions stored in the data store, wherein the list includes any delete marker objects stored in the data store.

8. One or more non-transitory, computer-readable storage media storing program instructions that when executed on or across one or more processors cause the one or more processors to perform:

receiving a request directed to an object from a data store, wherein the data store stores multiple versions of objects for at least some objects stored in the data store, wherein the request specifies an object name but does not specify a version identifier;

determining an object version of the object having the specified object name with a latest object version identifier; and

performing the request for the object version of the object with the specified object name having the latest object version identifier.

9. The one or more non-transitory, computer-readable storage medium of claim 8, wherein when executed on or across the one or more processors, the program instructions further cause the one or more processors to perform:

receiving a request for an object from the data store, wherein the request specifies an object name and a version identifier; and

performing the request for a particular version of the object having the specified object name and the specified version identifier.

10. The one or more non-transitory, computer-readable storage media of claim 8, wherein when executed on or across the one or more processors, the program instructions further cause the one or more processors to perform:

receiving a request to perform a delete operation, wherein the request to perform the delete operation specifies an object name but does not specify a version identifier; and

in response to receiving the request to perform the delete operation;

generating a new, unique version identifier;

creating a delete marker object, wherein the delete marker object comprises the specified object name, specified in the request to perform the delete operation, and the new, unique version identifier, and wherein the delete marker object becomes the latest object version that is stored in a bucket for one or more objects with the specified object name, specified in the request to perform the delete operation; and

refraining from deleting any of the plurality of objects previously stored in the bucket in response to receiving the request to perform the delete operation; and

storing the delete marker object in the bucket.

11. The one or more non-transitory, computer-readable storage media of claim 10, wherein the delete marker object stored in the bucket does not contain object data.

12. The one or more non-transitory storage media of claim 8, wherein when executed on the one or more processors, the program instructions further cause the one or more processors to perform:

receiving a request to perform a delete operation, wherein the request specifies an object name and a version identifier, and wherein the specified object name and version identifier pair identifies a particular delete marker object; and

in response to receiving the request to perform the delete operation, deleting the particular delete marker object from the data store.

13. The one or more non-transitory, computer-readable storage media of claim 8, wherein when executed on the one or more processors, the program instructions further cause the one or more processors to perform:

suspending an object versioning feature for at least a portion of the data store;

receiving a request to perform a delete operation, wherein the request specifies an object name, but does not specify a version identifier; and

in response to receiving the request to perform the delete operation;

creating a new delete marker object comprising a special, sentinel version identifier and the specified object name, specified in the request to perform the delete operation; and

storing the new delete marker object in a bucket for one or more objects with the specified object name;

wherein creating the new delete marker comprises;

deleting object data contained in an object previously stored in the data store comprising the specified object name and the special sentinel version identifier; and

marking the object previously stored in the data store and comprising the specified object name and the special sentinel version identifier as a delete marker object.

14. A method, comprising:

receiving a request directed to an object from a data store, wherein the data store stores multiple versions of objects for at least some objects with a same object name, wherein the request specifies an object name for the object, but does not specify a version identifier;

determining an object version of the object having the specified object name with a latest object version identifier; and

performing the request for an object version of the object with the specified object name having the latest object version identifier.

15. The method of claim 14, further comprising:

receiving another request directed to an object from the data store, wherein the request specifies an object name for the object and a particular version identifier, wherein the version identifier is not a version identifier for the latest version of the object; and

performing the request for an object version of the object with the specified object name having the specified object version identifier.

16. The method of claim 14, further comprising:

receiving another request directed to an object from the data store, wherein the request specifies an object name for the object, but does not specify a version identifier;

determining an object version of the object having the specified object name with a latest object version identifier, wherein the latest object version identifier is a delete marker object; and

refraining from returning object data contained in any of a plurality of archived objects with the specified object name stored in the data store.

17. The method of claim 14, further comprising:

receiving another request directed to an object from the data store, wherein the request specifies an object name for the object and a particular version identifier, wherein the version identifier is not a version identifier for the latest version of the object, and wherein the version identifier is a delete marker object; and

returning an error indication.

18. The method of claim 14, further comprising:

receiving a request to delete an object from the data store, wherein the request specifies an object name for the object and a particular version identifier, wherein the version identifier is not a version identifier for the latest version of the object, and wherein the version identifier is a delete marker object; and

deleting the delete marker object from the data store.

19. The method of claim 14, further comprising:

suspending an object versioning feature for the data store;

receiving a request to perform a delete operation, wherein the request specifies an object name, but does not specify a version identifier; and

in response to receiving the request to perform the delete operation;

creating a new delete marker object comprising a special, sentinel version identifier and the specified object name; and

storing the new delete marker object in a bucket for one or more objects with the specified object name;

wherein creating the new delete marker comprises;

deleting object data contained in an object previously stored in the bucket and comprising the specified object name and the special sentinel version identifier; and

marking the object previously stored in the bucket and comprising the specified object name and the special sentinel version identifier as a delete marker object.

20. The method of claim 14, further comprising:

receiving a request to perform a delete operation, wherein the request to perform the delete operation specifies an object name but does not specify a version identifier; and

in response to receiving the request to perform the delete operation;

generating a new, unique version identifier;

creating a delete marker object, wherein the delete marker object comprises the specified object name and the new, unique version identifier, and wherein the delete marker object becomes the latest object version that is stored in a bucket that comprises the specified object name;

refraining from deleting any of the plurality of objects previously stored in the bucket in response to receiving the request to perform the delete operation; and

storing the delete marker object in the bucket.