Storage object deletion job management

• US 9,052,942 B1
• Filed: 12/14/2012
• Issued: 06/09/2015
• Est. Priority Date: 12/14/2012
• Status: Expired due to Fees

• Current Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Original Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Inventors: Barber, Jeffrey Michael, Gattu, Praveen Kumar, Denny-Brown, Derek Ernest II, Perry, Carl Yates, Elving, Christopher Henning
• Primary Examiner(s): Lee, Adam
• Application Number:
  US13/715,883
• Time in Patent Office: 907 Days
• Field of Search: None
• US Class Current: 1/1
• CPC Class Codes: G06F 3/0604 Improving or facilitating administration, e.g. storage management
• CPC Class Codes: G06F 3/0652 Erasing, e.g. deleting, data cleaning, moving of data to a wastebasket
• CPC Class Codes: G06F 3/067 Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
• CPC Class Codes: G06F 9/4843 by program, e.g. task dispatcher, supervisor, operating system
• CPC Class Codes: G06F 9/5022 Mechanisms to release resources

Abstract

Methods and apparatus for storage object deletion job management are disclosed. A storage medium stores program instructions that when executed on a processor, identify, during a particular deletion execution iteration, a particular deletion job object stored at a multi-tenant storage service, wherein the particular deletion job object indicates a collection of storage objects that are eligible for deletion from the storage service in accordance with specified deletion criteria. The instructions determine, based on a job validity criterion, whether deletion operations corresponding to the particular deletion job object of the one or more deletion job objects are to be scheduled. If the job object is validated, the instructions initiate a deletion operation for storage objects indicated in the particular deletion job object.

Claims

1. A system, comprising one or more computing devices configured to:

implement a network-accessible multi-tenant storage service for storage objects owned by a plurality of clients;

identify a set of storage objects of the storage service to be deleted in accordance with an object retention policy;

store, within the storage service in accordance with a priority-based object naming policy, a plurality of deletion job objects, wherein each deletion job object of the plurality of deletion job objects comprises an indication of a subset of the set of storage objects;

determine, based at least in part on a job validity criterion, whether one or more deletion operations corresponding to a particular deletion job object of the plurality of deletion job objects are to be scheduled, wherein the job validity criterion includes an indication of an expiration period for the particular deletion job object;

in response to a determination that the one or more deletion operations are to be scheduled,

add, to a collection of task objects awaiting implementation, a task object indicating at least one storage object indicated in the particular deletion job object, and

implement a respective at least one deletion operation for the at least one storage object indicated in the task object; and

in response to a determination that the one or more deletion operations are not to be scheduled, initiate removal of the particular deletion job object to avoid scheduling redundant deletion job objects that indicate one or more same storage objects that are to be deleted in accordance with the object retention policy.

2. The system as recited in claim 1, wherein the particular deletion job object comprises an indication of a particular storage object to be deleted, wherein the one or more computing devices are further configured to:

store, within the storage service, in accordance with the deletion criterion, a second deletion job object indicating that the particular storage object is to be deleted, without determining whether deletion operations corresponding to the particular deletion job object were scheduled.

3. The system as recited in claim 1, wherein in accordance with the priority-based object naming policy, the particular deletion job object is stored in a logical container whose name comprises an indication of an expiration time of at least one storage object indicated in the particular deletion job object.

4. The system as recited in claim 1, wherein the one or more computing devices are further configured to:

in response to a determination that the deletion operations are not to be scheduled, initiate a removal of the particular deletion job object from the storage service.

5. A method, comprising:

implementing a network-accessible multi-tenant storage service for storage objects owned by a plurality of clients;

identifying a set of storage objects of the storage service to be deleted;

storing, within the storage service, a plurality of deletion job objects, wherein each deletion job object of the plurality of deletion job objects comprises an indication of a subset of the set of storage objects;

determining, based at least in part on a job validity criterion, whether one or more deletion operations corresponding to a particular deletion job object of the plurality of deletion job objects are to be scheduled, wherein the job validity criterion includes an indication of an expiration period for the particular deletion job object;

in response to determining that the one or more deletion operations are to be scheduled, initiating a respective at least one deletion operation for at least one storage object indicated in the particular deletion job object; and

in response to determining that one or more deletion operations for another deletion job object are not to be scheduled, initiating removal of the other deletion job object to avoid scheduling redundant deletion job objects that indicate one or more same storage objects that are to be deleted in accordance with the object retention policy.

6. The method as recited in claim 5, wherein the particular deletion job object comprises an indication of a particular storage object to be deleted, further comprising:

storing, within the storage service, in accordance with a deletion criterion, a second deletion job object indicating that the particular storage object is to be deleted, without determining whether deletion operations corresponding to the particular deletion job object were scheduled.

7. The method as recited in claim 5, wherein said storing the one or more deletion job objects in the storage service comprises:

storing the particular deletion job object in a logical container whose name comprises an indication of an expiration time of at least one storage object indicated in the particular deletion job object.

8. The method as recited in claim 5, further comprising:

determining, based at least in part on the job validity criterion, whether deletion operations corresponding to a different deletion job object of the plurality of deletion job objects are to be scheduled; and

in response to determining that the deletion operations corresponding to the different deletion job object are not to be scheduled, initiating a deletion of the different deletion job object from the storage service.

9. The method as recited in claim 5, wherein said initiating a deletion operation for at least one storage object indicated in the particular deletion job object comprises inserting a task object in a collection of task objects awaiting execution.

10. The method as recited in claim 9, wherein the collection of task objects comprises an ordered list of task objects, wherein said inserting the task object comprises:

inserting the task object in a particular position in the ordered list selected in accordance with a task execution sequencing criterion.

11. The method as recited in claim 10, wherein the ordered list is ordered based at least in part on a respective universally unique identifier (UUID) assigned to each task object of the ordered list, further comprising:

generating a particular UUID for the task object, wherein at least a portion of the UUID is computed using a random object generator.

12. The method as recited in claim 9, further comprising:

prior to initiating the deletion operation for the at least one storage object indicated in the particular deletion job object, removing, by a worker thread, the task object from the collection of task objects awaiting execution, and inserting the task object in a collection of in-progress task objects.

13. The method as recited in claim 12, wherein said inserting the task object in the collection of task objects awaiting execution is performed by a task dispatcher, further comprising:

notifying the task dispatcher, by the worker thread, that a deletion of a metadata entry of the at least one storage object has been completed; and

in response to said notifying, the task dispatcher determining whether to initiate a deletion of the particular deletion job object from the storage service.

14. A non-transitory computer-accessible storage medium storing program instructions that when executed on one or more processors:

identify, during a particular deletion execution iteration, a particular deletion job object of a plurality of deletion job objects stored in a multi-tenant storage service, wherein the particular deletion job object indicates a collection of storage objects that are eligible for deletion from the storage service in accordance with one or more deletion criteria;

determine, based at least in part on a job validity criterion, whether one or more deletion operations corresponding to the particular deletion job object of the plurality of deletion job objects are to be scheduled, wherein the job validity criterion includes an indication of an expiration period for the particular deletion job object;

in response to a determination that the one or more deletion operations are to be scheduled, initiate a respective at least one deletion operation for at least one storage object indicated in the particular deletion job object; and

in response to a determination that the one or more deletion operations are not to be scheduled, initiate removal of the particular deletion job object to avoid scheduling redundant deletion job objects that indicate one or more same storage objects that are to be deleted in accordance with the object retention policy.

15. The non-transitory computer-accessible storage medium as recited in claim 14, wherein the instructions when executed on the one or more processors:

determine, based at least in part on the job validity criterion, whether deletion operations corresponding to a different deletion job object of the one or more deletion job objects are to be scheduled; and

in response to a determination that the deletion operations corresponding to the different deletion job object are not to be scheduled, initiate a deletion of the different deletion job object from the storage service.

16. The non-transitory computer-accessible storage medium as recited in claim 14, wherein the instructions when executed on the one or more processors:

in response to a determination that the deletion operations are to be scheduled, insert a task object in a collection of task objects awaiting execution.

17. The non-transitory computer-accessible storage medium as recited in claim 16, wherein the collection of task objects comprises an ordered list of task objects, wherein the instructions when executed on the one or more processors:

insert the task object in a particular position in the ordered list selected in accordance with a task execution sequencing criterion.

18. The non-transitory computer-accessible storage medium as recited in claim 17, wherein the ordered list is ordered based at least in part on a respective universally unique identifier (UUID) assigned to each task object of the ordered list, wherein the instructions when executed on the one or more processors:

generate a particular UUID for the task object using a random object generator.

19. The non-transitory computer-accessible storage medium as recited in claim 16, wherein the instructions when executed on the one or more processors:

in response to a request for a task assignment from a worker thread, provide an indication of the task object to the worker thread.

20. The non-transitory computer-accessible storage medium as recited in claim 19, wherein the instructions when executed on the one or more processors:

in response to a determination that at least one deletion operation indicated in the task object has not been completed, remove the task object from an in-progress task collection into which the task object was placed by the worker thread, and re-insert the task object in the collection of task objects awaiting execution.

21. The non-transitory computer-accessible storage medium as recited in claim 14, wherein the particular deletion job object is stored in a particular logical container of the storage service, wherein the name of the particular logical container comprises an indication of a priority of a deletion of at least one storage object listed in the particular deletion job object.