Methods and apparatus for remotely updating executing processes

• US 8,706,834 B2
• Filed: 06/30/2011
• Issued: 04/22/2014
• Est. Priority Date: 06/30/2011
• Status: Active Grant

• Current Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Original Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Inventors: Sorenson, James Christopher III, Lin, Yun, Brugiolo, Ivan
• Primary Examiner(s): Vu, Viet
• Assistant Examiner(s): Emdadi, Keyvan
• Application Number:
  US13/174,172
• Publication Number:
  US 20130007183A1
• Time in Patent Office: 1,027 Days
• Field of Search: 709/213, 709/214, 709/216, 709/217
• US Class Current: 709/214
• CPC Class Codes: G06F 3/0607 by facilitating the process of upgrading existing storage systems, e.g. for improving compatibility between host and storage device
• CPC Class Codes: G06F 3/0632 by initialisation or re-initialisation of storage systems
• CPC Class Codes: G06F 3/067 Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
• CPC Class Codes: G06F 8/65 Updates security arrangements therefor G06F21/57

Abstract

Methods, apparatus, and computer-accessible storage media for remotely updating an executing process that receives I/O requests on I/O port(s), stores write data to a write log on local storage, and uploads data from the write log to remote storage. An update for the process is detected and downloaded, and an updated process is instantiated from the update. The current process is directed to perform a shutdown for update during an update window. In response, the current process saves its current configuration, flushes an in-memory portion of the write log to local storage, and releases its I/O port(s). The updated process loads the saved configuration, detects that the port(s) have been released, and starts accepting I/O requests on the ports. During flushing, the current process flushes current data in memory while continuing to append new write data, stops accepting new write requests, and then flushes the new write data.

Claims

1. A method, comprising:

detecting, by an update agent executing on a computing device on a local network, that an update package for a current process executing on the computing device is available on a remote network, wherein the current process receives I/O requests including write requests from one or more client processes via one or more I/O ports, appends write data indicated by the write requests to an in-memory portion of a write log on a local data store, and uploads write data from the write log to a remote data store;

downloading the update package to the computing device in response to said detecting;

directing the current process to start an update sequence;

storing, by the current process, a current configuration to an external store, wherein the current configuration includes at least an indication of the one or more I/O ports;

instantiating an updated process on the computing device according to the downloaded update package;

loading, by the updated process, the current configuration from the external store;

flushing, by the current process, write data from the in-memory portion of the write log to the local data store;

releasing, by the current process, the one or more I/O ports; and

receiving, by the updated process, I/O requests including write requests from the one or more client processes via the one or more I/O ports.

2. The method as recited in claim 1, wherein the remote data store is maintained by a storage service on the remote network, wherein the current process is a current version of a storage gateway that serves as an interface between the one or more client processes on the local network and the storage service on the remote network, and wherein the updated process is a newer version of the storage gateway.

3. The method as recited in claim 1, wherein said flushing, by the current process, write data from the in-memory portion of the write log to the local data store comprises:

flushing current write data in the in-memory portion of the write log to the local data store while accepting new write requests on the one or more I/O ports and appending new write data indicated by the new write requests to the in-memory portion of the write log; and

after completing said flushing of the current write data in the in-memory portion of the write log to the local data store, stop accepting write requests on the one or more I/O ports and flushing the new write data in the in-memory portion of the write log to the local data store.

4. The method as recited in claim 1, further comprising performing said directing, said storing, said instantiating, said loading, said flushing, and said releasing during a specified update window.

5. The method as recited in claim 1, wherein said directing the current process to start an update sequence is performed by the update agent or by the updated process.

6. The method as recited in claim 1, further comprising the updated process:

opening the one or more I/O ports to receive the I/O requests from the one or more client processes in response to determining that the one or more I/O ports have been released by the current process; and

appending write data indicated by the received write requests to the write log and uploading the write data from the write log to the data store.

7. The method as recited in claim 1, wherein the updated process is not instantiated on the computing device until after said flushing is complete.

8. The method as recited in claim 1, wherein the updated process is instantiated on the computing device prior to said directing the current process to start the update sequence.

9. A device, comprising:

at least one processor; and

a memory comprising program instructions, wherein the program instructions are executable by the at least one processor to implement a storage gateway process that receives I/O requests including write requests from one or more client processes on a local network via one or more I/O ports, appends write data indicated by the write requests to an in-memory portion of a write log on a local data store, and uploads write data from the write log to a remote data store;

wherein the program instructions are further executable by the at least one processor to download an update package from a remote network, direct the storage gateway process to shut down, and instantiate an updated storage gateway process in the memory according to the downloaded update package;

wherein, in response to receiving the direction to shut down, the storage gateway process persists a storage gateway configuration that includes an indication of the one or more I/O ports, flushes write data from the in-memory portion of the write log to the local data store, and releases the one or more I/O ports; and

wherein the updated storage gateway process loads the persisted storage gateway configuration and, subsequent to the storage gateway process releasing the one or more I/O ports, takes over storage gateway operations from the storage gateway process.

10. The device as recited in claim 9, wherein, to take over storage gateway operations from the storage gateway process, the updated storage gateway process detects that the one or more I/O ports have been released by the storage gateway process, opens the one or more I/O ports, and begins receiving and processing I/O requests including write requests from the one or more client processes via the one or more I/O ports.

11. The device as recited in claim 10, wherein the write requests received from the one or more client processes by the updated storage gateway process include at least one retry of a write request that was not accepted by the storage gateway process during said flushing write data from in-memory portion of the write log.

12. The device as recited in claim 10, wherein, to process the write requests received from the one or more client processes via the one or more I/O ports, the updated storage gateway process appends write data indicated by the write requests to the write log and uploads the write data from the write log to the remote data store.

13. The device as recited in claim 9, wherein, to flush write data from the in-memory portion of the write log to the local data store, the storage gateway process:

flushes current write data in the in-memory portion of the write log to the local data store while accepting new I/O requests including write requests on the one or more I/O ports and appending new write data indicated by the new write requests to the in-memory portion of the write log; and

after flushing the current write data in the in-memory portion of the write log to the local data store, stops accepting subsequent write requests on the one or more I/O ports and flushes the new write data in the in-memory portion of the write log to the local data store.

14. The device as recited in claim 13, wherein the storage gateway process replies to each of the subsequent write requests to inform a respective client process to retry the write request.

15. The device as recited in claim 9, wherein the storage gateway process persists the storage gateway configuration, flushes the write data from the in the in-memory portion of the write log to the local data store, and releases the one or more I/O ports during a specified update window.

16. The device as recited in claim 15, wherein the updated storage gateway process loads the persisted storage gateway configuration and takes over storage gateway operations from the storage gateway process during the specified update window.

17. The device as recited in claim 9, wherein the storage gateway process terminates subsequent to said releasing the one or more I/O ports.

18. A non-transitory computer-accessible storage medium storing program instructions computer-executable to implement a storage gateway process on a computing device, the storage gateway process operable to:

perform storage gateway operations that include receiving I/O requests including write requests from one or more client processes on a local network via one or more I/O ports, appending write data indicated by the write requests to an in-memory portion of a write log on a local data store, and uploading write data from the write log to a remote data store;

receive an indication that the storage gateway process is to shut down so that an updated storage gateway process can take over the storage gateway operations; and

in response to receiving said indication;

persist a storage gateway configuration that includes an indication of the one or more I/O ports;

flush write data from the in-memory portion of the write log to the local data store, wherein, to flush write data from the in-memory portion of the write log to the local data store, the storage gateway process is further operable to;

flush current write data in the in-memory portion of the write log to the local data store while accepting I/O requests including new write requests on the one or more I/O ports and appending new write data indicated by the new write requests to the in-memory portion of the write log, and

after flushing the current write data in the in-memory portion of the write log to the local data store, flush the new write data in the in-memory portion of the write log to the local data store while rejecting subsequent write requests on the one or more I/O ports; and

release the one or more I/O ports so that the updated storage gateway process can take over the storage gateway operations.

19. The non-transitory computer-accessible storage medium as recited in claim 18, wherein, to take over storage gateway operations from the storage gateway process, the updated storage gateway process is operable to detect that the one or more I/O ports have been released by the storage gateway process, open the one or more I/O ports, and receive and process I/O requests including write requests from the one or more client processes via the one or more I/O ports.

20. The non-transitory computer-accessible storage medium as recited in claim 18, wherein the storage gateway process is further operable to reply to each of the subsequent write requests to inform a respective client process to retry the write request.

21. The non-transitory computer-accessible storage medium as recited in claim 18, wherein the storage gateway process is further operable to persist the storage gateway configuration, flush the write data from the in-memory portion of the write log to the local data store, and release the one or more I/O ports during a specified update window.

22. The non-transitory computer-accessible storage medium as recited in claim 18, wherein the storage gateway process is further operable to terminate subsequent to said releasing the one or more I/O ports.