PROVIDING EXECUTING PROGRAMS WITH ACCESS TO STORED BLOCK DATA OF OTHERS

• US 20100037031A1
• Filed: 08/08/2008
• Published: 02/11/2010
• Est. Priority Date: 08/08/2008
• Status: Active Grant

• Current Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Original Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Inventors: Jorgensen, Alte Normann, DeSantis, Peter N., Certain, Tate Andrew, Garman, Matthew S., Paterson-Jones, Roland
• Granted Patent: US 8,725,967 B2
• Time in Patent Office: Days
• US Class Current: 711/162
• CPC Class Codes: G06F 11/1461 Backup scheduling policy
• CPC Class Codes: G06F 11/1464 for networked environments
• CPC Class Codes: G06F 11/2071 using a plurality of controllers
• CPC Class Codes: G06F 2201/84 Using snapshots, i.e. a logical point-in-time copy of the data
• CPC Class Codes: G06F 3/0619 in relation to data integrity, e.g. data losses, bit errors
• CPC Class Codes: G06F 3/0622 in relation to access
• CPC Class Codes: G06F 3/065 Replication mechanisms
• CPC Class Codes: G06F 3/067 Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]

Abstract

Techniques are described for managing access of executing programs to non-local block data storage. In some situations, a block data storage service uses multiple server storage systems to reliably store copies of network-accessible block data storage volumes that may be used by programs executing on other physical computing systems, and snapshot copies of some volumes may also be stored (e.g., on remote archival storage systems). A group of multiple server block data storage systems that store block data volumes may in some situations be co-located at a data center, and programs that use volumes stored there may execute on other computing systems at that data center, while the archival storage systems may be located outside the data center. The snapshot copies of volumes may be used in various ways, including to allow users to obtain their own copies of other users'"'"' volumes (e.g., for a fee).

Claims

1. A method for a computing system of a block data storage service to provide access to remote stored block data for executing programs, the method comprising:

receiving a request to initiate access of a first executing program to non-local block data storage provided by a block data storage service, the first program executing on a first of a plurality of computing systems that are co-located at a first geographical location and that share one or more internal networks, the block data storage service using a first group of multiple other of the plurality of computing systems as block data storage systems that provide block data storage to multiple executing programs, the first computing system not being part of the first group;

in response to the received request, attaching a first block data storage volume to the first computing system for use by the executing first program, the executing first program and the first block data storage volume being associated with a first user, the first block data storage volume having a primary copy stored on a second computing system and having a mirror copy stored on a third computing system, the second and third computing systems each being part of the first group of block data storage systems, the attaching of the first block data storage volume including associating a first logical local block storage device of the first computing system with the first block data storage volume; and

under control of a system manager module of the block data storage service, managing the providing of the block data storage to the multiple executing programs, by;

after receiving indications of one or more data access requests initiated by the first executing program to the first logical local block storage device, automatically responding by performing the data access requests on the first block data storage volume, the performing of the data access requests causing one or more modifications to block data stored on the first block data storage volume in such a manner that the primary and mirror copies of the first block data storage volume each store the same block data;

at a first time after the performing of the data access requests, creating a first snapshot copy of the first block data storage volume on one or more remote archival data storage systems at a second geographical location that is distinct from the first geographical location, the first snapshot copy including a copy of the block data stored on the first block data storage volume at the first time; and

after creating the first snapshot copy,

receiving a request from a second user for a second executing program associated with the second user to have access to a second block data storage volume that includes the block data contained in the created first snapshot copy; and

after obtaining a fee from the second user for the access to the second block data storage volume and determining to grant the request, creating the second block data storage volume based on the created first snapshot copy and providing access to the second block data storage volume for the second executing program, the providing of the access including performing one or more other data access requests received from the second program on the created second block data storage volume.

2. The method of claim 1 wherein the creating of the first snapshot copy of the first block data storage volume is performed in response to instructions from the first user, wherein the determining to grant the request is based at least in part on the first user providing authorization to provide access to other users to the block data contained in the created first snapshot copy, and wherein the method further comprises, under control of a node manager module of the block data storage service that manages access of the first computing system to the internal networks:

receiving the data access requests initiated by the first executing program to the first logical local block storage device; and

facilitating performance of the received data access requests by interacting over the internal networks with the system manager module to provide the indications of the data access requests, the interacting over the internal networks being performed in a manner that is transparent to the first executing program.

3. The method of claim 2 wherein the multiple programs are executed by a program execution service on a second group of multiple of the plurality of computing systems, wherein the computing systems of the second group are distinct from the computing systems of the first group, wherein the first computing system hosts multiple virtual machines that are each able to execute at least one program, wherein the first executing program is one of the multiple programs and is a virtual machine image being executed by at least one of the multiple virtual machines hosted by the first computing system, and wherein the first node manager module executes as part of a virtual machine monitor for the first computing system.

4. A computer-implemented method for providing access of executing programs to block data storage functionality, the method comprising:

receiving one or more indications of one or more data access requests initiated by a first program executing on a first computing system to access block data stored on a non-local block data storage volume, the first block data storage volume being provided by a distinct second data storage system that is at a common first geographical location with the first computing system and that is separated from the first computing system via one or more networks, the first block data storage volume and the first program being associated with a first user;

automatically responding to the received indications of the data access requests by interacting with the second data storage system on behalf of the first executing program so as to initiate performance of the data access requests on the first block data storage volume provided by the second data storage system, the performance of the data access requests causing one or more modifications to the block data stored on the first block data storage volume;

after the responding to the received indications of the data access requests, creating a first snapshot copy of the first block data storage volume on one or more data storage systems at a second geographical location that is distinct from the first geographical location, the created first snapshot copy containing the block data stored on the first block data storage volume at a first time, the block data contained in the first snapshot copy including the one or more modifications; and

providing access to the created first snapshot copy to one or more users other than the first user for a fee, such that each of the one or more other users who provides the fee receives access for one or more executing programs to a distinct block data storage volume that includes the block data contained in the created first snapshot copy.

5. The method of claim 4 wherein the providing of the access to the first snapshot copy to one of the other users includes receiving an indication of a second program associated with the one other user, automatically attaching a second block data storage volume to a second computing system that is executing the second program, the second block data storage volume being distinct from the first block data storage volume and being stored on a third block data storage system and including the block data contained in the created first snapshot copy, and in response to received indications of one or more other data access requests initiated by the second program for the second block data storage volume, interacting with the third block data storage system on behalf of the second program so as to initiate performance of the other data access requests on the second block data storage volume.

6. The method of claim 5 wherein the providing of the access to the first snapshot copy to the one other user is performed in response to a request from the one other user for the access and further includes creating the second block data storage volume for the one other user in response to the request.

7. The method of claim 4 wherein the creating of the first snapshot copy is performed in response to instructions from the first user, the instructions further indicating criteria regarding whether other users receive access to the first snapshot copy.

8. The method of claim 7 wherein the indicated criteria include information about at least one other user who is authorized to receive access to the first snapshot copy, and wherein the providing of the access to the first snapshot copy to each of the one or more other users includes automatically determining that the other user is among the at least one authorized other users.

9. The method of claim 8 wherein the providing of the access to the first snapshot copy to each of the one or more other users includes receiving a request from the other user for the access to the first snapshot copy, the received request from the other user including information for use in determining whether the other user is among the at least one authorized other users.

10. The method of claim 4 wherein the providing of the access to the first snapshot copy to each of the one or more other users includes obtaining the fee from the other user, and wherein the first user receives at least some of the obtained fees from the other users.

11. The method of claim 4 further comprising, before the receiving of the indications of the one or more data access requests, attaching the first block data storage volume to the first computing system for use by the first executing program, the attaching of the first block data storage volume including associating a logical local block storage device of the first computing system with the first block data storage volume provided by the second data storage system and being performed by a node manager module that manages access of the first computing system to the one or more networks, the node manager module being part of a block data storage service that previously creates the block data storage volume in response to a request from the first user, and wherein the initiating by the first executing program of the one or more data access requests includes interactions by the first executing program with the logical local block storage device.

12. The method of claim 4 wherein the first computing system and the second data storage system are a subset of a plurality of computing systems co-located at the first geographical location, wherein the plurality of computing systems includes multiple block data storage systems that are provided by a block data storage service, wherein the second data storage system is one of the multiple block data storage systems, wherein the first geographical location is a data center, wherein the second geographical location is external to the data center, and wherein the one or more data storage systems at the second geographical location are archival data storage systems provided by a remote storage service.

13. The method of claim 12 wherein the first program is executed by a program execution service that executes multiple programs for multiple users on multiple of the plurality of computing systems, wherein the first computing system hosts multiple virtual machines that are each able to execute at least one program, wherein the first executing program is one of the multiple programs and is a virtual machine image being executed by at least one of the multiple virtual machines hosted by the first computing system, wherein the receiving of the indications of the data access requests and the automatic responding to the received indications of the data access requests are performed as part of execution of a virtual machine monitor for the first computing system, and wherein the first computing system includes one or more other actual local storage devices that are available for use by the first executing program.

14. A computer-readable medium whose contents enable one or more computing systems to provide access of executing programs to block data storage functionality, by performing a method comprising:

providing access of a first program executing on a first computing system for a first user to a non-local block data storage volume, the provided access enabling the first program to initiate data access requests to the block data storage volume, the block data storage volume being provided by a second block data storage system that is separated from the first computing system by one or more networks;

automatically responding to one or more received indications of one or more data access requests initiated by the first program for the block data storage volume, the responding including initiating performance of the data access requests on the block data storage volume on the second block data storage system;

after the responding to the received indications of the data access requests, providing paid access of a second executing program for a distinct second user to a stored first copy of the block data storage volume; and

automatically responding to one or more received indications of one or more other data access requests initiated by the second program to the block data storage volume first copy, the responding including initiating performance of the other data access requests on the block data storage volume first copy.

15. The computer-readable medium of claim 14 wherein the method further comprises, after the responding to the received indications of the data access requests, creating a first snapshot copy of the block data storage volume on one or more data storage systems remote from the second block data storage system, the created first snapshot copy containing block data stored on the block data storage volume at a first time, wherein the stored first copy of the block data storage volume is a block data storage volume copy that is based on the created first snapshot copy, wherein the block data storage volume and the executing first program are associated with the first user, and wherein the providing of the paid access to the second user includes providing the access in exchange for a fee from the second user to the first user.

16. The computer-readable medium of claim 15 wherein the first computing system and the second block data storage system are located at a single first geographical location, wherein the one or more data storage systems are remote archival storage systems at a distinct second geographical location, wherein the performance of the data access requests on the block data storage volume and the performance of the other data access requests on the block data storage volume first copy are automatically performed by a system manager module of a block data storage service, wherein the providing of the access for the first executing program to the block data storage volume is performed by a node manager module of the block data storage service that is associated with the first computing system and that manages access of the first executing program to the one or more networks, and wherein the automatic responding to the received indications of the data access requests is performed in part under control of the node manager module by initiating sending of the received data access requests over the one or more networks to the second block data storage system.

17. The computer-readable medium of claim 14 wherein the first computing system and the second block data storage system are located at a single first geographical location, wherein the second program executes on a second computing system at a distinct second geographical location, and wherein the stored first copy of the block data storage volume is a distinct second block data storage volume that is a copy of the block data storage volume created at the second geographical location.

18. The computer-readable medium of claim 14 wherein the second program executes on a second computing system at a common geographical location with the first computing system and the second block data storage system, wherein the performance of the data access requests on the block data storage volume on the second block data storage system includes performing the data access requests on a primary copy of the block data storage volume that is stored on the second block data storage system, and wherein the stored first copy of the block data storage volume is the primary copy of the block data storage volume stored on the second block data storage system.

19. The computer-readable medium of claim 14 wherein the computer-readable medium is at least one of a memory of a computing system that stores the contents and a data transmission medium that includes a generated stored data signal containing the contents.

20. The computer-readable medium of claim 14 wherein the contents are instructions that when executed cause the one or more computing systems to perform the method.

21. A system configured to provide access of executing programs to block data storage functionality, comprising:

one or more memories; and

a block data storage system manager module configured to provide a block data storage service that uses multiple block data storage systems to store block data storage volumes that are created by users of the block data storage service and accessed over one or more networks on behalf of one or more executing programs associated with the users, the providing of the block data storage service including;

creating one or more block data storage volumes for use by one or more executing programs, each of the block data storage volumes being stored on one of the multiple block data storage systems;

after access to a first of the created block data storage volumes is provided to a first of the one or more executing programs, responding to one or more data access requests initiated by the first program to the first created block data storage volume by initiating performance of the one or more data access requests on the first created block data storage volume, at least one of the one or more data access requests modifying block data stored on the first block data storage volume; and

after the responding to the received data access requests, providing access of a second executing program to a second created block data storage volume that is based on a snapshot copy of the first created block data storage volume, the providing of the access including responding to one or more other data access requests initiated by the second program to the second created block data storage volume by initiating performance of the one or more other data access requests on the second created block data storage volume.

22. The system of claim 21 wherein the first block data storage volume is stored on a first of the block data storage systems, wherein the first program is executing on a first computing system, wherein the first computing system and the first block data storage system are located at a single first geographical location and separated by the one or more networks, wherein the snapshot copy of the first block data storage volume is stored on one or more remote archival storage systems at a distinct second geographical location and includes the block data stored on the first block data storage volume at a first time, wherein the first block data storage volume and the executing first program are associated with a first user, wherein the second executing program is associated with a distinct second user, and wherein the providing of the access of the second program to the second created block data storage volume includes providing the access in exchange for a fee from the second user to the first user.

23. The system of claim 22 wherein the providing of the block data storage service further includes creating the snapshot copy of the first block data storage volume in response to a request from the first user and creating the second block data storage volume in response to a request from the second user, the creating of the snapshot copy being performed before the responding to the data access requests, such that the created snapshot copy does not contain the modified block data.

24. The system of claim 21 wherein the system includes a first computing system that includes at least one of the one or memories, and wherein the block data storage system manager module includes software instructions for execution by the first computing system using the at least one memory.

25. The system of claim 21 wherein the block data storage system manager module consists of one or more means for providing a block data storage service that uses multiple block data storage systems to store block data storage volumes that are created by users of the block data storage service and accessed over one or more networks on behalf of one or more executing programs associated with the users, the providing of the block data storage service including:

creating one or more block data storage volumes for use by one or more executing programs, each of the block data storage volumes being stored on one of the multiple block data storage systems;

after access to a first of the created block data storage volumes is provided to a first of the one or more executing programs, responding to one or more data access requests initiated by the first program to the first created block data storage volume by initiating performance of the one or more data access requests on the first created block data storage volume, at least one of the one or more data access requests modifying block data stored on the first block data storage volume; and

after the responding to the received data access requests, providing access of a second executing program to a second created block data storage volume that is based on a snapshot copy of the first created block data storage volume, the providing of the access including responding to one or more other data access requests initiated by the second program to the second created block data storage volume by initiating performance of the one or more other data access requests on the second created block data storage volume.