Providing executing programs with access to stored block data of others

• US 8,725,967 B2
• Filed: 08/08/2008
• Issued: 05/13/2014
• Est. Priority Date: 08/08/2008
• Status: Active Grant

• Current Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Original Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Inventors: DeSantis, Peter N., Jorgensen, Atle Normann, Garman, Matthew S., Certain, Tate Andrew, Paterson-Jones, Roland
• Primary Examiner(s): Rigol, Yaima
• Application Number:
  US12/188,952
• Publication Number:
  US 20100037031A1
• Time in Patent Office: 2,104 Days
• Field of Search: 711161-163, 711170-173
• US Class Current: 711/162
• CPC Class Codes: G06F 11/1461 Backup scheduling policy
• CPC Class Codes: G06F 11/1464 for networked environments
• CPC Class Codes: G06F 11/2071 using a plurality of controllers
• CPC Class Codes: G06F 2201/84 Using snapshots, i.e. a logical point-in-time copy of the data
• CPC Class Codes: G06F 3/0619 in relation to data integrity, e.g. data losses, bit errors
• CPC Class Codes: G06F 3/0622 in relation to access
• CPC Class Codes: G06F 3/065 Replication mechanisms
• CPC Class Codes: G06F 3/067 Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]

Abstract

Techniques are described for managing access of executing programs to non-local block data storage. In some situations, a block data storage service uses multiple server storage systems to reliably store copies of network-accessible block data storage volumes that may be used by programs executing on other physical computing systems, and snapshot copies of some volumes may also be stored (e.g., on remote archival storage systems). A group of multiple server block data storage systems that store block data volumes may in some situations be co-located at a data center, and programs that use volumes stored there may execute on other computing systems at that data center, while the archival storage systems may be located outside the data center. The snapshot copies of volumes may be used in various ways, including to allow users to obtain their own copies of other users'"'"' volumes (e.g., for a fee).

Claims

1. A computer-implemented method comprising:

receiving, by one or more configured computing systems of a block data storage service, indications of one or more data access requests initiated by a first executing program on a first computing system to a first logical local block storage device of the first computing system, the first logical block storage system being associated with a first block data storage volume that is non-local to the first computing system, the executing first program and the first block data storage volume being operated on behalf of a first user;

responding, by the one or more configured computing systems, to the one or more data access requests by performing the data access requests on the first block data storage volume, the first block data storage volume and having a primary copy stored on a second computing system and having a mirror copy stored on a third computing system, the performing of the data access requests causing one or more modifications to block data stored on the first block data storage volume in such a manner that the primary and mirror copies of the first block data storage volume each store the same block data;

at a first time after the performing of the data access requests, creating a first snapshot copy of the first block data storage volume on one or more remote data storage systems distinct from the first, second and third computing systems, the first snapshot copy including a copy of the block data stored on the first block data storage volume at the first time; and

after creating the first snapshot copy,

determining to grant a request received from a distinct second user for a second executing program associated with the second user to have access to a second block data storage volume that includes the block data contained in the created first snapshot copy, the second user not being associated with the first block data storage volume prior to the receiving of the request from the second user;

creating the second block data storage volume based on the created first snapshot copy; and

providing access to the second block data storage volume for the second executing program, the providing of the access including performing one or more other data access requests received from the second program on the created second block data storage volume.

2. The method of claim 1 wherein the creating of the first snapshot copy of the first block data storage volume is performed in response to instructions from the first user, and wherein the determining to grant the request is based at least in part on the first user providing authorization to provide access to other users to the block data contained in the created first snapshot copy.

3. The method of claim 2 wherein the first computing system hosts multiple virtual machines, and wherein the indications of the one or more data access requests received by the one or more configured computing systems are sent by a first node manager module executing as part of a virtual machine monitor for the first computing system.

4. A computer-implemented method for providing access of executing programs to block data storage functionality, the method comprising:

receiving, by a computing system of a storage service, an indication of a data access request that is initiated by a first program executing on a first computing system to modify block data stored on a non-local block data storage volume, the first block data storage volume being provided by a distinct second data storage system of the storage service, the first block data storage volume and the first program being associated with a first user;

responding, by the computing system, to the received indication of the data access request by causing one or more modifications to the block data stored on the first block data storage volume;

after responding to the received indication of the data access request, creating, by the computing system, a first snapshot copy of the first block data storage volume on one or more other data storage systems, the created first snapshot copy containing the block data stored on the first block data storage volume at a first time, the block data contained in the first snapshot copy including the one or more modifications; and

providing access to the created first snapshot copy for a fee to one or more other second users that are distinct from the first user, such that each of the one or more other second users who provides the fee receives access for one or more executing programs to a distinct block data storage volume that includes the block data contained in the created first snapshot copy, wherein the other second users are not associated with the first block data storage volume prior to the providing of the access.

5. The method of claim 4 wherein, for one of the one or more other second users, the providing of the access to the first snapshot copy to the one other second user includes attaching a second block data storage volume to a second computing system that is executing a second program associated with the one other second user, the second block data storage volume being distinct from the first block data storage volume and being stored on a third block data storage system and including the block data contained in the created first snapshot copy, and subsequently interacting with the third block data storage system on behalf of the second program to initiate performance on the second block data storage volume of other data access requests initiated by the second program.

6. The method of claim 5 wherein providing access to the first snapshot copy to the one other second user is performed in response to a request from the one other second user for the access and further includes creating the second block data storage volume for the one other second user in response to the request.

7. The method of claim 4 wherein the first snapshot copy is created in response to instructions from the first user, the instructions further indicating criteria regarding whether the other second users receive access to the first snapshot copy.

8. The method of claim 7 wherein the indicated criteria include information about at least one indicated other second user who is authorized to receive access to the first snapshot copy, and wherein providing access to the first snapshot copy to each of the one or more other second users includes determining that the other second user is among the at least one indicated authorized other second users.

9. The method of claim 8 wherein providing access to the first snapshot copy to each of the one or more other second users includes receiving a request from the other second user for the access to the first snapshot copy, the received request from the other second user including information for use in determining whether the other second user is among the at least one indicated authorized other second users.

10. The method of claim 4 wherein providing access to the first snapshot copy to each of the one or more other second users includes obtaining the fee from the other second user, and wherein the first user receives at least some of the obtained fees from the other second users.

11. The method of claim 4 further comprising, before the receiving of the indication of the data access request, attaching the first block data storage volume to the first computing system for use by the first executing program by associating a logical local block storage device of the first computing system with the first block data storage volume provided by the second data storage system, and wherein the received indication of the data access request is generated in response to the first executing program interacting with the logical local block storage device.

12. The method of claim 4 wherein the first computing system and the second data storage system are a subset of a plurality of computing systems co-located at a first geographical location, and wherein the one or more data storage systems on which the first snapshot copy is created are located at a second distinct geographical location.

13. The method of claim 12 wherein the first computing system is configured to host multiple virtual machines that are each able to execute at least one program, wherein the first executing program is a virtual machine image being executed by the first computing system, and wherein receiving the indication of the data access request and responding to the received indication are performed by a virtual machine monitor running on the first computing system.

14. A non-transitory computer-readable medium having stored contents that configure one or more computing systems to:

provide, by the one or more configured computing systems, access of a first program executing on a first computing system to a non-local block data storage volume associated with an account of a first user, the block data storage volume being provided by a second block data storage system that is separated from the first computing system by one or more networks;

respond, by the one or more configured computing systems, to one or more received indications of one or more data access requests initiated by the first program for the block data storage volume by performing the data access requests on the block data storage volume on the second block data storage system;

after responding to the received indications of the data access requests, provide, by the one or more configured computing systems, paid access of a second executing program for a distinct second user to a stored first copy of the block data storage volume, the second user not being associated with the block data storage volume prior to the providing of the paid access; and

respond, by the one or more configured computing systems, to one or more received indications of one or more data access requests initiated by the second program to the block data storage volume first copy by performing the data access requests initiated by the second program on the block data storage volume first copy.

15. The non-transitory computer-readable medium of claim 14 wherein the stored contents further configure the one or more computing systems to:

after responding to the received indications of the data access requests, create a first snapshot copy of the block data storage volume on one or more data storage systems remote from the second block data storage system, the created first snapshot copy containing block data stored on the block data storage volume at a first time, and wherein the stored first copy of the block data storage volume is a block data storage volume copy that is based on the created first snapshot copy.

16. The non-transitory computer-readable medium of claim 15 wherein the first computing system and the second block data storage system are located at a single first geographical location, wherein the one or more data storage systems are remote archival storage systems at a distinct second geographical location, and wherein providing access for the first executing program to the block data storage volume is performed by a node manager module of the block data storage service that is associated with the first computing system and that manages access of the first executing program to the one or more networks separating the first computing system and the second block data storage system.

17. The non-transitory computer-readable medium of claim 14 wherein the first computing system and the second block data storage system are located at a single first geographical location, wherein the second program executes on a second computing system at a distinct second geographical location, and wherein the stored first copy of the block data storage volume is a distinct second block data storage volume that is a copy of the block data storage volume created at the second geographical location.

18. The non-transitory computer-readable medium of claim 14 wherein the second program executes on a second computing system at a common geographical location with the first computing system and the second block data storage system, wherein performing the data access requests on the block data storage volume on the second block data storage system includes performing the data access requests on a primary copy of the block data storage volume that is stored on the second block data storage system, and wherein the stored first copy of the block data storage volume is the primary copy of the block data storage volume stored on the second block data storage system.

19. The non-transitory computer-readable medium of claim 14 wherein the stored contents are instructions that when executed program the one or more computing systems.

20. A system configured to provide access of executing programs to block data storage functionality, comprising:

one or more processors; and

a block data storage system manager module configured to, wherein executed by at least one of the one or more processors, provide a block data storage service that uses multiple block data storage systems to store block data storage volumes that are created by users of the block data storage service and accessed over one or more networks on behalf of one or more executing programs associated with the users, the providing of the block data storage service including;

creating one or more block data storage volumes for use by one or more executing programs, each of the block data storage volumes being stored on one of the multiple block data storage systems;

after access to a first of the created block data storage volumes is provided to a first of the one or more executing programs, responding to one or more data access requests initiated by the first program to the first created block data storage volume by initiating performance of the one or more data access requests on the first created block data storage volume, the first program and the first created block data storage volume being associated with a first user, at least one of the one or more data access requests modifying block data stored on the first block data storage volume; and

after the responding to the received data access requests, providing access of a second executing program associated with a distinct second user to a second created block data storage volume that is based on a snapshot copy of the first created block data storage volume, the providing of the access including responding to one or more other data access requests initiated by the second program to the second created block data storage volume by initiating performance of the one or more other data access requests on the second created block data storage volume, wherein the distinct second user is not associated with the first block data storage volume prior to the initiating of the one or more other data access requests.

21. The system of claim 20 wherein the first block data storage volume is stored on a first of the block data storage systems, wherein the first program is executing on a first computing system, wherein the first computing system and the first block data storage system are located at a single first geographical location and separated by the one or more networks, wherein the snapshot copy of the first block data storage volume is stored on one or more remote archival storage systems at a distinct second geographical location and includes the block data stored on the first block data storage volume at a first time, and wherein the providing of the access of the second program to the second created block data storage volume includes providing the access in exchange for a fee from the second user.

22. The system of claim 21 wherein the providing of the block data storage service further includes creating the snapshot copy of the first block data storage volume in response to a request from the first user and creating the second block data storage volume in response to a request from the second user, the creating of the snapshot copy being performed before the responding to the data access requests, such that the created snapshot copy does not contain the modified block data.

23. The system of claim 20 wherein the system includes a first computing system that includes the at least one processor, and wherein the block data storage system manager module includes software instructions for execution by the first computing system using the at least one processor.

24. The system of claim 20 wherein the block data storage system manager module consists of one or more means for performing the providing of the block data storage service.