Object loss reporting in a data storage system
First Claim
1. A system, comprising:
- a data store; and
one or more computing devices, each comprising at least one processor and memory, wherein the one or more computing devices implement a key-durable storage service, and wherein the key-durable storage service is configured to;
receive a request from a client to store a data object in the data store, wherein the request comprises the data object;
generate a key for the data object, wherein the key comprises information usable to determine the location of the data object in the data store;
store the data object and the key in the data store, wherein the data store stores a plurality of data objects and respective keys generated therefor, and wherein the key-durable storage service does not maintain a data structure that maps the plurality of data objects or the respective keys to the locations of the data objects in the data store;
return the key to the client; and
in response to a loss of data in the data store;
determine that the lost data comprises the data object; and
return a notification of lost data to the client, wherein the notification comprises the key.
1 Assignment
0 Petitions
Accused Products
Abstract
In response to receiving a request from a client to store an object, a key-durable storage system may assign the object to a volume in its data store, generate a key for the object (e.g., an opaque identifier that encodes information for locating the object in the data store), store the object on one disk in the assigned volume, store the key redundantly in the assigned volume (e.g., using a replication or erasure coding technique), and may return the key to the client. To retrieve the object, the client may send a request including the key, and the system may return the object to the client. If a disk fails, the system may determine which objects were lost, and may return the corresponding keys to the appropriate clients in a notification. The system may be used to back up a more expensive object-redundant storage system.
-
Citations
30 Claims
-
1. A system, comprising:
-
a data store; and one or more computing devices, each comprising at least one processor and memory, wherein the one or more computing devices implement a key-durable storage service, and wherein the key-durable storage service is configured to; receive a request from a client to store a data object in the data store, wherein the request comprises the data object; generate a key for the data object, wherein the key comprises information usable to determine the location of the data object in the data store; store the data object and the key in the data store, wherein the data store stores a plurality of data objects and respective keys generated therefor, and wherein the key-durable storage service does not maintain a data structure that maps the plurality of data objects or the respective keys to the locations of the data objects in the data store; return the key to the client; and in response to a loss of data in the data store; determine that the lost data comprises the data object; and return a notification of lost data to the client, wherein the notification comprises the key. - View Dependent Claims (2, 3, 4, 5, 6)
-
-
7. A method, comprising:
performing by one or more computers; storing a plurality of data objects in a data store on behalf of one or more clients with a given durability, wherein the data store comprises a plurality of storage nodes; generating, for each of the plurality of data objects, a respective key comprising information usable to determine the location at which the data object is stored in the data store; storing, in the data store, the respective key for each of the plurality of data objects with a higher durability than the given durability; detecting a failure or loss of one of the plurality of storage nodes; determining that one or more of the plurality of data objects were stored on the one of the plurality of storage nodes; and generating a notification that the one or more data objects are no longer stored in the data store, wherein the notification comprises the respective key for each of the one or more data objects. - View Dependent Claims (8, 9, 10, 11, 12, 13, 14)
-
15. A non-transitory, computer-readable storage medium storing program instructions that when executed on one or more computers cause the one or more computers to:
-
implement a key-durable storage service on one or more devices on a service provider network, wherein the key-durable storage service is configured to; store a plurality of data objects in a data store of the service provider network on behalf of one or more storage service clients in accordance with an object durability policy; and store a respective key that was generated by the key-durable storage service for each of the plurality of data objects in the data store, wherein storing the respective keys comprises storing the respective keys in the data store in accordance with a key durability policy that specifies a higher durability than a durability specified by the object durability policy; and expose application programming interfaces (APIs) of the key-durable storage service to one or more clients on a client network of the service provider network, wherein the APIs of the key-durable storage service provide a standard interface between storage operations of the key-durable storage service and the one or more clients, and wherein the APIs of the key-durable storage service include; a put object API for uploading data objects from the client network to the data store; a get object API for downloading stored data objects from the data store to the client network; and a notify API for sending a notification of a lost data from the key-durable storage service to a client that includes a respective key for a data object that is no longer stored in the data store due to a loss of data in the data store. - View Dependent Claims (16, 17, 18, 19, 20, 21, 22)
-
-
23. A system, comprising:
-
a plurality of storage devices configured as a volume-based data store that stores a plurality of data objects and respective keys generated therefor by a key-durable storage service, wherein the key-durable storage service does not maintain a data structure that maps the plurality of data objects or the respective keys to locations at which the data objects or respective keys are stored in the data store; and one or more computing devices, each comprising at least one processor and memory, wherein the one or more computing devices implement the key-durable storage service, and wherein the key-durable storage service is configured to; receive, from one or more storage service clients, a plurality of data objects to be stored in the data store; and for each of the received data objects; assign one of a plurality of volumes of the data store for storing the received data object; store, without redundancy, the received data object on the assigned volume; generate a key for the received data object; and store, with redundancy, the key for the received data object on the assigned volume; and in response to a loss of data on a particular volume in the data store; determine that the lost data comprises a given data object; and generate a notification of the lost data, wherein the notification comprises a respective key that was generated for the given data object by the key-durable storage service when the given data object was stored in the data store. - View Dependent Claims (24, 25, 26, 27, 28, 29, 30)
-
Specification