Data write caching for sequentially written media

• US 9,250,811 B1
• Filed: 08/08/2012
• Issued: 02/02/2016
• Est. Priority Date: 08/08/2012
• Status: Expired due to Fees

• Current Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Original Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Inventors: Patiejunas, Kestutis
• Primary Examiner(s): Bragdon, Reginald
• Assistant Examiner(s): KNIGHT, PAUL M
• Application Number:
  US13/570,005
• Time in Patent Office: 1,273 Days
• US Class Current: 1/1
• CPC Class Codes: G06F 3/061 Improving I/O performance
• CPC Class Codes: G06F 3/0653 Monitoring storage devices or systems
• CPC Class Codes: G06F 3/0659 Command handling arrangements, e.g. command buffers, queues, command scheduling
• CPC Class Codes: G06F 3/0676 Magnetic disk device
• CPC Class Codes: G11B 5/012 Recording on, or reproducing or erasing from, magnetic disks G11B17/00, G11B19/00 take precedence

Abstract

Techniques for implementing a data queuing and/or caching scheme for optimizing data storage are described herein. Data write requests are received and processed by at least queuing the requests and/or associated data for recording upon one or more data storage devices. The order within the queue, as well as the order in which the queued requests are serviced, may, in some embodiments, be optimized. The stored data are verified by determining the position of a write pointer implemented by the one or more data storage devices relative to the contents and/or position of the queued data requests.

Claims

1. A computer-implemented method for storing data, comprising:

under the control of one or more computer systems configured with executable instructions,

selecting a data storage device that is at least;

comprised of data storage media that stores data sequentially; and

capable of responding to a write pointer query by returning a write pointer that indicates a sequential last write position upon the data storage media at which data was most recently written; and

in response to receiving a plurality of write requests, at least;

caching data associated with the write requests in local memory of the computer system;

sequentially recording at least a subset of the cached data upon the data storage media; and

verifying, without flushing or committing the subset of cached data, that the data was written successfully by at least;

querying the data storage device for the write pointer; and

comparing the sequential last write position upon the data storage media indicated by the write pointer with a predetermined sequential destination upon the data storage media for the data being verified.

2. The computer-implemented method of claim 1, wherein the data storage device is configured to use shingled magnetic recording techniques.

3. The computer-implemented method of claim 1, wherein, before selecting the data storage device, adapting the data storage device to be capable of responding to the write pointer query.

4. The computer-implemented method of claim 3, wherein adapting the data storage device to be capable of responding to the write pointer query comprises adapting the data storage device to be capable of responding to the write pointer query by at least updating a firmware of the data storage device.

5. The computer-implemented method of claim 1, wherein the data storage device persists, in a solid state memory of the data storage device, information about the returned write pointer.

6. The computer-implemented method of claim 1, wherein the plurality of write requests is received by a storage system that is at least associated with a plurality of the data storage devices.

7. A computer-implemented method for storing data, comprising:

under the control of one or more computer systems configured with executable instructions,

receiving a plurality of data write requests; and

processing the plurality of data write requests by at least;

queuing the data write requests in local memory of the one or more computer systems;

recording, upon one or more data storage devices that store data sequentially, data associated with at least a subset of the queued data write requests; and

verifying that the data was written successfully by at least;

querying the data storage device upon which the data was written for a write pointer that indicates a final sequentially written location upon the data storage device; and

comparing the write pointer to a known sequential destination for the data being verified.

8. The computer-implemented method of claim 7, wherein at least a subset of the one or more data storage devices record data using shingled magnetic recording techniques.

9. The computer-implemented method of claim 7, wherein the data write requests are received through a programmatic interface exposed to users connected to the computer system.

10. The computer-implemented method of claim 7, wherein the data are verified as successfully written when a result of the comparing the write pointer to the known sequential destination indicates that the known sequential destination sequentially precedes the write pointer on the data storage device.

11. The computer-implemented method of claim 7, wherein the data associated with the queued data write requests is recorded in an analogous order to that of the queued data write requests.

12. The computer-implemented method of claim 7, wherein the data storage device is capable of random reads.

13. A data storage system, comprising:

one or more processors;

data storage media that is capable of storing data in a sequential fashion; and

memory, including instructions executable by the one or more processors to cause the data storage system to at least;

track a terminal write location of the data storage media that reflects a sequentially furthest location to which the data storage media has been written at a given time;

at a time after receiving an external request for the terminal write location of the data storage media, respond with the tracked terminal write location of the data storage media without flushing or committing the data to the data storage media.

14. The data storage system of claim 13, wherein the data storage media is capable of random data reads.

15. The data storage system of claim 13, wherein the data storage media operates using shingled magnetic recording techniques.

16. The data storage system of claim 13, wherein the external request is received from a second data storage system to which at least the data storage media is connected.

17. The data storage system of claim 13, wherein the terminal write location is persisted in the memory.

18. The data storage system of claim 13, wherein the terminal write location is stored upon the data storage media.

19. The data storage system of claim 13, wherein the terminal write location is tracked within the memory and wherein the instructions cause the data storage system to further respond with the terminal write location from the memory.

20. The data storage system of claim 13, wherein the external request is received through a programmatic interface exposed by the data storage system.

21. One or more non-transitory computer-readable storage media having collectively stored thereon executable instructions that, when executed by one or more processors of a computing resource provider'"'"'s computer system, cause the computer system to at least:

service data storage requests by at least;

pipelining the data storage requests by storing them in a local memory of the computer system; and

processing, in a specified order, each of the data storage requests by committing data associated with each of the data storage requests to a data storage device to which the data are written by appending the data after a sequentially final location to which the device was previously written; and

confirm that a given subset of the data storage requests were serviced by at least;

requesting, from the data storage device, a write location bookmark that indicates a current sequential write position on the data storage device; and

comparing the current sequential write position of the requested write location bookmark that indicates a current sequential write position on the data storage device with a predetermined intended destination for data associated with the subset of data storage requests on the data storage device.

22. The non-transitory computer-readable storage media of claim 21, wherein the subset of data storage requests is confirmed as serviced when a result of the comparing of the current sequential write position with the predetermined intended destination indicates that the predetermined intended destination sequentially precedes the current sequential write position on the data storage device.

23. The non-transitory computer-readable storage media of claim 21, wherein the data storage device is a hard disk drive that records data using shingled magnetic recording techniques.

24. The non-transitory computer-readable storage media of claim 21, wherein the data storage device returns the write location bookmark in response to the request.

25. The non-transitory computer-readable storage media of claim 24, wherein the write location bookmark is returned without a decrease in transaction rate due to flushing or committing the data during the processing of the data storage requests.

26. The non-transitory computer-readable storage media of claim 21, wherein the predetermined intended destination is determined by the computer system during the processing of the data storage requests.