Speculative copying of data from main buffer cache to solid-state secondary cache of a storage server

• US 9,037,800 B2
• Filed: 06/14/2013
• Issued: 05/19/2015
• Est. Priority Date: 03/18/2008
• Status: Active Grant

• Current Assignee: NetApp, Inc.
• Original Assignee: NetApp, Inc.
• Inventors: Ellard, Daniel J.
• Primary Examiner(s): Bansal, Gurtej
• Application Number:
  US13/918,804
• Publication Number:
  US 20140181374A1
• Time in Patent Office: 704 Days
• Field of Search: 711/118, 711/120, 711/122, 711/142
• US Class Current: 711/122
• CPC Class Codes: G06F 12/0246 in block erasable memory, e.g. flash memory
• CPC Class Codes: G06F 12/0804 with main memory updating G06F12/0806 takes precedence
• CPC Class Codes: G06F 12/0808 with cache invalidating means G06F12/0815 takes precedence
• CPC Class Codes: G06F 12/0868 Data transfer between cache memory and other subsystems, e.g. storage devices or host systems
• CPC Class Codes: G06F 2212/214 Solid state disk

Abstract

A network storage server includes a main buffer cache to buffer writes requested by clients before committing them to primary persistent storage. The server further uses a secondary cache, implemented as low-cost, solid-state memory, such as flash memory, to store data evicted from the main buffer cache or data read from the primary persistent storage. To prevent bursts of writes to the secondary cache, data is copied from the main buffer cache to the secondary cache speculatively, before there is a need to evict data from the main buffer cache. Data can be copied to the secondary cache as soon as the data is marked as clean in the main buffer cache. Data can be written to secondary cache at a substantially constant rate, which can be at or close to the maximum write rate of the secondary cache.

Claims

1. A method comprising:

writing data to a main buffer cache of a storage server in response to a write request;

allocating a set of solid-state memory for use by the storage server as a secondary cache; and

copying data from the main buffer cache to the secondary cache before there is a need to evict data from the main buffer cache, to reduce latency in accessing the secondary cache, the copying including copying the data at a write rate, which is determined as a function of a first write rate of the secondary cache and a data eviction rate of the main buffer cache.

2. The method as recited in claim 1, wherein the secondary cache comprises flash memory.

3. The method as recited in claim 1, further comprising:

copying data stored in the main buffer cache to a persistent storage subsystem of the storage server; and

marking data as clean in response to the data being copied from the main buffer cache to the persistent storage subsystem, wherein copying the data to the secondary cache is performed in response to the data being marked as clean.

4. The method as recited in claim 3, further comprising evicting data marked as clean from the main buffer cache.

5. The method as recited in claim 1, wherein said copying data from the main buffer cache to the secondary cache comprises maintaining the write rate substantially constant when writing data to the secondary cache, as long as the main buffer cache contains data to be written to the secondary cache.

6. The method as recited in claim 5, wherein said substantially constant write rate is the maximum write rate of the secondary cache.

7. A method of operating a storage server, the method comprising:

receiving read requests and write requests at the storage server from a client over a network;

responding to the read requests and write requests, wherein responding to the write requests includes writing data blocks to a volatile main buffer cache of the storage server in response to the write requests;

at a consistency point, writing the data blocks in the main buffer cache to a persistent storage subsystem of the storage server and thereafter marking the data blocks written to the persistent storage subsystem as clean;

concurrently with responding to read requests and write requests from the client, copying data blocks marked as clean to a secondary cache implemented in solid-state memory, before there is a need to evict data from the main buffer cache, the copying including copying the data at a write rate, which is determined as a function of a first write rate of the secondary cache and a data eviction rate of the main buffer cache; and

evicting from the main buffer cache data blocks that have been copied to the secondary cache, wherein said evicting is not a response to the copying of data blocks to the secondary cache.

8. The method as recited in claim 7, wherein said writing data blocks to the main buffer cache and said writing data blocks to the persistent storage subsystem are performed by a log-structured storage manager.

9. The method as recited in claim 7, wherein the secondary cache is implemented as flash memory.

10. The method as recited in claim 7, wherein said copying data blocks to the secondary cache is performed responsive to the data blocks being marked as clean.

11. The method as recited in claim 7, wherein said copying comprises maintaining the write rate substantially constant when writing data to the secondary cache.

12. The method as recited in claim 11, wherein said copying comprises sending a stream of writes to the secondary cache at the maximum write rate of the secondary cache.

13. A storage server system comprising:

a main buffer cache; and

a processor configured to service read and write requests received over a network from a set of clients, including writing data to the main buffer cache and thereafter copying data from the main buffer cache to a persistent storage subsystem prior to evicting the data from the main buffer cache, and further to mark the data as clean after the data has been copied to the persistent storage subsystem, wherein the processor further is configured to copy data stored in the main buffer cache to a secondary cache implemented in solid-state memory, including to copy data marked as clean to the secondary cache prior to a need to evict data from the main buffer cache, wherein the processor is further configured to copy the data from the main buffer cache to the secondary cache at a write rate, which is determined as a function of a first write rate of the secondary cache and a data eviction rate of the main buffer cache.

14. The storage server as recited in claim 13, wherein the secondary cache comprises flash memory.

15. The storage server as recited in claim 13, wherein the processor further is configured to evict data from the main buffer cache only after the data has been marked as clean.

16. The storage server as recited in claim 13, wherein the processor is configured to reduce the likelihood of bursts of writes to the secondary cache by maintaining the write rate substantially constant when writing data to the secondary cache, as long as the main buffer cache contains data to be written to the secondary cache.

17. The storage server as recited in claim 13, wherein the processor is configured to write the data marked as clean to the secondary cache at the maximum write rate of the secondary cache.

18. A network storage system comprising:

a network interface through which to communicate with a set of clients over a network;

a storage interface through which to communicate with a persistent storage subsystem of the storage system;

a main buffer cache, to cache data read from and data to be written to, the persistent storage subsystem;

a secondary cache implemented in non-volatile solid-state memory, to cache data initially stored in the main buffer cache; and

a storage manager to service read and write requests from the clients, including writing data to the main buffer cache and thereafter copying data from the main buffer cache to the persistent storage subsystem at a consistency point, wherein the storage manager further is to mark the data as clean when the data has been copied to the persistent storage subsystem, wherein the storage manager further is to reduce the likelihood of bursts of writes to the secondary cache by copying data marked as clean in the main buffer cache to the secondary cache before there is a need to evict the data from the main buffer cache, wherein the storage manager is further configured to copy the data from the main buffer cache to the secondary cache at a write rate, which is determined as a function of a first write rate of the secondary cache and a data eviction rate of the main buffer cache.

19. The network storage system as recited in claim 18, wherein the secondary cache comprises flash memory.

20. The network storage system as recited in claim 18, wherein the storage manager further is to evict from the main buffer cache data marked as clean.

21. The network storage system as recited in claim 20, wherein eviction of said data from the main buffer cache is not in response to the data being marked as clean.

22. The network storage system as recited in claim 18, wherein the storage manager maintains the write rate substantially constant when writing data to the secondary cache, as long as the main buffer cache contains data to be written to the secondary cache.

23. The network storage system as recited in claim 22, wherein the storage manager writes data to the secondary cache at approximately the maximum write rate of the non-volatile solid-state memory.