Adaptive file readahead based on multiple factors

• US 20050154825A1
• Filed: 01/08/2004
• Published: 07/14/2005
• Est. Priority Date: 01/08/2004
• Status: Active Grant

• Current Assignee: Network Appliance Incorporated (NetApp, Inc.)
• Original Assignee: Network Appliance Incorporated (NetApp, Inc.)
• Inventors: Fair, Robert L.
• Granted Patent: US 7,631,148 B2
• Time in Patent Office: Days
• US Class Current: 711/113
• CPC Class Codes: G06F 16/172 Caching, prefetching or hoarding of files
• CPC Class Codes: G06F 3/0611 in relation to response time
• CPC Class Codes: G06F 3/0643 Management of files
• CPC Class Codes: G06F 3/067 Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]

Abstract

A storage system is provided that implements a file system configured to optimize the amount of readahead data retrieved for each read stream managed by the file system. The file system relies on various factors to adaptively select an optimized readahead size for each read stream. Such factors may include the number of read requests processed in the read stream, an amount of client-requested data requested in the read stream, a read-access style associated with the read stream, and so forth. The file system is also configured to minimize cache pollution by adaptively selecting when readahead operations are performed for each read stream and determining how long each read stream'"'"'s retrieved data is retained in memory.

Claims

1. A method for a storage operating system implemented in a storage system to optimize the amount of readahead data retrieved for a read stream established in a data container stored in the storage system, the method comprising:

receiving a client read request at the storage system, the client read request indicating client-requested data for the storage operating system to retrieve from the data container containing the read stream;

determining whether the storage operating system is permitted to retrieve readahead data from the data container in response to the received client read request;

if it is determined that the storage operating system is permitted to retrieve readahead data from the data container, performing the steps of;

(i) selecting an amount of readahead data to retrieve from the data container based on one or more factors; and

(ii) retrieving the selected amount of readahead data from the data container.

2. The method of claim 1, wherein the data container is a file, directory, vdisk or lun.

3. The method of claim 1, wherein the storage operating system is determined to be permitted to retrieve readahead data from the data container when the client-requested data extends the read stream past a predetermined next readahead value.

4. The method of claim 3, wherein the predetermined next readahead value is stored in a readset data structure associated with the read stream.

5. The method of claim 3, wherein the predetermined next readahead value is updated based on a percentage of the selected amount of readahead data.

6. The method of claim 1, wherein a read-access style associated with the data container is one of the one or more factors used to select the amount of readahead data.

7. The method of claim 6, wherein the selected amount of readahead data equals zero if the read-access style corresponds to a random read-access style.

8. The method of claim 1, wherein a number of client read requests processed in the read stream is one of the one or more factors used to select the amount of readahead data.

9. The method of claim 8, wherein the number of client read requests processed in the read stream is stored as a count value in a readset data structure associated with the read stream.

10. The method of claim 1, wherein the amount of client-requested data is one of the one or more factors used to select the amount of readahead data.

11. The method of claim 10, wherein the selected amount of readahead data is set equal to a predetermined upper limit for large amounts of client-requested data.

12. The method of claim 1, wherein the selected amount of readahead data is doubled if the number of client read requests processed in the read stream is greater than a first threshold value.

13. The method of claim 1, wherein the client-requested data is identified as read-once data when either (i) the number of client read requests processed in the read stream is greater than a second threshold value or (ii) a set of metadata associated with the read stream indicates that the client-requested data is read-once data.

14. The method of claim 1, wherein the selected amount of readahead data is stored in one or more buffers enqueued on a flush queue, the flush queue being configured to reuse buffers after a predetermined period of time.

15. The method of claim 14, wherein the predetermined period of time equals two seconds.

16. An apparatus configured to implement a storage operating system that optimizes the amount of readahead data retrieved for a read stream established in a data container stored in the apparatus, the apparatus comprising:

means for receiving a client read request, the client read request indicating client-requested data for the storage operating system to retrieve from the data container containing the read stream;

means for determining whether the storage operating system is permitted to retrieve readahead data from the data container in response to the received client read request;

means for selecting an amount of readahead data to retrieve from the data container based on one or more factors; and

means for retrieving the selected amount of readahead data from the data container.

17. The apparatus of claim 16, wherein the data container is a file, directory, vdisk or lun.

18. The apparatus of claim 16, wherein the storage operating system is determined to be permitted to retrieve readahead data from the data container when the client-requested data extends the read stream past a predetermined next readahead value.

19. The apparatus of claim 18, further comprising means for updating the predetermined next readahead value based on a percentage of the selected amount of readahead data.

20. The apparatus of claim 16, wherein the one or more factors used to select the amount of readahead data includes at least one of:

(i) the amount of client-requested data,

(ii) a number of client read requests processed in the read stream, and

(iii) a read-access style associated with the data container.

21. The apparatus of claim 16, wherein the selected amount of readahead data is doubled if the number of client read requests processed in the read stream is greater than a first threshold value.

22. A storage system configured to optimize the amount of readahead data retrieved for a read stream established in a data container stored in the storage system, the storage system comprising:

a network adapter for receiving a client read request, the client read request indicating client-requested data to retrieve from the data container containing the read stream; and

a memory configured to store instructions for implementing a storage operating system that performs the steps of;

determining whether the storage operating system is permitted to retrieve readahead data from the data container in response to the received client read request, and

if it is determined that the storage operating system is permitted to retrieve readahead data from the data container;

(i) selecting an amount of readahead data to retrieve from the data container based on one or more factors; and

(ii) retrieving the selected amount of readahead data from the data container.

23. The storage system of claim 22, wherein the data container is a file, directory, vdisk or lun.

24. The storage system of claim 22, wherein the storage operating system is determined to be permitted to retrieve readahead data from the data container when the client-requested data extends the read stream past a predetermined next readahead value.

25. The storage system of claim 24, wherein the predetermined next readahead value is updated based on a percentage of the selected amount of readahead data.

26. The storage system of claim 22, wherein the one or more factors used to select the amount of readahead data includes at least one of:

(i) the amount of client-requested data,

(ii) a number of client read requests processed in the read stream, and

(iii) a read-access style associated with the data container.

27. The storage system of claim 22, wherein the selected amount of readahead data is doubled if the number of client read requests processed in the read stream is greater than a first threshold value.

28. A computer-readable media comprising instructions for execution in a processor for the practice of a method for a storage operating system implemented in a storage system to optimize the amount of readahead data retrieved for a read stream established in a data container stored in the storage system, the method comprising:

receiving a client read request at the storage system, the client read request indicating client-requested data for the storage operating system to retrieve from the data container containing the read stream;

determining whether the storage operating system is permitted to retrieve readahead data from the data container in response to the received client read request;

if it is determined that the storage operating system is permitted to retrieve readahead data from the data container, performing the steps of;

(i) selecting an amount of readahead data to retrieve from the data container based on one or more factors; and

(ii) retrieving the selected amount of readahead data from the data container.

29. The computer-readable media of claim 28, wherein the data container is a file, directory, vdisk or lun.