Prefetching and multithreading for improved file read performance

• US 7,206,795 B2
• Filed: 12/22/2003
• Issued: 04/17/2007
• Est. Priority Date: 12/22/2003
• Status: Active Grant

• Current Assignee: Emc IP Holding Company LLC (Dell Technologies Inc.)
• Original Assignee: EMC Corporation (Dell Technologies Inc.)
• Inventors: Bono, Jean-Pierre
• Primary Examiner(s): Mizrahi; Diane D.
• Application Number:
  US10/744,536
• Publication Number:
  US 20050138091A1
• Time in Patent Office: 1,212 Days
• Field of Search: 707 1- 10, 707100-1041, 707200-206, 717/158, 718/100, 709/214
• US Class Current: 1/1
• CPC Class Codes: G06F 16/10 File systems; File servers
• CPC Class Codes: Y10S 707/99942 Manipulating data structure, e.g. compression, compaction, compilation
• CPC Class Codes: Y10S 707/99954 Version management

Abstract

A problem of latency during a read of a file is solved by activating multiple concurrent read threads and also by prefetching blocks of file data from storage to a file system cache. Prefetching ensures that by the time that a client or application needs more file data, the file data will already have been read from storage and written to the cache as a background operation. For example, a file server responds to a request to read an extent of file data consisting of eight consecutive file blocks by activating concurrent block read threads for reading the file blocks and by prefetching from the storage to the cache eight more consecutive blocks of data from the file.

Claims

1. In a file server having a cache memory and storage containing a file, a method of responding to a request for reading an extent of data from the file, the extent of data spanning a first series of multiple blocks of file data, the file also spanning a second series of multiple blocks of file data following the first series of multiple blocks of file data, the method comprising:

(a) activating a respective read thread for reading each of the multiple blocks of file data in the first series of multiple blocks of file data so that the read threads are concurrently reading the multiple blocks of file data in the first series of multiple blocks of file data; and

(b) prefetching the second series of multiple blocks of file data by reading from the storage and writing to the cache memory each block of file data in the second series of multiple blocks of file data that is not found in the cache memory.

2. The method as claimed in claim 1, wherein one of the read threads is a main thread executed in response to the request for reading the extent of data from the file, and wherein the main thread reads one of the blocks of file data in the first series of multiple blocks of file data, and the main thread activates the other read threads for reading the other blocks of file data in the first series of multiple blocks of file data.

3. The method as claimed in claim 2, wherein the main thread activates a prefetch thread for prefetching the second series of multiple blocks of file data.

4. The method as claimed in claim 3, which includes the prefetch thread activating read threads for prefetching at least some of the blocks of file data in the second series of multiple blocks of file data.

5. The method as claimed in claim 2, which includes suspending the main thread until completion of the reading of all of the blocks of file data in the first series of multiple blocks of file data.

6. The method as claimed in claim 1, which includes executing a main thread for responding to the request for reading the extent of data from the file, the main thread activating a number of the read threads for reading at least some of the blocks of file data in the first series of multiple blocks of file data, and the main thread activating a prefetch thread for prefetching the second series of multiple blocks of file data.

7. The method as claimed in claim 6, which includes suspending the main thread after the main thread activates the prefetch thread, and activating the suspended main thread once said number of the read threads have finished reading said at least some of the blocks of file data in the first series of multiple blocks of file data.

8. The method as claimed in claim 6, which includes the prefetch thread activating read threads for prefetching at least some of the blocks of file data in the second series of multiple blocks of file data.

9. The method as claimed in claim 1, wherein the blocks of file data have a block size of eight kilobytes, and the first series of multiple blocks of file data consists of eight blocks of file data, and the second series of multiple blocks of file data consists of eight blocks of file data.

10. The method as claimed in claim 1, wherein the request for reading the extent of data is from a client or application of the file server, and by the time that the client or application of the file server needs more data of the file, the prefetching has already read said more data of the file from the storage and has written said more data of the file to the cache memory, whereby said more data of the file is more readily available to the client or application of the file server.

11. In a file server having a cache memory and storage containing a file, a method of responding to a request for reading an extent of data from the file, the extent of data spanning a first series of multiple blocks of file data, the file also spanning a second series of multiple blocks of file data following the first series of multiple blocks of file data, the method comprising:

executing a main thread for responding to the request for reading the extent of data from the file,

the main thread activating block read threads for reading at least some of the blocks of file data in the first series of multiple blocks of file data so that the block read threads are concurrently reading said at least some of the blocks of file data in the first series of multiple blocks of file data, and then

the main thread activating a prefetch thread for prefetching the second series of multiple blocks of file data by reading from the storage and writing to the cache memory each block of file data in the second series of multiple blocks of file data that is not found in the cache memory; and

then

suspending the main thread, and then

activating the main thread once the block read threads have finished reading said at least some of the blocks of file data in the first series of multiple blocks of file data.

12. The method as claimed in claim 11, wherein the main thread reads one of the blocks of file data in the first series of multiple blocks of file data, and the block read threads activated by the main thread read the other blocks of file data in the first series of multiple blocks of file data.

13. The method as claimed in claim 11, which includes the prefetch thread activating block read threads for prefetching at least some of the blocks of file data in the second series of multiple blocks of file data.

14. The method as claimed in claim 13, wherein the main thread and the prefetch thread share a pool of block read threads.

15. The method as claimed in claim 11, wherein the blocks of file data have a block size of eight kilobytes, and the first series of multiple blocks of file data consists of eight blocks of file data, and the second series of multiple blocks of file data consists of eight blocks of file data.

16. The method as claimed in claim 11, wherein the request for reading the extent of data is from a client or application of the file server, and when the client or application of the file server needs to read more data of the file, the prefetching has already read said more data of the file from the storage and has written said more data of the file to the cache memory, whereby said more data of the file is more readily available to the client or application of the file server.

17. In a file server having a file system cache memory and storage containing a file, a method of responding to a request for reading an extent of data from the file, the extent of data including a first series of multiple blocks of file data, the file including a second series of multiple blocks of file data immediately following the first series of multiple blocks of file data, the method comprising:

activating concurrent read operations, each of the concurrent read operations reading a respective one of the multiple blocks of file data, and prefetching from the storage to the file system cache memory the second series of multiple blocks of file data immediately following the first series of multiple blocks of file data.

18. The method as claimed in claim 17, which includes executing a main thread for responding to the request for reading the extent of data from the file, the main thread performing one of the concurrent read operations, and the main thread activating block read threads for performing the other concurrent read operations.

19. The method as claimed in claim 18, wherein the main thread activates a prefetch thread for prefetching from the storage to the file system cache memory the second series of multiple blocks of file data.

20. The method as claimed in claim 17, wherein the prefetch thread prefetches one of the blocks in the second series of multiple blocks of file data, and the prefetch thread activates block read threads for prefetching the other blocks in the second series of multiple blocks of file data.

21. The method as claimed in claim 17, which includes executing a main thread for responding to the request for reading the extent of data from the file, the main thread activating block read threads for performing at least some of the concurrent read operations, and the main thread activating a prefetch thread for prefetching from the storage to the file system cache memory the second series of multiple blocks of file data immediately following the first series of multiple blocks of file data.

22. The method as claimed in claim 21, which includes suspending the main thread after the main thread activates the prefetch thread, and activating the suspended main thread once the block read threads have finished said at least some of the concurrent read operations.

23. The method as claimed in claim 17, wherein the request for reading the extent of data is from a client or application of the file server, and when the client or application of the file server needs to read more data of the file, the prefetching has already read said more data of the file from the storage and has written said more data of the file to the cache memory, whereby said more data of the file is more readily available to the client or application of the file server.

24. A file server comprising a cache memory and storage containing a file, the file server being programmed for responding to a request for reading an extent of data from the file, the extent of data spanning a first series of multiple blocks of file data, the file also spanning a second series of multiple blocks of file data following the first series of multiple blocks of file data, the file server being programmed for responding to the request for reading the extent of data from the file by:

(a) activating a respective read thread for reading each of the multiple blocks of file data in the first series of multiple blocks of file data so that the read threads are concurrently reading the multiple blocks of file data in the first series of multiple blocks of file data; and

(b) prefetching the second series of multiple blocks of file data by reading from the storage and writing to the cache memory each block of file data in the second series of multiple blocks of file data that is not found in the cache memory.

25. The file server as claimed in claim 24, wherein one of the read threads is a main thread executed in response to the request for reading the extent of data from the file, and wherein the main thread is programmed for reading one of the blocks of file data in the first series of multiple blocks of file data, and for activating the other read threads for reading the other blocks of file data in the first series of multiple blocks of file data.

26. The file server as claimed in claim 25, wherein the main thread is programmed for activating a prefetch thread for prefetching the second series of multiple blocks of file data.

27. The file server as claimed in claim 26, wherein the prefetch thread is programmed for activating read threads for prefetching at least some of the blocks of file data in the second series of multiple blocks of file data.

28. The file server as claimed in claim 25, which is programmed for suspending the main thread until completion of the reading of all of the blocks of file data in the first series of multiple blocks of file data.

29. The file server as claimed in claim 24, which is programmed for executing a main thread for responding to the request for reading the extent of data from the file, and the main thread is programmed for activating a number of the read threads for reading at least some of the blocks of file data in the first series of multiple blocks of file data, and for activating a prefetch thread for prefetching the second series of multiple blocks of file data.

30. The file server as claimed in claim 29, which is programmed for suspending the main thread after the main thread activates the prefetch thread, and for activating the suspended main thread once said number of the read threads have finished reading said at least some of the blocks of file data in the first series of multiple blocks of file data.

31. The file server as claimed in claim 29, wherein the prefetch thread is programmed for activating read threads for prefetching at least some of the blocks of file data in the second series of multiple blocks of file data.

32. The file server as claimed in claim 24, wherein the blocks of file data have a block size of eight kilobytes, and the first series of multiple blocks of file data consists of eight blocks of file data, and the second series of multiple blocks of file data consists of eight blocks of file data.

33. The file server as claimed in claim 24, wherein the file server is capable of responding to the request for reading the extent of data from a client or application of the file server so that when the client or application of the file server needs to read more data of the file, the prefetching has already read said more data of the file from the storage and has written said more data of the file to the cache memory, whereby said more data of the file is more readily available to the client or application of the file server.

34. A file server comprising a cache memory and storage containing a file, the file server being programmed for responding to a request for reading an extent of data from the file, the extent of data spanning a first series of multiple blocks of file data, the file also spanning a second series of multiple blocks of file data following the first series of multiple blocks of file data, the file server being programmed for responding to the request for reading the extent of data from the file by executing a main thread for responding to the request for reading the extent of data from the file,

the main thread being programmed for activating block read threads for reading at least some of the blocks of file data in the first series of multiple blocks of file data so that the block read threads are concurrently reading said at least some of the blocks of file data in the first series of multiple blocks of file data, and then activating a prefetch thread for prefetching the second series of multiple blocks of file data, the prefetching of the second series of multiple blocks of file data reading from the storage and writing to the cache memory each block of file data in the second series of multiple blocks of file data that is not found in the cache memory, and then suspending the main thread, and then activating the main thread once the block read threads have finished reading the number of blocks of file data in the first series of multiple blocks of file data.

35. The file server as claimed in claim 34, wherein the main thread is programmed for reading one of the blocks of file data in the first series of multiple blocks of file data, and wherein the block read threads activated by the main thread are programmed for reading the other blocks of file data in the first series of multiple blocks of file data.

36. The file server as claimed in claim 34, wherein the prefetch thread is programmed for activating block read threads for prefetching at least some of the blocks of file data in the second series of multiple blocks of file data.

37. The file server as claimed in claim 36, wherein the main thread and the prefetch thread share a pool of block read threads.

38. The file server as claimed in claim 34, wherein the blocks of file data have a block size of eight kilobytes, and the first series of multiple blocks of file data consists of eight blocks of file data, and the second series of multiple blocks of file data consists of eight blocks of file data.

39. The file server as claimed in claim 34, wherein the file server is capable of responding to the request for reading the extent of data from a client or application of the file server so that when the client or application of the file server needs more data of the file, the prefetching has already read said more data of the file from the storage and has written said more data of the file to the cache memory, whereby said more data of the file is more readily available to the client or application of the file server.

40. A file server comprising a file system cache memory and storage containing a file, the file server being programmed for responding to a request for reading an extent of data from the file, the extent of data spanning a first series of multiple blocks of file data, the file also spanning a second series of multiple blocks of file data following the first series of multiple blocks of file data, the file server being programmed for responding to the request for reading the extent of data from the file by activating concurrent read operations, each of the concurrent read operations reading a respective one of the multiple blocks of file data, and prefetching from the storage to the file system cache memory a second series of multiple blocks of file data immediately following the first series of multiple blocks of file data.

41. The file server as claimed in claim 40, which is programmed for executing a main thread for responding to the request for reading the extent of data from the file, and wherein the main thread is programmed for performing one of the concurrent read operations, and activating block read threads for performing the other concurrent read operations.

42. The file server as claimed in claim 41, wherein the main thread is programmed for activating a prefetch thread for prefetching from the storage to the file system cache memory the second series of multiple blocks of file data.

43. The file server as claimed in claim 42, wherein the prefetch thread is programmed for prefetching one of the blocks in the second series of multiple blocks of file data, and for activating block read threads for prefetching the other blocks in the second series of multiple blocks of file data.

44. The file server as claimed in claim 40, which is programmed for executing a main thread for responding to the request for reading the extent of data from the file, the main thread being programmed for activating block read threads for performing at least some of the concurrent read operations, and the main thread is programmed for activating a prefetch thread for prefetching from the storage to the file system cache memory the second series of multiple blocks of file data immediately following the first series of multiple blocks of file data.

45. The file server as claimed in claim 44, which is programmed for suspending the main thread after the main thread activates the prefetch thread, and activating the suspended main thread once the block read threads have finished said at least some of the concurrent read operations.

46. The file server as claimed in claim 40, wherein the file server is capable of responding to the request for reading the extent of data from a client or application of the file server so that when the client or application of the file server needs more data of the file, the prefetching has already read said more data of the file from the storage and has written said more data of the file to the cache memory, whereby said more data of the file is more readily available to the client or application of the file server.