Data streaming and backup systems having multiple concurrent read threads for improved small file performance
First Claim
1. A method of streaming data from a series of files in a file system, the method comprising:
- initiating a series of concurrent file read requests and providing a respective thread for handling data read from each file in the series of files, the respective thread for handling data from said each file transferring the data read from said each file into a series of buffers for said each file, and streaming data from the buffers.
9 Assignments
0 Petitions
Accused Products
Abstract
Permitting multiple concurrent read operations upon a file system enhances small file performance. A respective instance of a thread routine stores the data from each file into a series of buffers for the file. Each instance is identified by a Process Identifier (PID) used for streaming the data from the buffers in the order in which the files are opened instead of the order in which the file data are returned by the read operations. In order to conserve buffer memory, read operations upon a file are suspended once a certain amount of data have been read from the file unless data from the file are being streamed from the buffers.
157 Citations
40 Claims
-
1. A method of streaming data from a series of files in a file system, the method comprising:
-
initiating a series of concurrent file read requests and providing a respective thread for handling data read from each file in the series of files, the respective thread for handling data from said each file transferring the data read from said each file into a series of buffers for said each file, and streaming data from the buffers. - View Dependent Claims (2, 3, 4, 5, 6, 7, 8, 9)
-
-
10. A method of streaming data from a file system, the method comprising:
-
searching the file system to find a series of files in the file system, and upon finding a file to be included in the series of files in the file system, creating a respective instance of a thread routine for handling file data returned from a read operation upon the file and initiating the read operation upon the file so that there are multiple concurrent read operations pending upon multiple files in the series of files in the file system;
the respective instance of the thread routine for handling file data returned from the read operation upon the file determining whether there are additional data in the file and upon finding that there are additional data in the file initiating at least one additional read operation upon the file, and placing all of the data of the file into a respective series of buffers for the file; and
streaming the data from the buffers in the order of the series of files in the file system. - View Dependent Claims (11, 12, 13, 14, 15, 16, 17, 18)
-
- 19. A method of streaming data from a series of files stored in a file server, the series of files being indicated by a list of the files, the method comprising the file server streaming the data from buffers indicated by a write list of buffers, and the file server executing concurrent instances of a thread routine, execution of each instance of the thread routine obtaining ownership of the list of files, and upon obtaining ownership of the list of files, obtaining a file from the list of files and reading data from the file obtained from the list of files and storing the data read from the file obtained from the list of files into buffers on a respective local list of buffers for said each instance of the thread routine, and then obtaining ownership of the write list, and upon obtaining ownership of the write list, linking the local list of buffers to the write list.
-
22. A file server comprising:
-
data storage for storing a file system; and
a processor programmed for streaming data from a series of files in the file system;
wherein the processor is programmed for initiating a series of concurrent file read requests and providing a respective thread for handling data read from each file in the series of files, the respective thread for handling data from said each file transferring the data read from said each file into a series of buffers for said each file, and wherein the processor is programmed for streaming data from the buffers. - View Dependent Claims (23, 24, 25, 26, 27, 28, 29, 30)
-
-
31. A file server comprising:
-
storage containing a file system; and
a processor programmed for searching the file system to find a series of files in the file system, and upon finding a file to be included in the series of files in the file system, initiating a respective instance of a thread routine for handling file data returned from a read operation upon the file and initiating the read operation upon the file so that there are multiple concurrent read operations pending to multiple files in the series of files in the file system;
the thread routine being programmed for handling file data returned from the read operation upon the file by determining whether there are additional data in the file and upon finding that there are additional data in the file initiating at least one additional read operation upon the file, and placing all of the data of the file into a respective series of buffers for the file; and
the processor being programmed for streaming the data from the buffers in the order of the series of files in the file system. - View Dependent Claims (32, 33, 34, 35, 36, 37)
-
-
38. A file server comprising:
-
data storage for storing a file system; and
a processor programmed for streaming data from a series of files in the file system, the series of files being indicated by a list of the files, the processor being programmed with a routine for streaming data from buffers in a write list of buffers, and the processor being programmed for concurrent execution of instances of a thread routine, each instance of the thread routine being executable for obtaining ownership of the list of files, and upon obtaining ownership of the list of files, obtaining a file from the list of files and reading data from the file obtained from the list of files and storing the data read from the file obtained from the list of files into buffers on a respective local list of buffers for said each instance of the thread routine, and then obtaining ownership of the write list, and upon obtaining ownership of the write list, linking the local list of buffers to the write list. - View Dependent Claims (39, 40)
-
Specification