Data streaming and backup systems having multiple concurrent read threads for improved small file performance
First Claim
1. A computer-implemented method of streaming data from a series of files in a file system in data storage of a file server and through buffers in the file server, the computer-implemented method comprising:
- initiating a series of concurrent file read requests and providing a respective thread for handling data read from the data storage from each file in the series of files, the respective thread for handling the data from said each file transferring the data read from said each file into a series of the buffers for said each file; and
streaming data from the buffers;
wherein the method further includes providing a respective process identifier for identifying each of the threads, and using the process identifiers for ordering the data streamed from the buffers in a certain order by file.
9 Assignments
0 Petitions
Accused Products
Abstract
Permitting multiple concurrent read operations upon a file system enhances small file performance. A respective instance of a thread routine stores the data from each file into a series of buffers for the file. Each instance is identified by a Process Identifier (PID) used for streaming the data from the buffers in the order in which the files are opened instead of the order in which the file data are returned by the read operations. In order to conserve buffer memory, read operations upon a file are suspended once a certain amount of data have been read from the file unless data from the file are being streamed from the buffers.
41 Citations
20 Claims
-
1. A computer-implemented method of streaming data from a series of files in a file system in data storage of a file server and through buffers in the file server, the computer-implemented method comprising:
-
initiating a series of concurrent file read requests and providing a respective thread for handling data read from the data storage from each file in the series of files, the respective thread for handling the data from said each file transferring the data read from said each file into a series of the buffers for said each file; and streaming data from the buffers; wherein the method further includes providing a respective process identifier for identifying each of the threads, and using the process identifiers for ordering the data streamed from the buffers in a certain order by file. - View Dependent Claims (2, 3, 4)
-
-
5. A computer-implemented method of streaming data from a file system in data storage of a file server and thorough buffers in the file server, the computer-implemented method comprising:
-
searching the file system to find a series of files in the file system, and upon finding a file to be included in the series of files in the file system, creating a respective instance of a thread routine for handling file data returned from a read operation upon the file in the data storage and initiating the read operation upon the file in the data storage so that there are multiple concurrent read operations pending upon multiple files in the series of files in the file system; the respective instance of the thread routine for handling file data returned from the read operation upon the file in the data storage determining whether there are additional data in the file in the data storage and upon finding that there are additional data in the file in the data storage initiating at least one additional read operation upon the file in the data storage, and placing all of the data of the file into a respective series of the buffers for the file; and streaming the data from the buffers in the order of the series of files in the file system; wherein the data are streamed from the buffers in the order of the series of files in the file system by assigning a process identifier to the respective instance of the thread routine, the process identifier indicating the order of the file in the series of files in the file system, and using the process identifier to select the respective series of buffers for streaming of the data from the buffers. - View Dependent Claims (6, 7, 8, 9, 10)
-
-
11. A file server comprising:
-
data storage for storing a file system; buffers for buffering data; and a processor programmed for streaming data from the data storage from a series of files in the file system and through the buffers; wherein the processor is programmed for initiating a series of concurrent file read requests and providing a respective thread for handling data read from the data storage from each file in the series of files, the respective thread for handling the data read from said each file transferring the data read from said each file into a series of the buffers for said each file, the processor is programmed for streaming data from the buffers, and the processor is programmed for providing a respective process identifier for identifying each of the threads, and using the process identifiers for ordering the data streamed from the buffers in a certain order by file. - View Dependent Claims (12, 13, 14)
-
-
15. A file server comprising:
-
storage containing a file system; buffers; and a processor programmed for searching the file system to find a series of files in the file system, and upon finding a file to be included in the series of files in the file system, initiating a respective instance of a thread routine for handling file data returned from a read operation upon the file in the storage and initiating the read operation upon the file in the storage so that there are multiple concurrent read operations pending to multiple files in the series of files in the file system; wherein the thread routine is programmed for handling file data returned from the read operation upon the file in the storage by determining whether there are additional data in the file and upon finding that there are additional data in the file initiating at least one additional read operation upon the file in the storage, and placing all of the data of the file into a respective series of the buffers for the file; wherein the processor is programmed for streaming the data from the buffers in the order of the series of files in the file system; and wherein the processor is programmed for streaming the data from the buffers in the order of the series of files in the file system by assigning a process identifier to the respective instance of the thread routine, the process identifier indicating the order of the file in the series of files in the file system, and using the process identifier to select the respective series of the buffers for streaming of the data from the buffers. - View Dependent Claims (16, 17, 18, 19, 20)
-
Specification