METHOD AND SYSTEM FOR HEURISTIC THROTTLING FOR DISTRIBUTED FILE SYSTEMS

• US 20100077096A1
• Filed: 09/22/2008
• Published: 03/25/2010
• Est. Priority Date: 09/22/2008
• Status: Active Grant

• Current Assignee: Oracle America, Inc. (Oracle Corporation)
• Original Assignee: Sun Microsystems Incorporated (Oracle Corporation)
• Inventors: Philip, Binu J., Salam, Sudheer Abdul
• Granted Patent: US 8,275,902 B2
• Time in Patent Office: Days
• US Class Current: 709/233
• CPC Class Codes: G06F 16/182 Distributed file systems

Abstract

A method for throttling in a file system. The method includes receiving a request by a client from an application to write data to a server, wherein the application is associated with an application bandwidth and wherein the server is associated with a server bandwidth, queuing a first job to write the data to the server on an I/O queue using an application-to-client throttling mechanism using at least one selected from the group of the application bandwidth and the server bandwidth, and issuing the first job from the I/O queue to the server using a client-to-server throttling mechanism and at least one selected from the group of the application bandwidth and the server bandwidth.

Claims

1. A method for throttling in a file system, comprising:

receiving a request by a client from an application to write data to a server, wherein the application is associated with an application bandwidth and wherein the server is associated with a server bandwidth;

queuing a first job to write the data to the server on an I/O queue using an application-to-client throttling mechanism and at least one selected from a group consisting of the application bandwidth and the server bandwidth; and

issuing the first job from the I/O queue to the server using a client-to-server throttling mechanism and at least one selected from a group consisting of the application bandwidth and the server bandwidth.

2. The method of claim 1, further comprising:

determining a bandwidth using historical data comprising;

obtaining the amount data generated by the application within a time quanta;

obtaining the amount of data acknowledged by the server within the time quanta; and

modifying the bandwidth based on the amount of data generated by the application and the amount of data written to the server within the time quanta; and

setting the application bandwidth and the server bandwidth to the determined bandwidth.

3. The method of claim 1, wherein application-to-client throttling mechanism comprises the functionality to:

determine that the size of the data is greater than the application bandwidth;

determine that client storage is available;

determine that the application bandwidth is greater than zero;

allocate client storage to store the data;

write the data to the allocated client storage; and

decrement the application bandwidth.

4. The method of claim 1, wherein application-to-client throttling mechanism comprises the functionality to:

determine that the size of the data is greater than an application bandwidth;

determine that client storage is not available;

write the data to the client storage when the application bandwidth is available; and

decrement the application bandwidth.

5. The method of claim 1, wherein application-to-client throttling mechanism comprises the functionality to:

determine that the size of the data is not greater than an application bandwidth;

allocate client storage to store the data;

write the data to the client storage when the application bandwidth is available; and

decrement the application bandwidth.

6. The method of claim 1, wherein client-to-server throttling mechanism comprises the functionality to:

determine that the size of the data is not greater than a server bandwidth;

decrement the server bandwidth;

issue the first job to the server;

receive an acknowledgment that the first job has been received; and

increment the server bandwidth.

7. The method of claim 1, wherein client-to-server throttling mechanism comprises the functionality to:

determine that the size of the data is greater than a server bandwidth;

determine that client storage is available; and

postpone issuing the first job until the server bandwidth is at least the size of the data.

8. The method of claim 1, wherein client-to-server throttling mechanism comprises the functionality to:

determine that the size of data is not greater than a server bandwidth;

determining that client storage is available;

modifying the first job to write the data synchronously;

decrement the server bandwidth;

issue the first job to the server;

receive an acknowledgment that the first job has been received; and

receive an acknowledgment that the first job has been completed.

9. The method of claim 1, wherein the client is part of a client cluster.

10. A computer readable storage medium for throttling in a file system, comprising computer readable code, wherein the code is configured to:

receive a request by a client from an application to write data to a server, wherein the application is associated with an application bandwidth and wherein the server is associated with a server bandwidth;

queue a first job to write the data to the server on an I/O queue using an application-to-client throttling mechanism and at least one selected from a group consisting of the application bandwidth and the server bandwidth; and

issue the first job from the I/O queue to the server using a client-to-server throttling mechanism and at least one selected from a group consisting of the application bandwidth and the server bandwidth.

11. The computer readable storage medium of claim 10, further configured to:

determine a bandwidth using historical data comprising;

obtaining the amount data generated by the application within a time quanta;

obtaining the amount of data acknowledged by the server within the time quanta; and

modifying the bandwidth based on the amount of data generated by the application and the amount of data written to the server within the time quanta; and

setting the application bandwidth and the server bandwidth to the determined bandwidth.

12. The computer readable storage medium of claim 11, further configured to:

set the application bandwidth and the server bandwidth to the determined bandwidth.

13. The computer readable storage medium of claim 10, wherein application-to-client throttling mechanism comprises the functionality to:

determine that the size of the data is greater than the application bandwidth;

determine that client storage is available;

determine that the application bandwidth is greater than zero;

allocate client storage to store the data;

write the data to the allocated client storage; and

decrement the application bandwidth.

14. The computer readable storage medium of claim 10, wherein application-to-client throttling mechanism comprises the functionality to:

determine that the size of the data is greater than an application bandwidth;

determine that client storage is not available;

write the data to the client storage when the application bandwidth is available; and

decrement the application bandwidth.

15. The computer readable storage medium of claim 10, wherein application-to-client throttling mechanism comprises the functionality to:

determine that the size of the data is not greater than an application bandwidth;

allocate client storage to store the data;

write the data to the client storage when the application bandwidth is available; and

decrement the application bandwidth.

16. The computer readable storage medium of claim 10, wherein client-to-server throttling mechanism comprises the functionality to:

determine that the size of the data is not greater than a server bandwidth;

decrement the server bandwidth;

issue the first job to the server;

receive an acknowledgment that the first job has been received; and

increment the server bandwidth.

17. The computer readable storage medium of claim 10, wherein client-to-server throttling mechanism comprises the functionality to:

determine that the size of the data is not greater than a server bandwidth;

decrement the server bandwidth;

issue the first job to the server;

receive an acknowledgment that the first job has been received; and

increment the server bandwidth.

18. The computer readable storage medium of claim 10, wherein client-to-server throttling mechanism comprises the functionality to:

determine that the size of data is not greater than a server bandwidth;

determining that client storage is available;

modifying the first job to write the data synchronously;

decrement the server bandwidth;

issue the first job to the server;

receive an acknowledgment that the first job has been received; and

receive an acknowledgment that the first job has been completed.

19. A client, comprising:

a processor;

a storage device comprising instructions executable by the processor, wherein the instructions enable the client to perform a method, the method comprising;

receiving a request by a client from an application to write data to a server, wherein the application is associated with an application bandwidth and wherein the server is associated with a server bandwidth,

queuing a first job to write the data to the server on an I/O queue using an application-to-client throttling mechanism and at least one selected from a group consisting of the application bandwidth and the server bandwidth, and

issuing the first job from the I/O queue to the server using a client-to-server throttling mechanism and at least one selected from a group consisting of the application bandwidth and the server bandwidth.

20. The client of claim 19, wherein the method further comprises:

determining a bandwidth using historical data comprising;

obtaining the amount data generated by the application within a time quanta;

obtaining the amount of data acknowledged by the server within the time quanta; and

modifying the bandwidth based on the amount of data generated by the application and the amount of data written to the server within the time quanta; and

setting the application bandwidth and the server bandwidth to the determined bandwidth.