Method, controller, and system for service flow control in object-based storage system

• US 9,804,981 B2
• Filed: 04/18/2017
• Issued: 10/31/2017
• Est. Priority Date: 10/22/2014
• Status: Active Grant

• Current Assignee: Huawei Technologies Co., Ltd. (Huawei Investment & Holding Co., Ltd.)
• Original Assignee: Huawei Technologies Co., Ltd. (Huawei Investment & Holding Co., Ltd.)
• Inventors: Tong, Yanqun
• Primary Examiner(s): Sun, Michael
• Application Number:
  US15/490,118
• Publication Number:
  US 20170220500A1
• Time in Patent Office: 196 Days
• Field of Search: None
• CPC Class Codes: G06F 13/36 for access to common bus or bus system
• CPC Class Codes: G06F 2209/5013 Request control
• CPC Class Codes: G06F 2209/5022 Workload threshold
• CPC Class Codes: G06F 3/0613 in relation to throughput
• CPC Class Codes: G06F 3/0635 by changing the path, e.g. traffic rerouting, path reconfiguration
• CPC Class Codes: G06F 3/0659 Command handling arrangements, e.g. command buffers, queues, command scheduling
• CPC Class Codes: G06F 3/067 Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
• CPC Class Codes: G06F 9/5005 to service a request
• CPC Class Codes: G06F 9/546 Message passing systems or structures, e.g. queues
• CPC Class Codes: H04L 47/26 using explicit feedback to the source, e.g. choke packets
• CPC Class Codes: H04L 47/263 Rate modification at the source after receiving feedback
• CPC Class Codes: H04L 67/1097 for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]

Abstract

A method, a controller, and a system for service flow control in an object-based storage system are disclosed. The method is: receiving, by a controller, a first object IO request; acquiring a processing quantity threshold and a to-be-processed quantity; if the to-be-processed quantity is less than the processing quantity threshold, sending the first object IO request to a storage device client, and updating the to-be-processed quantity; receiving a first response message replied by the storage device client for the first object IO request, where the first response message carries a processing result of the first object IO request; and adjusting the processing quantity threshold according to a received processing result of an object IO request when a preset condition is met. The storage device is not overloaded with object IO requests and can use all resources to effectively, thereby improving performance and a success rate of the object-based storage system.

Claims

1. A service flow control method in an object-based storage system comprising a controller and a storage device client, the method comprising:

receiving, by the controller, a first object IO request;

acquiring, by the controller a processing quantity threshold and a to-be-processed quantity, wherein the processing quantity threshold is a threshold of a quantity of object IO requests to be processed by the object-based storage system, and the to-be-processed quantity is a quantity of to-be-processed object IO requests in the object-based storage system;

when the to-be-processed quantity is less than the processing quantity threshold, sending, by the controller, the first object IO request to the storage device client, and updating the to-be-processed quantity;

receiving, by the controller, a first response message from the storage device client in response to the first object IO request, wherein the first response message carries a processing result of the first object IO request; and

adjusting the processing quantity threshold according to a received processing result of an object IO request when a preset condition is met.

2. The method according to claim 1, further comprising:

when the controller receives a second object IO request, acquiring a current value of the processing quantity threshold and a current value of the to-be-processed quantity; and

when the current value of the to-be-processed quantity is greater than or equal to the current value of the processing quantity threshold, replying with a reject response message or a busy response message.

3. The method according to claim 1, further comprising:

starting a hibernation time period after sending the first object IO request to the storage device client;

when the controller receives a third object IO request, acquiring a current value of the processing quantity threshold and a current value of the to-be-processed quantity; and

when the current value of the to-be-processed quantity is less than the current value of the processing quantity threshold, sending the third object IO request to the storage device client after the hibernation time period ends, and updating the to-be-processed quantity.

4. The method according to claim 1, further comprising:

after the receiving a first response message from the storage device client in response to the first object IO request, updating a sent quantity, wherein the sent quantity is a quantity of object IO requests that are sent by the controller to the storage device client;

when the processing result carried in the first response message indicates a processing success, updating a successfully processed quantity, wherein the successfully processed quantity is a quantity of received object IO requests that are successfully processed by the storage device client; and

when the processing result carried in the first response message indicates a processing failure, retaining a successfully processed quantity unchanged.

5. The method according to claim 1, wherein the sending the first object IO request to the storage device client comprises:

acquiring a split size threshold, wherein the split size threshold is a size threshold of an object IO request to be processed by the object-based storage system; and

when a size of the first object IO request is greater than the split size threshold, splitting, by the controller, the first object IO request into multiple sub-object IO requests, wherein a size of each sub-object IO request is not greater than the split size threshold and carries a same identifier;

sending, by the controller, one of the multiple sub-object IO requests to the storage device client;

starting a hibernation time period;

sending a subsequent one of the multiple sub-object IO requests to the storage device client after the hibernation time period ends; and

repeating operations of the starting the hibernation time period and the sending the subsequent one of the multiple sub-object IO requests, until the multiple sub-object IO requests are sent to the storage device client.

6. The method according to claim 5, further comprising:

receiving a response message from the storage device client after each of the multiple sub-object IO requests was sent, wherein the response message carries a processing result of the each of the multiple sub-object IO requests;

updating a sent quantity, wherein the sent quantity is a quantity of object IO requests that are sent by the controller to the storage device client;

when the processing result indicates a processing success, updating a successfully processed quantity, wherein the successfully processed quantity is a quantity of received object IO requests that are successfully processed by the storage device client; and

when the processing result indicates a final processing failure, retaining a successfully processed quantity unchanged.

7. The method according to claim 4, wherein the adjusting the processing quantity threshold according to the received processing result of the object IO request comprises:

periodically acquiring, by the controller, a current value of the sent quantity and a current value of the successfully processed quantity according to a preset time period;

calculating a processing success rate of an object IO request according to the current value of the sent quantity and the current value of the successfully processed quantity; and

adjusting the processing quantity threshold by using the processing success rate as a first adjustment coefficient.

8. The method according to claim 7, further comprising:

adjusting, according to a second adjustment coefficient, a processing quantity threshold that is obtained after the adjusting of the processing quantity threshold is performed according to the first adjustment coefficient, wherein a value of the second adjustment coefficient is proportional to that of the first adjustment coefficient.

9. The method according to claim 7, further comprising:

setting, by the controller, an upper limit and a lower limit of the processing quantity threshold; and

modifying, according to the upper limit or the lower limit of the processing quantity threshold, the adjusted processing quantity threshold, so that a value of the adjusted processing quantity threshold is between the upper limit and the lower limit of the processing quantity threshold.

10. The method according to claim 4, further comprising:

periodically acquiring, by the controller, a current value of the sent quantity and a current value of the successfully processed quantity according to a preset time period;

calculating a processing success rate of an object IO request according to the current value of the sent quantity and the current value of the successfully processed quantity; and

adjusting a hibernation time by using the processing success rate as a first adjustment coefficient.

11. The method according to claim 10, further comprising:

adjusting the hibernation time according to a third adjustment coefficient and the first adjustment coefficient, wherein a value of the third adjustment coefficient is inversely proportional to that of the first adjustment coefficient.

12. The method according to claim 10, further comprising:

setting, by the controller, an upper limit and a lower limit of the hibernation time; and

modifying, according to the upper limit or the lower limit of the hibernation time, the adjusted hibernation time, so that a value of the adjusted hibernation time is between the upper limit and the lower limit of the hibernation time.

13. The method according to claim 1, further comprising:

sending, by the controller, a response message to a requesting party that sends an object IO request, wherein the response message carries a final processing result of the object IO request; and

updating the to-be-processed quantity.

14. A service flow control method in an object-based storage system comprising a controller and a storage device client, the method comprising:

receiving, by the controller, a first object IO request;

acquiring, by the controller, a split size threshold, wherein the split size threshold is a size threshold of an object IO request to be processed by the object-based storage system;

when a size of the first object IO request is greater than the split size threshold, splitting, by the controller, the first object IO request into multiple sub-object IO requests, wherein a size of each sub-object IO request is not greater than the split size threshold and carries a same identifier;

sending a first sub-object IO request to the storage device client;

starting a hibernation time period;

receiving a response message replied by the storage device client for the first sub-object IO request, wherein the response message carries a processing result of the first sub-object IO request; and

adjusting the hibernation time period according to a received processing result of an object IO request when a preset condition is met.

15. The method according to claim 14, wherein after the starting the hibernation time period, the method further comprises:

after the hibernation time period ends, sending a second sub-object IO request to the storage device client.

16. The method according to claim 14, further comprising:

after the receiving a first response message from the storage device client in response to the first sub-object IO request, updating a sent quantity, wherein the sent quantity is a quantity of object IO requests that are sent by the controller to the storage device client;

when the processing result of a first object IO request that is carried in the first response message indicates a processing success, updating a successfully processed quantity, wherein the successfully processed quantity is a quantity of received object IO requests that are successfully processed by the storage device client; and

when the processing result of the first object IO request that is carried in the first response message indicates a final processing failure, retaining a successfully processed quantity unchanged.

17. The method according to claim 14, wherein the adjusting the hibernation time period comprises:

periodically acquiring, by the controller, a current value of the sent quantity and a current value of the successfully processed quantity according to a preset time period;

calculating a processing success rate of an object IO request according to the current value of the sent quantity and the current value of the successfully processed quantity; and

adjusting a hibernation time by using the processing success rate as a first adjustment coefficient.

18. The method according to claim 17, further comprising:

adjusting the hibernation time obtained after adjustment is performed according to the first adjustment coefficient and a third adjustment coefficient, wherein a value of the third adjustment coefficient is inversely proportional to that of the first adjustment coefficient.

19. The method according to claim 17, further comprising:

setting, by the controller, an upper limit and a lower limit of the hibernation time; and

modifying, according to the upper limit or the lower limit of the hibernation time, the adjusted hibernation time, so that a value of the adjusted hibernation time is between the upper limit and the lower limit of the hibernation time.

20. A device for implementing service flow control in an object-based storage system, comprising:

a controller, wherein the controller is configured to

receive a first object IO request;

acquire a processing quantity threshold and a to-be-processed quantity wherein the processing quantity threshold is a threshold of a quantity of object IO requests to be processed by the object-based storage system, and the to-be-processed quantity is a quantity of to-be-processed object IO requests in the object-based storage system;

when the to-be-processed quantity is less than the processing quantity threshold, send the first object IO request to the storage device client and update the to-be-processed quantity;

receive a first response message from the storage device client in response to the first object IO request, wherein the first response message carries a processing result of the first object IO request; and

adjust the processing quantity threshold according to a received processing result of an object IO request when a preset condition is met.

21. The device according to claim 20, wherein the controller is further configured to

update a sent quantity after receiving the first response message from the storage device client in response to the first object IO request, wherein the sent quantity is a quantity of object IO requests that are sent by the controller to the storage device client;

and

when the processing result carried in the first response message indicates a processing success, update a successfully processed quantity, wherein the successfully processed quantity is a quantity of received object IO requests that are successfully processed by the storage device client.

22. The device according to claim 20 wherein the controller is further configured to

split the first object IO request into multiple sub-object IO requests when the first object IO request is greater than a split size threshold, wherein a size of each sub-object IO request is not greater than the split size threshold and carries a same identifier, and the split size threshold is a size threshold of an object IO request to be processed by the object-based storage system;

send one of the multiple sub-object IO requests to the storage device client; and

start a hibernation time period, and send a subsequent one of the multiple sub-object IO requests to the storage device client after the hibernation time period ends.

23. The device according to claim 22, wherein the controller is further configured to

receive a response message from the storage device client after each of the multiple sub-object IO requests was sent, wherein the response message carries a processing result of the each of the multiple sub-object IO requests;

update a sent quantity, wherein the sent quantity is a quantity of object IO requests that are sent to the storage device client; and

when the processing result indicates a processing success, update a successfully processed quantity, wherein the successfully processed quantity is a quantity of received object IO requests that are successfully processed by the storage device client.

24. The device according to claim 21, wherein the controller is further configured to:

periodically acquire a current value of the sent quantity and a current value of the successfully processed quantity according to a preset time period;

calculate a processing success rate of an object IO request according to the current value of the sent quantity and the current value of the successfully processed quantity; and

adjust the processing quantity threshold by using the processing success rate as a first adjustment coefficient.

25. The device according to claim 24, wherein the controller is further configured to

adjust, according to a second adjustment coefficient, a processing quantity threshold that is obtained after the adjusting of the processing quantity threshold is performed according to the first adjustment coefficient, wherein a value of the second adjustment coefficient is proportional to that of the first adjustment coefficient.

26. The device according to claim 24, wherein the controller is further configured to

set an upper limit and a lower limit of the processing quantity threshold;

modify, according to the upper limit or the lower limit of the processing quantity threshold, the adjusted processing quantity threshold, so that a value of the adjusted processing quantity threshold is between the upper limit and the lower limit of the processing quantity threshold.

27. The device according to claim 21, wherein the controller is further configured to

periodically acquire a current value of the sent quantity and a current value of the successfully processed quantity according to a preset time period;

calculate a processing success rate of an object IO request according to the current value of the sent quantity and the current value of the successfully processed quantity; and

adjust a hibernation time by using the processing success rate as a first adjustment coefficient.

28. The device according to claim 27, wherein the controller is further configured to

adjust the hibernation time according to a third adjustment coefficient and the first adjustment coefficient, wherein a value of the third adjustment coefficient is inversely proportional to that of the first adjustment coefficient.

29. The device according to claim 28, wherein:

the controller is further configured to

set an upper limit and a lower limit of the hibernation time; and

modify, according to the upper limit or the lower limit of the hibernation time, the adjusted hibernation time, so that a value of the adjusted hibernation time is between the upper limit and the lower limit of the hibernation time.

30. The device according to claim 20, wherein the controller is further configured to send a response message to a requesting party that sends an object IO request, wherein the response message carries a final processing result of the object IO request;

and update the to-be-processed quantity.