File block placement in a distributed file system network

• US 9,633,041 B2
• Filed: 09/26/2013
• Issued: 04/25/2017
• Est. Priority Date: 09/26/2013
• Status: Active Grant

• Current Assignee: Taiwan Semiconductor Manufacturing Company Limited
• Original Assignee: Taiwan Semiconductor Manufacturing Company Limited
• Inventors: Xu, Haibo, Gell, David, Stanwood, Kenneth L.
• Primary Examiner(s): Etienne, Ario
• Assistant Examiner(s): Hurst, Andrea
• Application Number:
  US14/038,123
• Publication Number:
  US 20150088827A1
• Time in Patent Office: 1,307 Days
• CPC Class Codes: G06F 16/1827 Management specifically adapted to NAS management of storage area networks [SAN] G06F3/067
• CPC Class Codes: G06F 16/1844 Management specifically adapted to replicated file systems
• CPC Class Codes: G06F 3/0605 by facilitating the interaction with a user or administrator
• CPC Class Codes: G06F 3/0619 in relation to data integrity, e.g. data losses, bit errors
• CPC Class Codes: G06F 3/0632 by initialisation or re-initialisation of storage systems
• CPC Class Codes: G06F 3/064 Management of blocks
• CPC Class Codes: G06F 3/0643 Management of files
• CPC Class Codes: G06F 3/065 Replication mechanisms
• CPC Class Codes: G06F 3/067 Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
• CPC Class Codes: H04L 67/1097 for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]

Abstract

A method for file block placement in a distributed file system network that includes a plurality of data storage nodes, the method comprising the steps of generating a plurality of file block placement options for a file block, each block placement option being associated with at least one of the data storage nodes, the file block placement options being based on a set of network parameters associated with the distributed file system, determining a cost valuation parameter associated with each of the plurality of file block placement options, and selecting one of the plurality of file block placement options based at least in part on the cost valuation parameter associated with each file block placement option.

Claims

1. A method for file block placement in a distributed file system network that includes a plurality of data storage nodes, the method comprising the steps of:

generating a plurality of file block placement options for a file block, each block placement option being associated with at least one of the data storage nodes, the file block placement options being based on a set of network parameters associated with the distributed file system network, wherein the set of network parameters comprises current network status information;

determining a cost valuation parameter associated with each of the plurality of file block placement options, wherein the cost valuation parameter for each file block placement option is determined based at least in part on a current network congestion level associated with one or more network connections in a network path to the at least one of the data storage nodes associated with the file block placement option; and

selecting one of the plurality of file block placement options based at least in part on the cost valuation parameter associated with each file block placement option.

2. The method of claim 1, further including the step of receiving a file block request to create a file block in the distributed file system network, wherein the generating step is conducted in response to receiving the file block request.

3. The method of claim 2, wherein the file block request includes a replication factor that indicates how many of the plurality of data storage nodes are required to store a replica of the file block.

4. The method of claim 3, wherein, in the generating step, the file block placement options are further based on the replication factor.

5. The method of claim 1, further including the step of:

providing the selected file block placement option to a block file manager node in the distributed file system network.

6. The method of claim 1, wherein the set of network parameters further comprises current network topology information.

7. The method of claim 1, further including the steps of:

generating a set of network control instructions based on the selected file block placement option; and

sending the set of network control instructions to a controller node in communication with the distributed file system network.

8. The method of claim 7, wherein the set of network control instructions include an instruction to adjust a bandwidth reservation for a network traffic flow.

9. The method of claim 7, wherein the set of network control instructions includes an instruction to reconfigure a network connection between two or more network nodes in the distributed file system network.

10. The method of claim 7, wherein the set of network control instructions includes an instruction to require network traffic to be routed according to a specific network path defined by the selected file block placement option.

11. The method of claim 1, further including the step of:

receiving the set of network parameters from a controller node in the distributed file system network.

12. The method of claim 1, wherein the step of generating the plurality of file block placement options is further based on at least one of a current block placement policy, a current service level agreement (SLA) policy and a current quality of service (QoS) policy.

13. The method of claim 1, wherein the cost valuation parameter associated with each file block placement option is determined based at least in part on the evaluation of a cost function for the corresponding file block placement option.

14. The method of claim 13, wherein the cost function is comprised of at least one of a network cost function and a processing cost function.

15. The method of claim 14, wherein the network cost function is based on at least one of a network use factor, a current network utilization factor, an energy usage factor, and a monetary cost factor.

16. The method of claim 1, wherein the selection of the one of the plurality of file block placement options is based at least in part on a minimum cost valuation parameter of the cost valuation parameters associated with the plurality of block placement options.

17. The method of claim 1, wherein the selection of the one of the plurality of file block placement options is based at least in part on selecting a maximum cost valuation parameter of the cost valuation parameters associated with the plurality of block placement options.

18. The method of claim 1, wherein the file block is one of a plurality of file blocks which together comprise a data file, and the plurality of file blocks have a non-uniform block size.

19. A computing device for file block placement in a distributed file system network that includes a plurality of data storage nodes, the computing device comprising:

a memory configured to store data and processing instructions; and

a processor configured to retrieve and execute the processing instructions stored in the memory to cause the processor to perform the steps of;

generating a plurality of file block placement options for a file block, each block placement option being associated with at least one of the data storage nodes, the file block placement options being based on a set of network parameters associated with the distributed file system network, wherein the set of network parameters comprises current network status information;

determining a cost valuation parameter associated with each of the plurality of file block placement options, wherein the cost valuation parameter for each file block placement option is determined based at least in part on a current network congestion level associated with one or more network connections in a network path to the at least one of the data storage nodes associated with the file block placement option; and

selecting one of the plurality of file block placement options based at least in part on the cost valuation parameter associated with each file block placement option.

20. The computing device of claim 19, wherein the processing instructions further cause the processor to perform the step of receiving a file block request to create a file block in the distributed file system network, wherein the generating step is conducted in response to receiving the file block request.

21. The computing device of claim 20, wherein the file block request includes a replication factor that indicates how many of the plurality of data storage nodes are required to store a replica of the file block.

22. The computing device of claim 21, wherein, in the generating step, the file block placement options are further based on the replication factor.

23. The computing device of claim 19, wherein the processing instructions further cause the processor to perform the step of:

providing the selected file block placement option to a block file manager node in the distributed file system network.

24. The computing device of claim 19, wherein the set of network parameters further comprises current network topology information.

25. The computing device of claim 19, wherein the processing instructions further cause the processor to perform the steps of:

generating a set of network control instructions based on the selected file block placement option; and

sending the set of network control instructions to a controller node in communication with the distributed file system network.

26. The computing device of claim 25, wherein the set of network control instructions includes an instruction to adjust a bandwidth reservation for a network traffic flow.

27. The computing device of claim 25, wherein the set of network control instructions includes an instruction to reconfigure a network connection between at least two nodes in the distributed file system network.

28. The computing device of claim 25, wherein the set of network control instructions includes an instruction to require network traffic to be routed according to a specific network path defined by the selected file block placement option.

29. The computing device of claim 19, wherein the processing instructions further cause the processor to perform the step of:

receiving the set of network parameters from a controller node in the distributed file system network.

30. The computing device of claim 19, wherein the step of generating the plurality of file block placement options is further based on at least one of a current block placement policy, a current service level agreement (SLA) policy and a current quality of service (QoS) policy.

31. The computing device of claim 19, wherein the cost valuation parameter associated with each file block placement option is determined based at least in part on the evaluation of a cost function for the corresponding file block placement option.

32. The computing device of claim 31, wherein the cost function is comprised of at least one of a network cost function and a processing cost function.

33. The computing device of claim 32, wherein the network cost function is based on at least one of a network use factor, a current network utilization factor, an energy usage factor, and a monetary cost factor.

34. The computing device of claim 19, wherein the selection of the one of the plurality of file block placement options is based at least in part on a minimum cost valuation parameter of the cost valuation parameters associated with the plurality of block placement options.

35. The computing device of claim 19, wherein the selection of the one of the plurality of file block placement options is based at least in part on selecting a maximum cost valuation parameter of the cost valuation parameters associated with the plurality of block placement options.

36. The computing device of claim 19, wherein the file block is one of a plurality of file blocks which together comprise a data file, and the plurality of file blocks have a non-uniform block size.