Network Data Storing System and Data Accessing Method

• US 20100061375A1
• Filed: 08/27/2007
• Published: 03/11/2010
• Est. Priority Date: 10/26/2006
• Status: Active Grant

• Current Assignee: Alibaba Group Holding Ltd.
• Original Assignee: Alibaba Group Holding Ltd.
• Inventors: Yang, Jinsheng, Tang, Zhengrong, Pan, Lei
• Granted Patent: US 8,953,602 B2
• Time in Patent Office: Days
• US Class Current: 370/395.700
• CPC Class Codes: G06F 16/1824 implemented using Network-attached Storage [NAS] architecture distributed or networked storage systems G06F3/067; protocols for distributed storage of data in a network H04L67/1097
• CPC Class Codes: H04L 45/00 Routing or path finding of packets in data switching networks routing or path finding in wireless networks H04W40/00
• CPC Class Codes: H04L 67/1097 for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]

Abstract

The present disclosure provides a network data storage system and a network data access method. The disclosed network data storage system includes: a data node, used for storing a data unit; a metadata node, used for storing and managing routing information and providing the routing information according to a data processing request of a client; and a data management node, used for processing the requested data unit in the data node according to a data access request of the client. The disclosed system and method may improve access performance of network data, enhance expansibility of a system, and lower expansion cost.

Claims

1. A system for storing network data, characterized in that the system comprises:

a data node, used for storing a data unit;

a metadata node, used for storing and managing routing information and providing the routing information according to a data processing request of a client; and

a data management node, used for processing the requested data unit in the data node according to a data access request of the client.

2. The system as recited in claim 1, wherein:

the metadata node, the data management node, and the data node are connected to each other in a tree structure;

the metadata node is a root node of the tree structure, the root node connecting to one or more data management nodes below; and

each data management node connects to one or more data nodes below.

3. The system as recited in claim 1, characterized in that the routing information stored in the metadata node comprises:

routing information for routing from the metadata node to the data management node; and

routing information for routing from the data management node to the data node.

4. The system as recited in claim 3, characterized in that the metadata node further stores a routing algorithm of the data unit, the routing algorithm being for computing an identifier of the data node which stores the data unit and position information of the data unit in the data node.

5. The system as recited in claim 1, characterized in that the data management node has a data access service and/or a redundancy strategy deployed therein.

6. The system as recited in claim 1, characterized in that the data unit stored in the data node is the smallest data set for a business application.

7. The system as recited in claim 6, characterized in that the data unit contains therein multiple files and/or directories.

8. The system as recited in claim 7, characterized in that the files in the data unit include a data file and/or an index file.

9. The system as recited in claim 1, characterized in that the system further comprises:

a log management node, used for storing a log file and providing log management service.

10. The system as recited in claim 1, characterized in that the system further comprises:

a lock node, used for storing a lock file and providing lock management service.

11. A method for accessing network data, characterized in that the method comprises:

in a client, sending a request for accessing a data unit to a metadata node, and receiving routing information of a data management node from the metadata node;

in the client, sending the request for accessing the data unit to the data management node according to the routing information of the data management node; and

in the data management node, obtaining routing information of a data node which stores the data unit from the metadata node upon receiving the request, and processing the data unit in the data node according to the routing information of the data node and operation requested by the client.

12. The method as recited in claim 11, characterized in that, after receiving the request for accessing the data unit from the client, the metadata node sends the routing information of the data management node to the client using a process comprising:

in the metadata node, obtaining information of the data unit from the request for accessing the data unit, and obtaining an identifier of the data unit based on a mapping relationship between the information of the data unit and the identifier of the data unit;

computing an identifier of the data node which stores the data unit from the identifier of the data unit based on a routing algorithm of the data unit; and

obtaining an identifier of the data management node based on a mapping relationship between the identifier of the data node and the identifier of the data management node, and providing the identifier of the data management node to the client.

13. The method as recited in claim 11, characterized in that, the metadata node provides the routing information of the data node to the data management node using a procedure comprising:

obtaining an identifier of the data unit from a request sent from the data management node;

according to a routing algorithm of the data unit and based on the identifier of the data unit, computing an identifier of the data node which stores the data unit and position information of the data unit in the data node; and

providing the identifier of the data node and the position information to the data management node.

14. The method as recited in claim 11, characterized in that the data management node separates storing operation from computing operation.

15. The method as recited in claim 14, characterized in that the storing operation is executed using a thread pool corresponding to a storing job queue, and the computing operation is executed using another thread pool corresponding to a computing job queue.

16. The method as recited in claim 11, characterized in that the method further comprises:

upon receiving a data operation command from the data management node, processing at the data node the data unit using a local file system in the data node according to the operation command.

17. The method as recited in claim 11, characterized in that the data unit has a unique identifier.

18. The method as recited in claim 17, characterized in that the identifier of the data unit is computed through mapping using the identifier of the data node storing the data unit and the position information of the data unit in the data node.

19. The method as recited in claim 11, characterized in that, processing the data unit in the data node further comprises:

submitting a copy of a to-be-written block to a log file;

submitting the to-be-written block to the local file system in the data node upon successfully submitting the copy of the to-be-written block to the log file; and

deleting the copy of the block from the log file if successfully submitting the to-be-written block to the file system, and keeping the copy otherwise.

20. The method as recited in claim 19, characterized in that the method performs data recovery according to the copy of the block recorded in the log file if the system recovers to normal status from an abnormal status.

21. The method as recited in claim 11, characterized in that the method further comprises:

lock-protecting access to the data unit using a file lock of the data node and/or a file lock of the network file system.