Network data storing system and data accessing method

• US 8,953,602 B2
• Filed: 08/27/2007
• Issued: 02/10/2015
• Est. Priority Date: 10/26/2006
• Status: Active Grant

• Current Assignee: Alibaba Group Holding Ltd.
• Original Assignee: Alibaba Group Holding Ltd.
• Inventors: Pan, Lei, Yang, Jinsheng, Tang, Zhengrong
• Primary Examiner(s): Kang, Suk Jin
• Application Number:
  US12/447,036
• Publication Number:
  US 20100061375A1
• Time in Patent Office: 2,724 Days
• Field of Search: None
• US Class Current: 370/392
• CPC Class Codes: G06F 16/1824 implemented using Network-attached Storage [NAS] architecture distributed or networked storage systems G06F3/067; protocols for distributed storage of data in a network H04L67/1097
• CPC Class Codes: H04L 45/00 Routing or path finding of packets in data switching networks routing or path finding in wireless networks H04W40/00
• CPC Class Codes: H04L 67/1097 for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]

Abstract

The present disclosure provides a network data storage system and a network data access method. The disclosed network data storage system includes: a data node, used for storing a data unit; a metadata node, used for storing and managing routing information and providing the routing information according to a data processing request of a client; and a data management node, used for processing the requested data unit in the data node according to a data access request of the client. The disclosed system and method may improve access performance of network data, enhance expansibility of a system, and lower expansion cost.

Claims

1. A system comprising:

a data node that stores a data unit; and

a metadata node that;

receives a request for accessing the data unit;

obtains information of the data unit from the request for accessing the data unit;

obtains an identifier of the data unit based on a mapping relationship between the information of the data unit and the identifier of the data unit, the identifier of the data unit including information of an identifier of the data node and position information of the data unit in the data node;

computes the identifier of the data node at least based on the identifier of the data unit; and

obtains an identifier of a data management node based on a mapping relationship between the identifier of the data node and the identifier of the management node.

2. The system as recited in claim 1, wherein:

the metadata node, the data management node, and the data node are connected to each other in a tree structure;

the data management node processes the data unit in the data node;

the metadata node is a root node of the tree structure and is connected to one or more data management nodes; and

each data management node is connected to one or more data nodes.

3. The system as recited in claim 1, wherein the identifier of the data node is computed based on the identifier of the data unit and the position information of the data unit in the data node.

4. The system as recited in claim 1, wherein the data management node has a data access service and/or a redundancy strategy deployed therein.

5. The system as recited in claim 1, wherein the data unit stored in the data node is a smallest data set for a business application.

6. The system as recited in claim 5, wherein the data unit contains therein multiple files and/or directories.

7. The system as recited in claim 6, wherein the files in the data unit include a data file and/or an index file.

8. The system as recited in claim 1, wherein the system further comprises:

a log management node that stores a log file and provides log management service.

9. The system as recited in claim 1, wherein the system further comprises:

a lock node that stores a lock file and provides lock management service.

10. A method comprising:

receiving a request for accessing a data unit from a client;

obtaining information of the data unit from the request for accessing the data unit;

obtaining an identifier of the data unit based on a mapping relationship between the information of the data unit and the identifier of the data unit, the identifier of the data unit including information of an identifier of the data node and position information of the data unit in the data node;

computing the identifier of the data node at least based on the identifier of the data unit;

according to a routing algorithm of the data unit and based on the identifier of the data unit, computing the position information of the data unit in the data node;

obtaining an identifier of a data management node based on a mapping relationship between the identifier of the data node and the identifier of the management node; and

providing the identifier of the data node and the position information of the data unit in the data node to the data management node.

11. The method as recited in claim 10, further comprising:

providing the identifier of the data management node to the client.

12. The method as recited in claim 10, further comprising:

processing at the data node the data unit using a local file system in the data node according to an operation command.

13. The method as recited in claim 10, wherein the identifier of the data unit is unique.

14. The method as recited in claim 13, wherein the identifier of the data unit is computed through mapping using the identifier of the data node storing the data unit and the position information of the data unit in the data node.

15. The method as recited in claim 10, further comprising:

submitting a copy of a to-be-written block to a log file;

submitting the to-be-written block to a local file system in the data node upon successfully submitting the copy of the to-be-written block to the log file; and

deleting the copy of the block from the log file if successfully submitting the to-be-written block to the local file system, and keeping the copy otherwise.

16. The method as recited in claim 15, further comprising performing data recovery according to the copy of the block recorded in the log file if the system recovers to normal status from an abnormal status.

17. The method as recited in claim 10, wherein the method further comprises:

lock-protecting access to the data unit using a file lock of the data node and/or a file lock of the network file system.

18. A method comprising:

receiving a request for accessing a data unit from a client, the request including information of the data unit; and

returning to the client routing information of a data node which stores the data unit upon receiving the request and routing information of a data management node that processes the data unit in the data node, the returning including;

based on a mapping relationship between the information of the data unit and an identifier of the data unit, obtaining the identifier of the data unit, the identifier of the data unit being computed through mapping by using an identifier of the data node storing the data unit and position information of the data unit in the data node;

computing the identifier of the data node at least based on the identifier of the data unit;

based on a mapping relationship between the identifier of the data node and an identifier of a data management node, obtaining the identifier of the data management node; and

providing the identifier of the data management node to the client.

19. The method as recited in claim 18, wherein:

the metadata node, the data management node, and the data node are connected to each other in a tree structure;

the metadata node is a root node of the tree structure, the root node connecting to one or more data management nodes below; and

each data management node connects to one or more data nodes below.

20. The method as recited in claim 18, wherein:

the data management node separates a storing operation from a computing operation;

the storing operation is executed using a thread pool corresponding to a storing job queue; and

the computing operation is executed using another thread pool corresponding to a computing job queue.