Distributed storage system and method

• US 9,609,060 B2
• Filed: 07/31/2012
• Issued: 03/28/2017
• Est. Priority Date: 08/02/2011
• Status: Active Grant

• Current Assignee: NEC Corporation
• Original Assignee: NEC Corporation
• Inventors: Kan, Masaki, Torii, Takashi
• Primary Examiner(s): Jacobs, Lashonda
• Application Number:
  US14/236,666
• Publication Number:
  US 20140173035A1
• Time in Patent Office: 1,701 Days
• US Class Current: 1/1
• CPC Class Codes: G06F 11/2094 Redundant storage or storage space G06F11/2056 takes precedence
• CPC Class Codes: G06F 16/184 implemented as replicated file system
• CPC Class Codes: G06F 3/061 Improving I/O performance
• CPC Class Codes: G06F 3/0617 in relation to availability
• CPC Class Codes: G06F 3/065 Replication mechanisms
• CPC Class Codes: G06F 3/067 Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
• CPC Class Codes: H04L 67/1095 Replication or mirroring of data, e.g. scheduling or transport for data synchronisation between network nodes
• CPC Class Codes: H04L 67/1097 for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]
• CPC Class Codes: H04L 67/568 Storing data temporarily at an intermediate stage, e.g. caching

Abstract

Provided a plurality of data nodes connected in a network, each including a data storage unit. The data node of data replication destination temporarily stores data to be updated in an intermediate data structure, and converts asynchronously with respect to the update request to a target data structure to store the converted data in the data storage unit. Based on access history information stored in an access history recording unit, trigger information, concerning timing for execution of conversion to the target data structure performed asynchronously by the data node is changed.

Claims

1. A distributed storage system comprising:

a plurality of data nodes connected in a network, each data node including a data storage unit, each data node configured for converting a data structure of data to be updated to a target data structure and store the data having data structure converted to the target data structure in the storage unit thereof, the target data structure being suited to read access performed on the data node such that the data is able to be referenced with the target data structure as is by an application that uses the data with the target data structure,

wherein, in response to an update request of the data, the data node corresponding to a replication destination of the data, stores the data to be updated temporarily in an intermediate structure for holding write data, the data node performs conversion of the data to be updated stored in the intermediate structure to the target data structure asynchronously with respect to the update request received, based on a trigger information that prescribes a timer value as from reception of the data to be updated and held in the intermediate structure to the conversion to the target data structure, to store the resulting data structure in the data storage unit thereof;

an access history recording unit configured for storing history of frequency of access to the data node; and

a unit configured for changing the trigger information based on access history information recorded in the access history recording unit.

2. The distributed storage system according to claim 1, wherein the data node corresponding to the replication destination stores the data to be updated in the intermediate structure to return a response,

the data node, in response to an elapse of a time, prescribed by the trigger information, as from reception of the data to be updated, performing conversion of a data structure of the data to be updated held in the intermediate structure asynchronously to the target data structure to store the resulting data structure in the data storage unit thereof.

3. The distributed storage system according to claim 1, comprising

a unit configured for controlling a data node at a data arrangement destination including the replication destination and a target data structure in the data node at the arrangement destination, on a per preset table basis, the data to be updated being converted and stored in the target data structure with a form of the preset table structure adapted to a form in which the application uses the data, in the data node at the data arrangement destination.

4. The distributed storage system according to claim 1, comprising:

a structure information management apparatus including a structure information holding unit that performs storage management of;

data structure management information including, in correspondence with a table identifier to identify data to be stored, a replica identifier to identify a replica;

data structure information to identify a type of data structure corresponding to the replica identifier; and

trigger information that is timer information of a time until the data being converted to a specified target data structure and stored,

the number of information entries each including the table identifier, the replica identifier, the data structure information, and the trigger information corresponding to the number of types of the data structure associated with the table identifier, and

data arrangement specifying information including, in correspondence with a table identifier,

the replica identifier; and

data node information of one or a plurality of data arrangement destinations corresponding to the replica identifier;

a client function implementation unit including

a data access unit that refers to the data structure management information and the data arrangement specifying information to identify an access destination of update processing and read processing; and

a plurality of the data nodes each including the data storage unit, the plurality of the data nodes being connected to the structure information management apparatus and the client function implementation unit, wherein the data node comprises

a data management and processing unit including;

an access receiving and processing unit configured to hold data in an intermediate structure and to return a response to the client function implementation unit, when performing update processing, based on an access request from the client function implementation unit; and

a data structure conversion unit configured to refer to the data structure management information and to perform processing of converting the data to be updated held in the intermediate structure to a target data structure specified by the data structure management information, in response to a specified update trigger.

5. The distributed storage system according to claim 4, comprising:

a change determination unit configured for determining whether or not to change the trigger information of the data structure management information in the structure information holding unit, by using access information recorded in the access history recording unit or other access information obtained by processing the access information,

in the case where the change determination unit determines to change the trigger information of the data structure management information, the change determination unit giving notification to the structure information management apparatus, wherein

the structure information management apparatus comprises

a structure information change unit that on reception of the notification of change of the trigger information from the change determination unit, changes the trigger information of the data structure management information.

6. The distributed storage system according to claim 1, wherein the access information recorded in the access history recording unit includes

frequency information on access to read from the data storage unit and access to write data to the intermediate structure.

7. The distributed storage system according to claim 5, wherein the access receiving and processing unit in the data node comprises:

an access receiving unit; and

an access processing unit, wherein

the data storage unit in the data node comprises

a structure-specific storage unit, wherein

the access receiving unit receives an update request from the client function implementation unit and forwards the update request to a specified data node corresponding to a replica identifier in the data arrangement specifying information, and further records an access request in the access history recording unit, and

the access processing unit performs processing of the update request received, executes update processing with reference to information of the data structure management information, the access processing unit converting data structure of the data to be updated, when the trigger information regarding the data node is zero, based on information of the data structure management information to a target data structure specified in the data structure management information, to store the converted update data in the structure-specific data storage unit,

the access processing unit, when the update trigger information is not zero, writing the data to be updated temporarily in the intermediate structure to make a response of processing-completion,

the access receiving unit, on reception of the completion notification from the access processing unit, or a completion notification from the access processing unit and a completion notification from each data node of a replica destination, returning a response to the client function implementation unit, and wherein

the data structure conversion unit converts data of the intermediate structure to a data structure specified in the data structure management information, and stores the converted data in the structure-specific data storage unit of a conversion destination.

8. The distributed storage system according to claim 1, wherein the plurality of data nodes includes at least two data nodes in which the target data structure is identical,

the two data nodes, performing conversion from the data held in the intermediate structure for holding write data to the target data structure at respective timings that do not overlap, based on the trigger information set,

when one of the two data nodes, converting the data held in the intermediate structure for holding write data to the target data structure, the other data node performing reading of data converted to the target data structure.

15. The distributed storage system according to claim 1, wherein the data node corresponding to the replication destination, in response to an elapse of a time prescribed by the trigger information, converts the data stored in the intermediate structure to the target data structure of a row store form or a column store form on a per table basis.

9. A data replication method for distributed storage including a plurality of data nodes connected in a network, each data node including a data storage unit, the method comprising:

in replication of data corresponding to an update request of data,

the data node corresponding to a replication destination storing data to be updated temporarily in an intermediate structure for holding write data;

the data node performing conversion of the data to be updated in the intermediate structure to an associated target data structure asynchronously with respect to the update request received, based on a trigger information that prescribes a timer value from reception of the data to be updated and held in the intermediate structure to the conversion to the target data structure, to store the resulting data having data structure converted to the target data structure in the data storage unit thereof,

the target data structure being suited to read access performed on the data node such that the data is able to be referenced with the target data structure as is by an application that uses the data with the target data structure; and

changing the trigger information based on access history information of access to the data node.

10. The data replication method according to claim 9, comprising

the data node corresponding to the replication destination storing the data to be updated in the intermediate structure to return a response, and

the data node, in response to an elapse of a time, prescribed by the trigger information, as from reception of the data to be updated, performing conversion of a data structure of the data to be updated held in the intermediate structure asynchronously to the target data structure.

11. The data replication method according to claim 9, comprising

controlling a data node at a data arrangement destination including the replication destination and a target data structure in the data node at the arrangement destination, on a per preset table basis, the data to be updated being converted and stored in the target data structure with a form of the preset table structure adapted to a form in which the application uses the data, in the data node at the data arrangement destination.

12. The data replication method according to claim 9, comprising:

storing in a structure information holding unit of a structure information management apparatus,

data structure management information including, in correspondence with a table identifier to identify data to be stored, a replica identifier to identify a replica;

data structure information to identify a type of data structure corresponding to the replica identifier; and

trigger information that is timer information of a time until the data being converted to a specified target data structure and stored,

the number of information entries each including the table identifier, the replica identifier, the data structure information, and the trigger information corresponding to the number of types of the data structure associated with the table identifier, and

data arrangement specifying information including, in correspondence with a table identifier,

the replica identifier; and

data node information of one or a plurality of data arrangement destinations corresponding to the replica identifier,

a data access unit referring to the data structure management information and the data arrangement specifying information to identify an access destination of update processing and read processing; and

the data node holding the data to be updated in an intermediate structure to return a response to the client function implementation unit, when performing update processing, based on an access request from the client; and

the data node referring to the data structure management information and converting the data to be updated held in the intermediate structure to a data structure specified by the data structure management information, in response to a specified update trigger.

13. The data replication method according to claim 9, wherein the access history information comprises

frequency information on access to read from the data storage unit and access to write data to the intermediate structure.

14. The data replication method according to claim 9, comprising:

preparing at least two identical data nodes in which the target data structure is identical;

the two data nodes, performing conversion from the data held in the intermediate structure for holding write data to the target data structure at respective timings that do not overlap, based on the trigger information set; and

when one of the two data nodes, converting the data held in the intermediate structure for holding write data to the target data structure, the other data node performing reading of data converted to the target data structure.

16. The data replication method according to claim 9, comprising the data node corresponding to the replication destination converting, in response to an elapse of a time prescribed by the trigger information, the data stored in the intermediate structure to the target data structure of a row store form or a column store form on a per table basis.