INFORMATION PROCESSING DEVICE, DATA PROCESSING METHOD, AND RECORDING MEDIUM

• US 20170270155A1
• Filed: 08/18/2015
• Published: 09/21/2017
• Est. Priority Date: 08/21/2014
• Status: Active Grant

• Current Assignee: NEC Corporation
• Original Assignee: NEC Corporation
• Inventors: KADOWAKI, Takayuki
• Granted Patent: US 10,657,121 B2
• Time in Patent Office: Days
• CPC Class Codes: G06F 11/14 Error detection or correction of the data by redundancy in operation G06F11/16 takes precedence
• CPC Class Codes: G06F 16/182 Distributed file systems
• CPC Class Codes: G06F 16/2365 Ensuring data consistency and integrity
• CPC Class Codes: G06F 16/27 Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
• CPC Class Codes: G06F 9/30 Arrangements for executing machine instructions, e.g. instruction decode for executing microinstructions G06F9/22

Abstract

Distributed batch processing on an eventually consistent storage system is efficiently performed. A control node includes an execution control unit and a re-execution control unit. The execution control unit causes a processing node to execute predetermined processing that includes reading of data for a key stored in a distributed data store. The re-execution control unit determines, after causing the processing node to execute the predetermined processing, presence or absence of a possibility that inconsistency occurred on data for the key stored in the distributed data store at a time of execution of the predetermined processing, based on a representative value of the data. Then, the re-execution control unit causes, when it is determined that there is a possibility that the inconsistency occurred, the processing node to re-execute the predetermined processing at a time point when resolution of the inconsistency is verified.

Claims

1. An information processing device comprising:

a memory storing instructions; and

one or more processors configured to execute the instructions to;

cause processing means to execute predetermined processing that includes reading of data for a key stored in an eventually consistent storage system; and

determine, after causing the processing means to execute the predetermined processing, presence or absence of a possibility that inconsistency occurred on data for the key stored in the eventually consistent storage system at a time of execution of the predetermined processing, based on a representative value of the data, and cause, when it is determined that there is a possibility that the inconsistency occurred, the processing means to re-execute the predetermined processing at a time point when resolution of the inconsistency is verified.

2. The information processing device according to claim 1, wherein

the predetermined processing includes reading of data for one or more designated keys stored in the eventually consistent storage system, and

the processing means is caused to execute the predetermined processing designating a plurality of keys,

after causing the processing means to execute the predetermined processing, presence or absence of a possibility that inconsistency occurred on data is determined for any of the plurality of keys stored in the eventually consistent storage system at a time of execution of the predetermined processing, and

when it is determined that there is a possibility that the inconsistency occurred, the processing means is caused to re-execute the predetermined processing designating a key having a possibility of the inconsistency at a time point when resolution of the inconsistency is verified.

3. The information processing device according to claim 2, wherein,

in the eventually consistent storage system, data for each of the plurality of keys is duplicated and stored in first and second regions, and

using a representative value of data for each of the plurality of keys and a representative value concerning all data for the plurality of keys which are generated for each of the first region and the second region, presence or absence of a possibility that inconsistency occurred on data for any of the plurality of keys is determined and a key having a possibility of the inconsistency is identified.

4. The information processing device according to claim 3, wherein,

when a representative value concerning all data for the plurality of keys in the first region and a representative value concerning all data for the plurality of keys in the second region, prior to execution of the predetermined processing, are different, it is determined that there is a possibility that inconsistency occurred on data for any of the plurality of keys, and

out of the plurality of keys, a key for which data in the first region and data in the second region have different representative values is identified, as a key having a possibility of the inconsistency.

5. The information processing device according to claim 1, wherein

the one or more processors are further configured to execute the instructions to cause the processing means to execute predetermined pre-processing that includes writing of data into the eventually consistent storage system, and

the processing means is caused to execute the predetermined processing after the predetermined pre-processing is executed by the processing means.

6. A data processing method comprising:

causing processing means to execute predetermined processing that includes reading of data for a key stored in an eventually consistent storage system;

determining, after causing the processing means to execute the predetermined processing, presence or absence of a possibility that inconsistency occurred on data for the key stored in the eventually consistent storage system at a time of execution of the predetermined processing, based on a representative value of the data; and

causing, when it is determined that there is a possibility that the inconsistency occurred, the processing means to re-execute the predetermined processing at a time point when resolution of the inconsistency is verified.

7. The data processing method according to claim 6, wherein

the predetermined processing includes reading of data for one or more designated keys stored in the eventually consistent storage system,

the processing means is caused to execute the predetermined processing designating a plurality of keys,

after causing the processing means to execute the predetermined processing, presence or absence of a possibility that inconsistency occurred on data is determined for any of the plurality of keys stored in the eventually consistent storage system at a time of execution of the predetermined processing, and

when it is determined that there is a possibility that the inconsistency occurred, the processing means is caused to re-execute the predetermined processing designating a key having a possibility of the inconsistency at a time point when resolution of the inconsistency is verified.

8. The data processing method according to claim 7, wherein,

in the eventually consistent storage system, data for each of the plurality of keys is duplicated and stored in first and second regions, and

using a representative value of data for each of the plurality of keys and a representative value concerning all data for the plurality of keys which are generated for each of the first region and the second region, presence or absence of a possibility that inconsistency occurred on data for any of the plurality of keys is determined and a key having a possibility of the inconsistency is identified.

9. The data processing method according to claim 8, wherein,

when a representative value concerning all data for the plurality of keys in the first region and a representative value concerning all data for the plurality of keys in the second region, prior to execution of the predetermined processing, are different, it is determined that there is a possibility that inconsistency occurred on data for any of the plurality of keys, and

out of the plurality of keys, a key for which data in the first region and data in the second region have different representative values is identified, as a key having a possibility of the inconsistency.

10. A non-transitory computer readable storage medium recording thereon a program causing a computer to perform a method comprising:

causing processing means to execute predetermined processing that includes reading of data for a key stored in an eventually consistent storage system;

determining, after causing the processing means to execute the predetermined processing, presence or absence of a possibility that inconsistency occurred on data for the key stored in the eventually consistent storage system at a time of execution of the predetermined processing, based on a representative value of the data; and

causing, when it is determined that there is a possibility that the inconsistency occurred, the processing means to re-execute the predetermined processing at a time point when resolution of the inconsistency is verified.