DATA CONSISTENCY MANAGEMENT

• US 20140149400A1
• Filed: 11/26/2012
• Published: 05/29/2014
• Est. Priority Date: 11/26/2012
• Status: Active Grant

• Current Assignee: Accenture Global Services Limited (Accenture PLC)
• Original Assignee: Accenture Global Services Limited (Accenture PLC)
• Inventors: Fu, Chen, Venugeethan, Sugi, Taneja, Kunal
• Granted Patent: US 9,111,012 B2
• Time in Patent Office: Days
• US Class Current: 707/723
• CPC Class Codes: G06F 16/215 Improving data quality; Data cleansing, e.g. de-duplication, removing invalid entries or correcting typographical errors
• CPC Class Codes: G06F 16/221 Column-oriented storage; Management thereof
• CPC Class Codes: G06F 16/2365 Ensuring data consistency and integrity
• CPC Class Codes: G06F 16/24564 Applying rules; Deductive queries
• CPC Class Codes: G06F 16/24578 using ranking
• CPC Class Codes: G06F 16/27 Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
• CPC Class Codes: G06F 16/278 Data partitioning, e.g. horizontal or vertical partitioning
• CPC Class Codes: G06F 16/90335 Query processing
• CPC Class Codes: G06F 16/951 Indexing; Web crawling techniques

Abstract

A data consistency management system may include a memory storing machine readable instructions to receive a query, and determine a suitability of the query for processing by a NoSQL data store, or a RDBMS. The memory may further include machine readable instructions to rank data tables based on a combination of read queries and query patterns suitable for the NoSQL data store. Based on the ranking, the memory may further include machine readable instructions to determine data tables that are to be managed by the NoSQL data store, or by the RDBMS, determine whether the query is for a data table managed by the NoSQL data store, and based on a determination that the query is for a data table managed by the NoSQL data store, translate the query to NoSQL API calls for using the NoSQL data store to respond to the query.

Claims

1. A data consistency management system comprising:

a memory storing machine readable instructions to;

receive a query;

determine a suitability of the query for processing by a not-only structured query language (NoSQL) data store, or a relational database management system (RDBMS);

rank data tables based on a combination of read queries for the data tables and query patterns suitable for the NoSQL data store for the data tables, at least one of the data tables containing information for responding to the query;

based on the ranking, determine data tables from the ranked data tables that are to be managed by the NoSQL data store, or by the RDBMS;

determine whether the query is for at least one data table managed by the NoSQL data store; and

based on a determination that the query is for the at least one data table managed by the NoSQL data store, translate the query to NoSQL application programming interface (API) calls for using the NoSQL data store to respond to the query; and

a processor to execute the machine readable instructions.

2. The data consistency management system of claim 1, further comprising machine readable instructions to:

based on a determination that the query is not for the at least one data table managed by the NoSQL data store, forward the query to the RDBMS.

3. The data consistency management system of claim 1, wherein the machine readable instructions to determine the suitability of the query for processing by the NoSQL data store, or the RDBMS further comprise:

determining whether the query is a select query that selects data from a data table via a primary key of the data table; and

determining whether the query is a select query that aggregates a single column of a data table.

4. The data consistency management system of claim 1, wherein the machine readable instructions to determine the suitability of the query for processing by the NoSQL data store, or the RDBMS further comprise:

determining whether the query is a select query that selects data from a data table via a primary key of the data table;

determining if a predetermined percentage of queries to the data table are select queries that select data from the data table via the primary key of the data table; and

based on a determination that a predetermined percentage of queries to the data table are select queries that select data from the data table via the primary key of the data table, using a key-value store for the NoSQL data store for processing the query.

5. The data consistency management system of claim 1, wherein the machine readable instructions to determine the suitability of the query for processing by the NoSQL data store, or the RDBMS further comprise:

determining whether the query is a select query that aggregates a single column of a data table;

determining if a predetermined percentage of queries to the data table are select queries that aggregate the single column of the data table; and

based on a determination that a predetermined percentage of queries to the data table are select queries that aggregate the single column of the data table, using a column store for the NoSQL data store for processing the query.

6. The data consistency management system of claim 1, wherein the machine readable instructions to rank the data tables further comprise:

ranking the data tables based on a linear combination of a percentage of the read queries for the data tables and a percentage of the query patterns suitable for the NoSQL data store for the data tables.

7. The data consistency management system of claim 1, wherein the machine readable instructions to rank the data tables further comprise:

ranking a data table based on a linear combination of a percentage of the read queries for the data table, a percentage of queries of the data table that matches a key-select pattern, and a percentage of queries of the data table that matches an aggregation pattern and aggregate over data in a column of the data table.

8. The data consistency management system of claim 1, wherein the machine readable instructions to determine the data tables from the ranked data tables that are to be managed by the NoSQL data store, or by the RDBMS further comprise:

determining data tables for which the ranking exceeds a predetermined threshold.

9. The data consistency management system of claim 1, wherein the machine readable instructions to determine the data tables from the ranked data tables that are to be managed by the NoSQL data store, or by the RDBMS further comprise:

ranking data tables based on a linear combination of a percentage of the read queries for the data tables, a percentage of queries of the data tables that matches a key-select pattern, and a percentage of queries of the data tables that matches an aggregation pattern and aggregate over data in columns of the data tables; and

determining data tables for which at least one of the rankings related to the percentage of the read queries, the percentage of queries of the data tables that matches a key-select pattern, and the percentage of queries of the data tables that matches an aggregation pattern and aggregate over data in a column of the data table, exceed at least one predetermined threshold related to the percentage of the read queries, the percentage of queries of the data tables that matches a key-select pattern, and the percentage of queries of the data tables that matches an aggregation pattern and aggregate over data in a column of the data table.

10. The data consistency management system of claim 1, wherein the machine readable instructions to translate the query further comprise:

creating a data structure in the NoSQL data store according to the structure of the at least one data table.

11. The data consistency management system of claim 1, wherein the machine readable instructions to translate the query further comprise:

determining if the at least one data table includes a high percentage of queries that match a key-select pattern; and

based on a determination that the at least one data table includes a high percentage of queries that match a key-select pattern, creating a key-value store for the NoSQL data store by using a primary key of the at least one data table.

12. The data consistency management system of claim 1, wherein the machine readable instructions to translate the query further comprise:

determining if the at least one data table includes a high percentage of queries that match an aggregation pattern; and

based on a determination that the at least one data table includes a high percentage of queries that match an aggregation pattern, creating a column store for the NoSQL data store.

13. The data consistency management system of claim 1, wherein the machine readable instructions to determine the suitability of the query for processing by the NoSQL data store, or the RDBMS further comprise:

determining whether the query is an update query that updates data in the at least one data table managed by the NoSQL data store;

determining whether a conflict exists in the data of the at least one data table based on processing of the update query; and

based on a determination that a conflict exists in the data of the at least one data table based on processing of the update query, resolving the conflict based on a conflict resolution policy.

14. A method for data consistency management, the method comprising:

receiving a query;

determining a suitability of the query for processing by a not-only structured query language (NoSQL) data store, or a relational database management system (RDBMS);

ranking data tables based on a combination of read queries for the data tables and query patterns suitable for the NoSQL data store for the data tables, at least one of the data tables containing information for responding to the query;

outputting the ranked data tables for selection for management by the NoSQL data store;

receiving selection of data tables from the ranked data tables that are to be managed by the NoSQL data store;

assigning the selected data tables from the ranked data tables for management by the NoSQL data store, and remaining data tables from the ranked data tables for management by the RDBMS;

determining whether the query is for at least one data table managed by the NoSQL data store; and

based on a determination that the query is for the at least one data table managed by the NoSQL data store, translating, by a processor, the query to NoSQL application programming interface (API) calls for using the NoSQL data store to respond to the query.

15. The method of claim 14, further comprising:

based on a determination that the query is not for the at least one data table managed by the NoSQL data store, forwarding the query to the RDBMS.

16. The method of claim 14, wherein determining the suitability of the query for processing by the NoSQL data store, or the RDBMS further comprises:

determining whether the query is a select query that selects data from a data table via a primary key of the data table; and

determining whether the query is a select query that aggregates a single column of a data table.

17. The method of claim 14, wherein determining the suitability of the query for processing by the NoSQL data store, or the RDBMS further comprises:

determining whether the query is a select query that selects data from a data table via a primary key of the data table;

determining if a predetermined percentage of queries to the data table are select queries that select data from the data table via the primary key of the data table; and

based on a determination that a predetermined percentage of queries to the data table are select queries that select data from the data table via the primary key of the data table, using a key-value store for the NoSQL data store for processing the query.

18. The method of claim 14, wherein determining the suitability of the query for processing by the NoSQL data store, or the RDBMS further comprises:

determining whether the query is a select query that aggregates a single column of a data table;

determining if a predetermined percentage of queries to the data table are select queries that aggregate the single column of the data table; and

based on a determination that a predetermined percentage of queries to the data table are select queries that aggregate the single column of the data table, using a column store for the NoSQL data store for processing the query.

19. The method of claim 14, wherein ranking the data tables further comprises:

ranking the data tables based on a linear combination of a percentage of the read queries for the data tables and a percentage of the query patterns suitable for the NoSQL data store for the data tables.

20. A non-transitory computer readable medium having stored thereon machine readable instructions for data consistency management, the machine readable instructions when executed cause a computer system to:

receive a query;

determine a suitability of the query for processing by a not-only structured query language (NoSQL) data store, or a relational database management system (RDBMS), wherein the NoSQL data store provides lower data consistency than the RDBMS;

rank, by a processor, data tables based on a combination of read queries for the data tables and query patterns suitable for the NoSQL data store for the data tables, at least one of the data tables containing information for responding to the query;

based on the ranking, determine data tables from the ranked data tables that are to be managed by the NoSQL data store, or by the RDBMS.