Automatic data store architecture detection

• US 9,607,022 B2
• Filed: 12/14/2009
• Issued: 03/28/2017
• Est. Priority Date: 12/15/2008
• Status: Active Grant

• Current Assignee: Tamiras Per Pte Ltd LLC (Intellectual Ventures LLC)
• Original Assignee: Tamiras Per Pte Ltd LLC (Intellectual Ventures LLC)
• Inventors: Shaked, Guy, Braverman, Valdimir, Sadetsky, Marina
• Primary Examiner(s): Waldron, Scott A
• Assistant Examiner(s): CHOJNACKI, MELLISSA M
• Application Number:
  US13/139,718
• Publication Number:
  US 20120066264A1
• Time in Patent Office: 2,661 Days
• US Class Current: 1/1
• CPC Class Codes: G06F 16/213 with details for schema evolution support
• CPC Class Codes: G06F 16/217 Database tuning G06F16/2282 takes precedence; database performance monitoring G06F11/3409
• CPC Class Codes: G06F 16/9535 Search customisation based on user profiles and personalisation

Abstract

The present invention is a system for automatic recognition of data store architecture and tracking dynamic changes and evolution in data store architecture. The system of the invention is a complementary system, which can be added onto an existing data store system using the existing interfaces or can be integrated with a data store system. The system comprises three main components that are configured to compose an approximation of the data store architecture. The first of these components is adapted to execute an analysis of the architecture of the data store; the second of the components is adapted to collect and compile statistics from said data store; and the third of the components is adapted to compose an approximation of the architecture of said data store. Methods for using the system of the invention are also described.

Claims

1. A complementary system comprising:

a processing device comprising hardware;

a non-transitory computer-readable medium encoded with;

a data store analyzer module comprising;

an architecture analysis composer and analyzer component, which when executed by the processing device, causes the system to execute an analysis of an architecture of a data store prior to having received any user queries, wherein the analysis of the architecture of the data store identifies data store table names, identifies different columns in different tables of the data store that have duplicated data, and determines column sizes;

a data store statistics composer component, which when executed by the processing device, causes the system to collect and compile statistics from the data store; and

a data store architecture composer component, which when executed by the processing device, causes the system to compose an initial approximation of the architecture of the data store based on the analysis of the architecture of the data store prior to having received any user queries and independent of any output from a queries analyzer module;

the queries analyzer module, which comprises;

a queries analyzer component, which when executed by the processing device, causes the system to perform a syntactic analysis of a user query when a query is available;

a queries statistics composer component, which when executed by the processing device, causes the system to compile statistics on query content and query environment properties;

a queries architecture composer component, which when executed by the processing device, causes the system to compose an updated approximation of the data store architecture, including at least an update of the initial approximation of the architecture of the data store, the initial approximation of the architecture of the data store based on the analysis of the architecture of the data store prior to having received any user queries and independent of any output from the queries analyzer module;

a query repository comprising content, the query repository coupled to the queries analyzer component which is configured to use the query repository content to execute an immediate analysis of the impact of data store architecture changes as the data store changes are recognized by the system;

wherein the queries analyzer module components are configured to enable the queries analyzer module to collect, to analyze and to generate statistics related to the user queries and to integrate the statistics related to the user queries with the approximation received from the data store analyzer, that are based on the analysis of the architecture of the data store executed prior to having received any user queries, to compose a more precise approximation of the architecture of the data store;

wherein the complementary system is configured to dynamically detect when tables are inserted, modified, or removed from the data store by automatically carrying out continuous data store architecture analysis and comparing successive approximations of the architecture of the data store, and in response to detecting that a table has been inserted, modified, or removed from the data store, the query repository receives a report of changes to components of the data store from the data architecture composer component and selects one or more query templates that do not contain the changed components of the data store and transfers the selected one or more query templates to the query analyzer component;

wherein the complementary system comprises interfaces enabling the complementary system to interface with an existing data store system using existing interfaces or the complementary system is integrated with a data store system.

2. The complementary system according to claim 1, wherein the data store approximation includes a collection of statistics constructed from data extracted from the data store and stored as separate entities.

3. The complementary system according to claim 2, wherein the data store approximation is used as an input to any data arrangement application.

4. The complementary system according to claim 1, wherein the data store approximation is used by a data store administrator to change existing data store architecture.

5. The complementary system according to claim 1, further comprising a resources limits detector component, which when executed by the processing device, causes the system to check designated resources of the data store system, analyze the data store system resources limits, and generates and provides a resources analysis report, wherein the resources limits detector component comprises a specific software component that checks the resources of the data store system by tracking a few indicators of a performance of the data store system, in order to know if the available resources of the data store system are available for use by the complementary system to carry out another cycle of data store architecture approximation improvement or if the data store system is too busy or does not have enough available memory, disc space, or CPU available for use by the complementary system to carry out another cycle of data store architecture approximation improvement.

6. The complementary system according to claim 1, further comprising a resources limits detector component, which when executed by the processing device, causes the system to check designated resources of the data store system, analyze the data store system resources limits, and generates and provides a resources analysis report.

7. The complementary system according to claim 1, wherein the complementary system is configured to work with a subset of a data store which is obtained by sampling or by any other method of volume reduction.

8. The complementary system according to claim 1, wherein the complementary system is configured to work with a mirrored data stores which is obtained by duplicating the original data store.

9. The complementary system according to claim 1, wherein the complementary system is configured to work with a data warehouse to improve performance by modifying an aggregation layer.

10. The complementary system according to claim 1, wherein the analysis of the data store architecture comprises:

identification of table columns in a data store table that have a same meaning;

a determination of column sizes of the table; and

identification of a hierarchical structure of the table.

11. A method comprising:

executing, by a complementary computing system comprising hardware, an analysis of an architecture of a data store in prior to having received any user queries, wherein the analysis of the architecture of the data store identifies data store table names, identifies different columns in different tables of the data store that have duplicated data, and determines column sizes;

collecting and compiling statistics from the data store;

composing an initial approximation of the architecture of the data store using the analysis of the architecture of the data store prior to having received any user queries;

performing a syntactic analysis of a user query when a query is available;

compiling statistics on query content and query environment properties when queries are available;

composing one or more updated approximations of the data store architecture using the statistics on query content and query environment properties, including at least an update of the initial approximation of the architecture of the data store, the initial approximation of the architecture of the data store based on the analysis of the architecture of the data store prior to having received any user queries;

performing an analysis of the impact of data store architecture changes as the data store changes are recognized;

analyzing and generating statistics related to the users queries;

integrating the analyzed statistics with the approximation composed using the analysis of the architecture of the data store prior to having received any user queries to compose a more precise approximation of the architecture of the data store; and

dynamically detecting when tables are inserted, modified, or removed from the data store by automatically carrying out continuous data store architecture analysis and comparing successive approximations of the architecture of the data store;

in response to detecting that a table has been inserted, modified, or removed from the data store, receiving a report of changes to components of the data store and selecting one or more query templates that do not contain the changed components of the data store; and

transferring the selected one or more query templates to a query analyzer.

12. The method of claim 11, wherein the data store approximation includes a collection of statistics constructed from data extracted from the data store and stored as separate entities.

13. The method according to claim 12, wherein the data store approximation is used as an input to any data arrangement application.

14. The method according to claim 11, wherein the data store approximation is used by a data store administrator to change existing data store architecture.

15. The method of claim 11, the method further comprising:

identifying table columns in a data store table that have a same meaning;

determining column sizes of the table; and

identifying a hierarchical structure of the table.

16. A non-transitory computer-readable medium encoded with instructions thereon, wherein the instructions are readable by a computing device in order to cause the computing device to perform operations comprising:

executing an analysis of an architecture of a data store prior to having received any user queries, wherein the analysis of the architecture of the data store identifies data store table names, identifies different columns in different tables of the data store that have duplicated data, and determines column sizes;

collecting and compiling statistics from the data store using the analysis of the architecture of the data store user queries;

composing an initial approximation of the architecture of the data store using the executed analysis of the architecture of a data store performed prior to having received any user queries;

performing a syntactic analysis of a user query when a query is available;

compiling statistics on query content and query environment properties when queries are available;

composing one or more updated approximations of the data store architecture, including at least an update of the initial approximation of the architecture of the data store, the initial approximation of the architecture of the data store based on the analysis of the architecture of the data store prior to having received any user queries, using the statistics on query content and query environment properties;

performing an analysis of the impact of data store architecture changes as the data store changes are recognized;

analyzing and generating statistics related to the users queries;

integrating the analyzed statistics with the approximation composed using the analysis of the architecture of the data store executed prior to having received any user queries to compose a more precise approximation of the architecture of the data store; and

dynamically detecting when tables are inserted, modified, or removed from the data store by automatically carrying out continuous data store architecture analysis and comparing successive approximations of the architecture of the data store;

in response to detecting that a table has been inserted, modified, or removed from the data store, receiving a report of changes to components of the data store and selecting one or more query templates that do not contain the changed components of the data store; and

transferring the selected one or more query templates to a query analyzer.

17. The non-transitory computer-readable medium of claim 16, wherein the data store approximation includes a collection of statistics constructed from data extracted from the data store and stored as separate entities.

18. The non-transitory computer-readable medium according to claim 17, wherein the data store approximation is used as an input to any data arrangement application.

19. The non-transitory computer-readable medium according to claim 16, wherein the data store approximation is used by a data store administrator to change existing data store architecture.

20. The non-transitory computer-readable medium according to claim 16, wherein the operations further comprise working with a data warehouse to improve performance by modifying an aggregation layer.

21. The non-transitory computer-readable medium according to claim 16, wherein the operations further comprise:

identifying table columns in a data store table that have a same meaning;

determining column sizes of the table; and

identifying a hierarchical structure of the table.