Automatic data store architecture detection

• US 10,223,390 B2
• Filed: 03/22/2017
• Issued: 03/05/2019
• Est. Priority Date: 12/15/2008
• Status: Active Grant

• Current Assignee: Tamiras Per Pte Ltd LLC (Intellectual Ventures LLC)
• Original Assignee: Tamiras Per Pte Ltd LLC (Intellectual Ventures LLC)
• Inventors: Shaked, Guy, Braverman, Vladimir, Sadetsky, Marina
• Primary Examiner(s): Thomas, Ashish
• Assistant Examiner(s): Ohba, Mellissa M.
• Application Number:
  US15/466,430
• Publication Number:
  US 20170193018A1
• Time in Patent Office: 713 Days
• Field of Search: None
• CPC Class Codes: G06F 16/213 with details for schema evolution support
• CPC Class Codes: G06F 16/217 Database tuning G06F16/2282 takes precedence; database performance monitoring G06F11/3409
• CPC Class Codes: G06F 16/9535 Search customisation based on user profiles and personalisation

Abstract

The present invention is a system for automatic recognition of data store architecture and tracking dynamic changes and evolution in data store architecture. The system of the invention is a complementary system, which can be added onto an existing data store system using the existing interfaces or can be integrated with a data store system. The system comprises three main components that are configured to compose an approximation of the data store architecture. The first of these components is adapted to execute an analysis of the architecture of the data store; the second of the components is adapted to collect and compile statistics from said data store; and the third of the components is adapted to compose an approximation of the architecture of said data store. Methods for using the system of the invention are also described.

Claims

1. A system comprising:

a processing device comprising hardware;

a non-transitory computer-readable medium encoded with;

a data store analyzer module comprising;

an architecture analysis composer and analyzer component, which when executed by the processing device, causes the system to execute an analysis of an architecture of a data store prior to having received any user queries, the analysis of the architecture of the data store comprising an analysis of a relationship between columns of data store tables; and

a data store architecture composer component, which when executed by the processing device, causes the system to compose an initial approximation of the architecture of the data store based on the analysis of the architecture of the data store prior to having received any user queries and independent of any output from a queries analyzer module, wherein the initial approximation of the architecture of the data store comprises an estimate of column sizes;

the queries analyzer module, which comprises;

a queries analyzer component, which when executed by the processing device, causes the system to perform a syntactic analysis of a user query when a query is available;

a queries statistics composer component, which when executed by the processing device, causes the system to compile statistics on query content;

a queries architecture composer component, which when executed by the processing device, causes the system to compose an updated approximation of the data store architecture, including at least an update of a first approximation of the architecture of the data store, the first approximation of the architecture of the data store based on the analysis of the architecture of the data store prior to having received any user queries and independent of any output from the queries analyzer module;

a query repository comprising content, the query repository coupled to the queries analyzer component which is configured to use the query repository content to execute an analysis of an impact of data store architecture changes as the data store changes are recognized by the system;

wherein the queries analyzer module is configured to collect, to analyze and to generate statistics related to the user queries and to integrate the statistics related to the user queries with the first approximation, that is based on an analysis of the architecture of the data store executed prior to having received any user queries;

wherein the system is configured to dynamically detect when tables are inserted, modified, or removed from the data store by automatically carrying out continuous data store architecture analysis comprising continuous analysis of the relationship between columns of data store tables, and comparing successive approximations of the architecture of the data store comprising respective estimates of the column sizes, and in response to detecting that a table has been inserted, modified, or removed from the data store, the query repository receives a report of changes to components of the data store from the data architecture component and selects one or more query templates that do not contain the changed components of the data store and transfers the selected one or more query templates to the query analyzer component.

2. The system according to claim 1, wherein at least one data store approximation includes a collection of statistics constructed from data extracted from the data store and stored as separate entities.

3. The system according to claim 1, wherein at least one data store approximation is used as an input to a data arrangement application.

4. The system according to claim 1, wherein at least one data store approximation is configured to be used by a data store administrator to change existing data store architecture.

5. The system according to claim 1, further comprising a resources limits detector component, which when executed by the processing device, causes the system to:

check designated resources of a data store system,

analyze the data store system resources limits, and

generate and provide a resources analysis report,

wherein the resources limits detector component is configured to check designated resources of the data store system by;

tracking indicators of a performance of the data store system,

determining resources of the data store system are available for use by the system to carry out another cycle of data store architecture approximation improvement or if the data store system is too busy or does not have enough available memory, disc space, or processing available for use by the system to carry out another cycle of data store architecture approximation improvement.

6. The system according to claim 1, further comprising a resources limits detector component, which when executed by the processing device, causes the system to:

check designated resources of a data store system;

analyze the data store system resources limits; and

generate and provide a resources analysis report.

7. The system according to claim 1, wherein the system is configured to work with a subset of a data store which is obtained by volume reduction.

8. The system according to claim 1, wherein the system is configured to work with a mirrored data store which is obtained by duplicating an existing data store.

9. The system according to claim 1, wherein the system is configured to work with a data warehouse to improve performance by modifying an aggregation layer.

10. A computerized method comprising:

executing, by a computing system comprising hardware, an analysis of an architecture of a data store in prior to having received any user queries, the analysis of the architecture of the data store comprising an analysis of a relationship between columns of data store tables;

collecting and compiling statistics, by the computing system, from the data store;

composing, by the computing system, a first approximation of the architecture of the data store using the analysis of the architecture of the data store prior to having received any user queries, wherein the first approximation of the architecture of the data store comprises a first estimate of column sizes;

performing, by the computing system, a syntactic analysis of a user query when a query is available;

compiling, by the computing system, statistics on query content when queries are available;

composing, by the computing system, one or more updated approximations of the data store architecture using the statistics on query content, including at least an update of the first approximation of the architecture of the data store, the first approximation of the architecture of the data store based on the analysis of the architecture of the data store prior to having received any user queries;

performing, by the computing system, an analysis of the impact of data store architecture changes as the data store changes are recognized;

compiling and analyzing, by the computing system, statistics related to the users queries;

integrating, by the computing system, the analyzed statistics with the first approximation composed using the analysis of the architecture of the data store prior to having received any user queries to compose a more precise approximation of the architecture of the data store; and

dynamically detecting, by the computing system, when tables are inserted, modified, or removed from the data store by automatically carrying out continuous data store architecture analysis comprising continuous analysis of the relationship between columns of data store tables and comparing successive approximations of the architecture of the data store comprising respective estimates of column sizes;

in response to detecting that a table has been inserted, modified, or removed from the data store, receiving a report of changes to components of the data store and selecting one or more query templates that do not contain the changed components of the data store; and

transferring the selected one or more query templates to a query analyzer.

11. The method of claim 10, wherein at least one data store approximation includes a collection of statistics constructed from data extracted from the data store and stored as separate entities.

12. The method of claim 10, wherein at least one data store approximation is used as an input to a data arrangement application.

13. The method of claim 10, wherein at least one data store approximation is configured to be used by a data store administrator to change existing data store architecture.

14. The method of claim 10, the method further comprising performing volume reduction on the data store and using the reduced data store to perform at least one approximation of the architecture of the data store.

15. The method of claim 10, the method further comprising mirroring the data store, wherein at least one approximation of the architecture of the data store is performed using the mirrored data store.

16. A non-transitory computer-readable medium encoded with instructions thereon, wherein the instructions are readable by a computing device in order to cause the computing device to perform operations comprising:

executing an analysis of an architecture of a data store in prior to having received any user queries, the analysis of the architecture of the data store comprising an analysis of a relationship between columns of data store tables;

collecting and compiling statistics from the data store;

composing a first approximation of the architecture of the data store using the analysis of the architecture of the data store prior to having received any user queries, wherein the first approximation of the architecture of the data store comprises a first estimate of column sizes;

performing a syntactic analysis of a user query when a query is available;

compiling statistics on query content when queries are available;

composing one or more updated approximations of the data store architecture using the statistics on query content, including at least an update of the first approximation of the architecture of the data store, the first approximation of the architecture of the data store based on the analysis of the architecture of the data store prior to having received any user queries;

performing an analysis of the impact of data store architecture changes as the data store changes are recognized;

compiling and analyzing statistics related to the users queries;

integrating the analyzed statistics with the first approximation composed using the analysis of the architecture of the data store prior to having received any user queries to compose a more precise approximation of the architecture of the data store; and

dynamically detecting when tables are inserted, modified, or removed from the data store by automatically carrying out continuous data store architecture analysis comprising continuous analysis of the relationship between columns of data store tables and comparing successive approximations of the architecture of the data store comprising respective estimates of column sizes;

in response to detecting that a table has been inserted, modified, or removed from the data store, receiving a report of changes to components of the data store and selecting one or more query templates that do not contain the changed components of the data store; and

transferring the selected one or more query templates to a query analyzer.

17. The non-transitory computer-readable medium according to claim 16, wherein at least one data store approximation includes a collection of statistics constructed from data extracted from the data store and stored as separate entities.

18. The non-transitory computer-readable medium according to claim 16, wherein at least one data store approximation is used as an input to a data arrangement application.

19. The non-transitory computer-readable medium according to claim 16, wherein at least one data store approximation is configured to be used by a data store administrator to change existing data store architecture.

20. The non-transitory computer-readable medium according to claim 16, the operations further comprising performing volume reduction on the data store and using the reduced data store to perform at least one approximation of the architecture of the data store.