Automated validation of database index creation

• US 10,061,678 B2
• Filed: 06/26/2015
• Issued: 08/28/2018
• Est. Priority Date: 06/26/2015
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: Jovanovic, Andrija, Jovandic, Isidora, Krakovic, Dejan, Acimovic, Milo{hacek over (s)}, Nikoli{hacek over (c)}ic, {hacek over (Z)}eljko, Lotrean, Petar, Grbic, Miroslav, Maksimovic, Dorde, Stanojevic, Svetlana
• Primary Examiner(s): Le, Hung
• Application Number:
  US14/752,575
• Publication Number:
  US 20160378634A1
• Time in Patent Office: 1,159 Days
• CPC Class Codes: G06F 11/3034 where the computing system component is a storage system, e.g. DASD based or network based digital input from or digital output to record carriers G06F3/06; digital recording or reproducing G11B20/18; for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS], H04L67/1097
• CPC Class Codes: G06F 11/3409 for performance assessment
• CPC Class Codes: G06F 16/217 Database tuning G06F16/2282 takes precedence; database performance monitoring G06F11/3409
• CPC Class Codes: G06F 16/2272 Management thereof

Abstract

Automated validation of the creation of indices in an environment that include multiple and potential many databases, such as perhaps a cloud computing environment. A validation module validates index impact of a created index by using a validation data store that contains validation data originating from a database collection. Index impact may be estimated by evaluating validation data generated prior to and after the creation of the index to thereby determine whether the created index results in overall improved query performance on the database collection for those queries that target the newly indexed database entity. Such validation data need not even contain private data that was contained within the query itself, and might be, for instance, query performance data, or execution plans associated with the query, with private data redacted.

Claims

1. A computing system comprising:

one or more processors; and

a validation module, executing on the one or more processors, that operates to automatically validate creation of an index within a particular database entity in an environment that includes multiple database entities, the validation module configured to;

automatically validate an overall index impact of the created index on the entire environment that includes the multiple databases by using a validation data store that contains validation data originating from a plurality of databases, wherein validating the overall index impact includes considering at least;

a first performance impact of the created index on one or more first queries that utilize the created index; and

a second performance impact of the created index on one or more second queries that do not utilize the created index; and

automatically retaining or deleting the index based on a consideration of both the first performance impact and the second performance impact.

2. The system in accordance with claim 1, the validation data store comprising a database that contains aggregated validation data from the plurality of databases.

3. The system in accordance with claim 1, further comprising:

the validation data store.

4. The system in accordance with claim 1, further comprising:

at least one of the plurality of databases.

5. The system in accordance with claim 1, the plurality of databases at least partially contained within a cloud computing environment.

6. The system in accordance with claim 1, the validation module configured to automatically validate performance improvement by evaluating validation data generated prior to creation of the index, evaluating validation data generated after the creation of the index, and comparing the evaluations.

7. The system in accordance with claim 6, the validation module configured to recommend or decide to revert or keep the created index based on the comparison.

8. The system in accordance with claim 1, the validation data comprising query performance data.

9. The system in accordance with claim 1, the validation data comprising live workload data.

10. The system in accordance with claim 1, the validation data comprising metadata from queries issued against the plurality of databases.

11. The system in accordance with claim 10, the metadata comprising at least a portion of execution plans for performing queries against the database entity.

12. The system in accordance with claim 1, the validation data excluding private data from the one or more first queries and the one or more second queries issued against the plurality of databases.

13. The system in accordance with claim 1, the validation module configured to perform validation for indexes created on any of the plurality of databases.

14. The system in accordance with claim 1, the plurality of databases comprising at least a million databases.

15. The computing system in accordance with claim 1, wherein the first performance impact comprises a first weighted value and the second performance impact comprises a second weighted value, wherein consideration of both the first performance impact and the second performance impact comprises considering both the first weighted value and the second weighted value.

16. The computing system in accordance with claim 15, the first performance impact indicating a performance increase, the second performance impact indicating a performance decrease, wherein the index is automatically retained when the performance increase of the first performance impact is greater than the performance decrease of the second performance impact.

17. A computer-implemented method, executed at a computer system comprising one or more processors, for a validation module to automatically validate creation of an index within a particular database entity in an environment that includes multiple database entities, the method comprising:

the validation module determining that creation of an index is to be validated;

the validation module communicating with a validation data store that contains validation data originating from a plurality of databases; and

evaluating validation data from the validation data store that was generated before and after the creation of the index to estimate index impact on the entire environment that includes the multiple database entities, wherein validating the overall index impact includes at least;

estimating a first performance impact of the created index on one or more first queries that utilize the created index; and

estimating a second performance impact of the created index on one or more second queries that do not utilize the created index; and

automatically retaining or deleting the created index based on both the estimated first performance impact and the estimated second performance impact.

18. The computer-implemented method in accordance with claim 17, the evaluated validation data comprising metadata of at least the one or more first queries or the one or more second queries made to the plurality of databases.

19. The method in accordance with claim 17, the method being performed for a plurality of index creations for indexes corresponding to a plurality of database entities of the plurality of databases.

20. A computing system comprising a validation module, wherein the validation module is configured to at least:

automatically validate index creation of an index of a database entity of a database using live workload data of the database entity, the live workload data including validation data generated prior to creation of the index and validation data generated after the creation of the index;

evaluate the validation data generated prior to and after creation of the index to thereby generate an overall database performance impact estimate, including;

a performance increase estimate on one or more first queries that utilize the created index; and

an performance decrease estimate on one or more second queries that do not utilize the created index; and

automatically retain the created index based on identifying that the overall database performance impact is improved or automatically delete the created index when it is determined that the overall database performance impact is degraded.

21. The system in accordance with claim 20,

the validation module further configured to recommend a keep or revert decision with respect to the created index based on the index impact estimate.

22. The system in accordance with claim 20,

the validation module further configured to trigger a keep or revert decision with respect to the created index based on the index impact estimate.