Handling user queries against production and archive storage systems, such as for enterprise customers having large and/or numerous files

• US 10,324,914 B2
• Filed: 06/30/2015
• Issued: 06/18/2019
• Est. Priority Date: 05/20/2015
• Status: Active Grant

• Current Assignee: CommVault Systems Incorporated
• Original Assignee: Commvalut Systems Incorporated
• Inventors: Kumarasamy, Paramasivam, Senthilnathan, Muthusamy, Thati, Ravi, Mishra, Hemant
• Primary Examiner(s): Hershley, Mark E
• Application Number:
  US14/755,561
• Publication Number:
  US 20160342661A1
• Time in Patent Office: 1,449 Days
• Field of Search: 707661
• CPC Class Codes: G06F 16/134 Distributed indices
• CPC Class Codes: G06F 16/182 Distributed file systems
• CPC Class Codes: G06F 16/214 Database migration support
• CPC Class Codes: G06F 16/219 Managing data history or versioning querying versioned data G06F16/2474; querying temporal data G06F16/2477
• CPC Class Codes: G06F 16/22 Indexing; Data structures therefor; Storage structures
• CPC Class Codes: G06F 16/2264 Multidimensional index structures
• CPC Class Codes: G06F 16/2423 Interactive query statement specification based on a database schema
• CPC Class Codes: G06F 16/2452 Query translation
• CPC Class Codes: G06F 16/2455 Query execution
• CPC Class Codes: G06F 16/24575 using context
• CPC Class Codes: G06F 16/248 Presentation of query results
• CPC Class Codes: G06F 16/252 between a Database Management System and a front-end application
• CPC Class Codes: G06F 16/283 Multi-dimensional databases or data warehouses, e.g. MOLAP or ROLAP
• CPC Class Codes: G06F 3/0482 Interaction with lists of selectable items, e.g. menus

Abstract

This application discloses a system for processing requests for data in production and archive/backup databases. The system stores a mapping of data in production and archive databases to multiple different data storage locations, including a production system storing at least one production database and an archive system storing at least one archive database. The system receives a data request from an application component, which manages or receives data from a user interface and has received the data request directly from the user interface. The system translates the data request to one or more database queries based on the mapping. The system causes to execute the database queries against data on the production or archive system. The system returns a result of executing the one or more database queries to the application component, where the application component returns the result to the user via the user interface.

Claims

1. A computer-implemented method of processing requests for data in production and archive data stores, the method comprising:

by a database gate component, maintaining a mapping of database data with respect to at least two different data storage targets,

wherein the database data is divided among the at least two different data storage targets, which reside in distinct data storage devices,

wherein the mapping identifies the distinct data storage devices of the database data, and

wherein the at least two different data storage targets include a production data storage system and an archive data storage system;

by the database gate component, receiving a data request from an application logic component,

wherein the application logic component receives the data request from a user via a user interface,

wherein the data request is directed to the production data storage system, but based on the mapping, the database gate component determines that some data needed to satisfy the data request is stored in the archive data storage system, and

wherein the database gate component interfaces between the application logic component and one or more database drivers for accessing the at least two different data storage targets including the production data storage system and the archive data storage system;

by the database gate component, translating the data request to a production database query based on the mapping;

by the database gate component, translating the data request to an archive database query based on the mapping;

by the database gate component, causing to execute the production database query against data in the production data storage system by invoking a first database driver that is suitable to the production database;

by the database gate component, causing to execute the archive database query against data in the archive data storage system by invoking a second database driver that is suitable to the archive database;

by the database gate component, receiving a production query result based on the production database query;

by the database gate component, receiving an archive query result based on the archive database query; and

by the database gate component, returning a combined query result to the application logic component,

wherein the database gate component generates the combined query result which includes a combination of the production query result and the archive query result, and

wherein the application logic component returns the combined query result to the user via the user interface.

2. The computer-implemented method of claim 1, wherein data on each of the data storage targets is managed by a separate database management system.

3. The computer-implemented method of claim 1, further comprising:

by the database gate component, monitoring migration events where data moves between the production data storage system and the archive data storage system; and

by the database gate component, updating the mapping for each detected migration event,

wherein the monitoring includes communicating with a data agent that assists in copying, archiving, migrating, and/or replicating data on the production data storage system.

4. The computer-implemented method of claim 1,

wherein the application logic component presents a specific list of data requests, including the received data request, from which the user can make a selection via the user interface,

wherein the translating includes identifying one or more initial queries corresponding to the received request, and

wherein the translating further includes converting the one or more initial queries to the production and archive database queries based on the mapping.

5. The computer-implemented method of claim 1, further comprising:

by the database gate component, determining which of multiple, different database systems contain data that satisfies the received data request, wherein the determining is performed before the translating.

6. The computer-implemented method of claim 1, further comprising:

receiving a request to move data that satisfies a criterion, from the production data storage system to the archive data storage system;

building a nested database query from an initial database query according to a database schema;

estimating a size of a result of executing the nested database query;

receiving at a reporting server a copy of metadata from the production data storage system;

receiving, at the reporting server, a request for a report of data on the production data storage system; and

producing a data report at the reporting server in response to the request for the report, wherein the reporting server generates the report using the copy of metadata received from the production data storage system.

7. At least one non-transitory, computer-readable medium carrying instructions, which when executed by at least one data processor, performs processing of requests for data in production and archive databases, the instructions comprising:

maintaining, by a database gate component, a mapping of data with respect to at least two different data storage targets, which reside in distinct data storage devices,

wherein the data is divided among the at least two different data storage targets,

wherein the mapping identifies the distinct data storage devices of the database data, and

wherein the at least two different data storage targets include a production data storage system and an archive data storage system;

by the database gate component, receiving a data request from an application component,

wherein the application component receives the data request via a user interface, and

wherein the database gate component interfaces between the application component and database drivers suitable for the at least two different data storage targets including the production data storage system and the archive data storage system;

by the database gate component, determining that the data request is directed to the production data storage system, but based on the mapping, further determining that some data needed to satisfy the data request is stored in the archive data storage system;

by the database gate component, based on the mapping, translating the data request into one or more first queries for the production data storage system and one or more second queries for the archive data storage system;

by the database gate component, causing the one or more first queries to be executed against the production data storage system by invoking a first database driver that is suitable to data in the production data storage system;

by the database gate component, causing the one or more second queries to be executed against the archive data storage system by invoking a second database driver that is suitable to data in the archive data storage system; and

by the database gate component, returning a combined query result of executing each query in the set of database queries, wherein the database gate component generates the combined query result which includes a combination of results from the one or more first queries and the one or more second queries, and

wherein the combined query result is returned to the user via the user interface.

8. The computer-readable medium of claim 7, wherein data on each of the data storage targets is managed by a separate database management system.

9. The computer-readable medium of claim 7, wherein the instructions further comprise:

by the database gate component, monitoring migration events where data moves from one of the data storage targets to another; and

by the database gate component, updating the mapping for each detected migration event.

10. The computer-readable medium of claim 9, wherein the monitoring includes communicating with a data agent that assists in copying, archiving, migrating, and/or replicating data on the production data storage system.

11. The computer-readable medium of claim 7,

wherein the application component presents a specific list of data requests, including the received data request, from which the user can make a selection via the user interface,

wherein the translating includes identifying one or more initial queries corresponding to the received data request, and

wherein the translating further includes converting the one or more initial queries to the one or more first queries and the one or more second queries based at least in part on the mapping.

12. The computer-readable medium of claim 7, wherein the database gate component, the application component, and the mapping reside on a client computing device.

13. The computer-readable medium of claim 7, wherein the instructions further comprise:

receiving a request to move data that satisfies a criterion, from the production data storage system to the archive data storage system;

building a nested database query from an initial database query according to a database schema;

estimating a size of a result of executing the nested database query;

receiving at a reporting server a copy of metadata from the production system;

receiving, at the reporting server, a request for a report of data on the production data storage system; and

producing a data report at the reporting server in response to the request, wherein the reporting server generates the data report using the copy of metadata received from the production data storage system.

14. A system configured to process requests related to database data, the system comprising:

at least one processor;

at least one memory coupled to the processor and storing a mapping data structure that maps distinct data storage devices of database data,

wherein the database data has been divided among at least a first data storage system and a second, separate data storage system, and

wherein the mapping data structure identifies the distinct data storage devices of the database data;

wherein the memory also stores instructions for execution by the processor, wherein the instructions encode a method comprising;

by a database gate component, maintaining the mapping data structure;

by the database gate component, receiving, via a user interface, a request for data, wherein the user interface is within an application logic layer, and wherein the database gate component interfaces between the application logic layer and one or more database drivers for accessing the first data storage system and second separate data storage system;

by the database gate component, converting the request for data into a first query directed at the first data storage system and a second query directed at the second data storage system, wherein the converting is based on data in the mapping data structure;

by the database gate component, causing the first query and the second query to execute against data in the first data storage system and the second data storage system, respectively; and

by the database gate component, generating a combined result of executing the first query and the second query, which combined result is returned to the application logic layer in response to the request for data.

15. The system of claim 14, wherein the method further comprises:

by the database gate component, detecting migration events where data moves between the first data storage system and the second data storage system; and

by the database gate component, updating the mapping data structure for each detected migration event.

16. The system of claim 15,

wherein the first data storage system resides on a production system, and the second data storage system resides on an archive system, and

wherein the detecting includes communicating with a data agent that assists in copying, archiving, migrating, and/or replicating data on the first data storage system.

17. The system of claim 14,

wherein the request for data is selected from a specific list of data requests via the user interface,

wherein the converting includes identifying one or more initial queries corresponding to the request for data, and

wherein the converting further includes converting the one or more initial queries to the first and second queries based on the mapping data structure.

18. The system of claim 14, wherein the causing includes invoking a respective database driver for the first data storage system and the second data storage system.