Predicting scale of data migration between production and archive storage systems, such as for enterprise customers having large and/or numerous files

• US 10,089,337 B2
• Filed: 07/28/2015
• Issued: 10/02/2018
• Est. Priority Date: 05/20/2015
• Status: Active Grant

• Current Assignee: CommVault Systems Incorporated
• Original Assignee: CommVault Systems Incorporated
• Inventors: Senthilnathan, Muthusamy, Thati, Ravi, Kumarasamy, Paramasivam, Mishra, Hemant
• Primary Examiner(s): Le, Hung
• Application Number:
  US14/810,546
• Publication Number:
  US 20160342633A1
• Time in Patent Office: 1,162 Days
• Field of Search: None
• CPC Class Codes: G06F 16/134 Distributed indices
• CPC Class Codes: G06F 16/182 Distributed file systems
• CPC Class Codes: G06F 16/214 Database migration support
• CPC Class Codes: G06F 16/219 Managing data history or versioning querying versioned data G06F16/2474; querying temporal data G06F16/2477
• CPC Class Codes: G06F 16/22 Indexing; Data structures therefor; Storage structures
• CPC Class Codes: G06F 16/2264 Multidimensional index structures
• CPC Class Codes: G06F 16/2423 Interactive query statement specification based on a database schema
• CPC Class Codes: G06F 16/2452 Query translation
• CPC Class Codes: G06F 16/2455 Query execution
• CPC Class Codes: G06F 16/24575 using context
• CPC Class Codes: G06F 16/248 Presentation of query results
• CPC Class Codes: G06F 16/252 between a Database Management System and a front-end application
• CPC Class Codes: G06F 16/283 Multi-dimensional databases or data warehouses, e.g. MOLAP or ROLAP
• CPC Class Codes: G06F 3/0482 Interaction with lists of selectable items, e.g. menus

Abstract

This application discloses a system for presenting data to a user based on requests to move data from primary storage to secondary storage. The system receives a request to move data that satisfies at least one criterion, from primary to secondary data storage. The system then identifies an initial database query from the request, and further builds a nested database query from the initial database query according to a database schema. Next, the system estimates a size of a result of executing the nested database query. Finally, the system provides for display to the user the estimated size.

Claims

1. A computer-implemented method of predicting an amount of data storage needed when moving data from primary data storage to secondary or archive data storage, the method comprising:

receiving, at a client computing device, a request to move data that satisfies at least one criterion,

wherein the moving of data is from a primary data storage system to a secondary data storage system, and

wherein the at least one criterion is related to age or size of data;

providing an initial database query based on the request,

wherein the initial database query is related to an initial table in an initial database;

building a nested database query from the initial database query according to a database schema,

wherein building the nested database query includes determining additional tables linked to the initial table;

predicting a size of data that will be returned as a result of executing the nested database query; and

providing the predicted size before moving the data from the primary data storage system to the secondary data storage system.

2. The computer-implemented method of claim 1, further comprising

presenting a user interface to a user showing a list of predetermined requests,

wherein the request received is from the user and is chosen from the list of predetermined requests.

3. The computer-implemented method of claim 1, wherein the building includes traversing the database schemas according to foreign keys, and further comprising determining whether the secondary storage system has sufficient data storage space based on the predicted size.

4. The computer-implemented method of claim 1, wherein the budding includes computing a count of a number of records that match the nested database query, and wherein the budding includes computing an aggregate size from sizes of the records that match the nested database query.

5. The computer-implemented method of claim 1, wherein the building includes computing a count of records in a database table referenced in the nested database query and a size of a record in the database table.

6. The computer-implemented method of claim 1, further comprising determining whether to satisfy the request based on the predicted size, and wherein the determining is based on a computing bad of the primary data storage system, a computing bad of the secondary data storage system, an amount of free space on the primary data storage system, an amount of free space on the secondary data storage system, or a user need of data affected by the request.

7. The computer-implemented method of claim 1, further comprising:

storing, by a database gate component, a mapping of data to a plurality of different data storage locations, including the primary data storage system storing at least one production database and the secondary data storage system storing at least one archive database;

receiving a data request from an application logic component,

wherein the application logic component manages or receives data from a user interface;

at a database gate component, translating the data request to one or more database queries based on a mapping of data between production and archive databases associated with the primary and secondary data storage systems;

causing to execute at least two database queries against data in the primary and secondary data storage systems;

returning to the application logic component a combined result of executing the database queries;

receiving at a reporting server a copy of metadata from the primary data storage system;

receiving, at the reporting server, a request for a report of data on the primary data storage system; and

producing a data report at the reporting server in response to the request,

wherein the reporting server generates the data report using the copy of metadata received from the primary data storage system.

8. At least one non-transitory, computer-readable medium carrying, instructions, which when executed by at least one data processor, performs a method of presenting data to a user based on requests to move data from primary storage to secondary storage, comprising:

receiving, from a client computing device, a request from a user to move data that satisfies a criterion, from a primary storage system to a secondary storage system;

identifying an initial database query from the request,

wherein the initial database query is related to an initial table in the initial database;

building a nested database query from the initial database query according to a database schema;

wherein building the nested database query includes determining additional tables linked to the initial table;

estimating a size of a result of executing the nested database query; and

providing the estimated size to be displayed to the user.

9. The computer-readable medium of claim 8, wherein the criterion is related to an age or size of the data.

10. The computer-readable medium of claim 8, further comprising providing data for a user interface to show to a user a list of predetermined requests, wherein the request received from the user is chosen from the list of predetermined requests.

11. The computer-readable medium of claim 8, wherein the building includes traversing the database schemas according to foreign keys.

12. The computer-readable medium of claim 8, wherein the building includes computing a count of a number of records that match the nested database query, and wherein the building includes computing an aggregate of sizes of the records that match the nested database query.

13. The computer-readable medium of claim 8, wherein the building includes computing a count of records in a database table referenced in the nested database query and a size of a record in the database table.

14. The computer-readable medium of claim 8, wherein the method further comprises determining whether to satisfy the request based on the estimated size, and wherein the determining is based on a computing bad of the primary storage system, a computing bad of the secondary storage system, an amount of free space on the primary storage system, an amount of free space on the secondary storage system, or a user need of data affected by the request.

15. The computer-readable medium of claim 8, further comprising:

storing, by a database gate component, a mapping of data in production and archive databases respectively on the primary and secondary storage systems;

receiving a data request from an application logic component,

wherein the application logic component receives data from a user interface;

at a database gate component, translating the data request to one or more database queries based on a mapping of data between production and archive databases;

causing to execute at least two database queries against data in the primary and secondary storage systems;

returning to the application logic component a combined result of executing the database queries;

receiving at a reporting server a copy of metadata from the primary storage system;

receiving, at the reporting server, a request for a report of data on the primary storage system; and

producing a data report at the reporting server in response to the request,

wherein the reporting server generates the data report using the copy of metadata received from the primary storage system.

16. A system configured to present data to a user related to requests to move data from primary storage to secondary storage, the system comprising:

at least one processor;

at least one memory coupled to the processor and storing instructions for execution by the processor, wherein the instructions encode a method comprising;

receiving a request from a user to move data that satisfies a criterion from a primary storage system to a secondary storage system;

providing the request to a remote computing device to provide an estimated size of data to be moved,

wherein the remote computing device

identifies an initial database query from the request,

 

wherein the initial database query is related to an initial table in an initial database;

builds a database query from the initial database query according to a database schema, wherein building the database query includes determining additional tables linked to the initial table, and

estimates a size of a result of executing the database query;

receiving from the remote computing device, the estimated size; and

providing for display to the user the estimated size.

17. The system of claim 16, the method further comprising

presenting a user interface to the user showing a list of predetermined requests,

wherein the request received is chosen from the list of predetermined requests.

18. The system of claim 16, wherein the estimated size is based on a count of a number of records that match the database query and an aggregate size from sizes of the records that match the database query.

19. The system of claim 16, wherein the estimated size is based on a count of records in each database table referenced in the database query and a size of a record in each database table.

20. The system of claim 16, the method further comprising determining whether to satisfy the request based on the predicted size, and wherein the determining is based on a computing load of the primary storage system, a computing load of the secondary storage system, an amount of free space on the primary storage system, an amount of free space on the secondary storage system, or a user need of data affected by the request.