SYSTEMS AND METHODS FOR AUTOMATING MANAGEMENT OF DISTRIBUTED DATABASES

• US 20170322954A1
• Filed: 07/19/2017
• Published: 11/09/2017
• Est. Priority Date: 12/15/2015
• Status: Active Grant

• Current Assignee: MongoDB, Inc.
• Original Assignee: MongoDB, Inc.
• Inventors: Horowitz, Eliot, Nelson, Cailin Anne, Berger, Louisa
• Granted Patent: US 10,031,931 B2
• Time in Patent Office: Days
• CPC Class Codes: G06F 16/211 Schema design and management
• CPC Class Codes: G06F 16/2228 Indexing structures
• CPC Class Codes: G06F 16/275 Synchronous replication
• CPC Class Codes: H04L 9/0891 Revocation or update of secret information, e.g. encryption key update or rekeying

Abstract

An automation system is provided to automate any administrative task in a distributed database, such that the end user can input a goal state (e.g., create database with a five node architecture) and the automation system generates and executes a plan to achieve the goal state without further user input. According to another aspect, bringing existing database systems into automated management can be as complex as designing the database itself. According to some embodiments, the automation system is configured to analyze existing database systems, capture and/or install monitoring components within the existing database, and generate execution pathways to integrate existing database systems into automation control systems. Based on the current state information, the automation system is configured to generate an installation pathway of one or more intermediate states to transition the existing system from no automation to a goal state having active automation agents distributed throughout the database.

Claims

1. A system for automatically integrating cloud based resources into a distributed database executing on local hardware, the system comprising:

at least one processor operatively connected to a memory;

a state component, executed by the at least one processor, configured to determine a current state for a plurality of database nodes, wherein the current state determination identifies a database instance executing on client hardware including at least one replica set having a primary node hosting a primary database instance that accepts database write operations from client systems, and at least two secondary nodes that host copies of the primary database instance that replicate logged write operations from the primary node, a database version executing on the replica set, and configuration data for at least one database instance;

a planning component, executed by the at least one processor, configured to generate an execution plan to provision an additional secondary node mirroring the primary node and the primary database; and

an automation component, executed by the at least one processor, configured to;

provision at least one cloud resource having processor and memory based on the execution plan;

install database binary files including automation agents for a secondary node on the at least one cloud resource;

trigger synchronization between the primary node and the secondary node executing on the at least one cloud resource; and

update configuration metadata for the database instance; and

replicate write operations on the primary node executing on client hardware received after synchronization to the secondary node executing on the at least one cloud resource.

2. The system of claim 1, wherein the secondary node executing on the at least one cloud resource is configured to query an operation log on the primary node executing on the client hardware.

3. The system of claim 2, wherein the secondary node captures new operations from the primary'"'"'s operation log, and replicates the operations on its copy of the database data.

4. The system of claim 1, wherein the system is configured to:

update the configuration metadata for the database instance; and

direct secondary read requests to any secondary nodes in the replica set include the secondary node executing on the at least one cloud resource.

5. The system of claim 1, wherein the system is configured to trigger provisioning of the secondary node on the least one cloud resource responsive to a user interface command to mirror the database.

6. The system of claim 1, wherein the system is configured to move processing from the original nodes over to the newly instantiated nodes

7. The system of claim 6, wherein the secondary node executing on the at least one cloud resource assumes a primary role for the replica set.

8. The system of claim 7, wherein the system is configured to monitor database performance with the primary node executing on the at least one cloud resource.

9. The system of claim 8, wherein the planning component is configured to generate an execution plan to return database processing to a primary node executing on the client hardware.

10. The system of claim 9, wherein the system is configured to generate a user interface tool, and the user interface tool is configured to trigger the automation agent to execute the execution plan to return the database processing to the primary node executing on the client hardware responsive to user selection.

11. The system of claim 1, wherein the planning component is configured to generate an execution plan with a goal state including a plurality of cloud resources hosting mirrored nodes for each node in the replica set executing on the client hardware.

12. The system of claim 1, wherein the at least one replica is configured to manage a dynamic schema database instance.

18. The method of claim 3, further comprising monitoring database performance with the primary node executing on the at least one cloud resource.

13. A computer implemented method for automatically integrating cloud based resources into a distributed database executing on local hardware, the method comprising:

determining, by at least one processor, a current state for a plurality of database nodes, wherein determining the current state includes identifying a database instance executing on client hardware including at least one replica set having a primary node hosting a primary database instance that accepts database write operations from client systems, and at least two secondary nodes that host copies of the primary database instance that replicate logged write operations from the primary node, a database version executing on the replica set, and configuration data for at least one database instance;

generating, by the at least one processor, an execution plan to provision an additional secondary node mirroring the primary node and the primary database;

provisioning, by the at least one processor, at least one cloud resource having processor and memory based on the execution plan;

installing, by the at least one processor, database binary files including automation agents for a secondary node on the at least one cloud resource;

triggering, by the at least one processor, synchronization between the primary node and the secondary node executing on the at least one cloud resource;

updating, by the at least one processor, configuration metadata for the database instance; and

replicating, by the at least one processor, any write operations receive on the primary node executing on client hardware after synchronization to the secondary node executing on the at least one cloud resource.

14. The method of claim 13, wherein replicating includes querying, by the secondary node executing on the at least one cloud resource, an operation log on the primary node executing on the client hardware.

15. The method of claim 14, further comprising capturing new operations from the primary'"'"'s operation log, and replicating the new operations on the secondary node'"'"'s copy of the database data.

16. The method of claim 13, wherein the act of provisioning is executed responsive to receiving a user interface command to mirror the database.

17. The method of claim 13, further comprising an act of moving processing from the original nodes over to the newly instantiated node.

19. The method of claim 13, further comprising generating an execution plan with a goal state including a plurality of cloud resources hosting mirrored nodes for each node in the replica set executing on the client hardware.

20. The method of claim 13, further comprising generating a user interface tool, wherein the user interface tool is configured to trigger an automation agent to execute an execution plan to return the database processing to the client hardware responsive to user selection.