System and method for managing a computing cluster

• US 9,338,064 B2
• Filed: 01/17/2013
• Issued: 05/10/2016
• Est. Priority Date: 06/23/2010
• Status: Active Grant

• Current Assignee: Twilio, Inc.
• Original Assignee: Twilio, Inc.
• Inventors: Stratton, Frank, Wilsher, Thomas, Law, Cheuk To
• Primary Examiner(s): Harrell, Robert B
• Application Number:
  US13/743,660
• Publication Number:
  US 20130132553A1
• Time in Patent Office: 1,209 Days
• Field of Search: 709/220
• US Class Current: 1/1
• CPC Class Codes: G06F 11/0709 in a distributed system consisting of a plurality of standalone computer nodes, e.g. clusters, client-server systems
• CPC Class Codes: G06F 11/076 by exceeding a count or rate limit, e.g. word- or bit count limit
• CPC Class Codes: G06F 11/3006 where the computing system is distributed, e.g. networked systems, clusters, multiprocessor systems multiprogramming arrangements G06F9/46; allocation of resources G06F9/50
• CPC Class Codes: G06F 11/3055 Monitoring arrangements for monitoring the status of the computing system or of the computing system component, e.g. monitoring if the computing system is on, off, available, not available error or fault processing without redundancy G06F11/0703; error detection or correction by redundancy in data representation G06F11/08; error detection or correction by redundancy in operation G06F11/14; error detection or correction by redundancy in hardware G06F11/16
• CPC Class Codes: G06F 11/3409 for performance assessment
• CPC Class Codes: G06F 9/5072 Grid computing
• CPC Class Codes: H04L 41/50 Network service management, e.g. ensuring proper service fulfilment according to agreements

Abstract

A system and method that includes managing a computer cluster with a plurality of machines of a first version in an active configuration; adding at least one machine of a second version to the computer cluster; monitoring the operational status of the machine of a second version; and transitioning the computer cluster to use the machine of a second version in an active configuration.

Claims

1. A hardware implemented method for managing a telephony platform computing cluster system, the method comprising:

at the telephony platform computing cluster system,

the cluster system including a telephony platform networked computing cluster and a configuration controller,

the computing cluster including a plurality of telephony call router services, a plurality of telephony load balancer services, a plurality of first version machines of a first version that are in an active configuration, and at least one second version machine of a second version that is in a simulation configuration,

each call router service being communicatively coupled with a dedicated telephony service load balancer service,

at least one load balancer service distributing telephony jobs of the associated telephony call router service to at least one first version machine in the active configuration and at least one second version machine in the simulation configuration,

the at least one first version machine actively performing telephony service operations of at least one dependent telephony service that is managed by the load balancer service,

the at least one second version machine simulating telephony service operations of the at least one dependent telephony service,

the at least one dependent telephony service including two or more of a media processor service, a routing service, a call queue service, a resource database service, and a proxy service;

a first load balancer service distributing a first telephony job of an associated first telephony call router service to a first version machine in the active configuration and a second version machine in the simulation configuration;

the first version machine actively performing at least one telephony service operation for the first telephony job;

the second version machine simulating the at least one telephony service operation for the first telephony job;

the configuration controller monitoring active operational status of the actively performed at least one telephony service operation and simulated operational status of the simulated at least one telephony service operation;

the configuration controller determining whether the simulated operational status satisfies a joining condition based on a comparison of the monitored simulated operational status with the monitored active operational status; and

responsive to a determination that the simulated operational status satisfies the joining condition, the configuration controller transitioning configuration of the second version machine from the simulation configuration to the active configuration, such that the second version machine actively performs the at least one telephony service operation for the computing cluster while the first version machine continues to actively perform the at least one telephony service operation for the computing cluster.

2. The method of claim 1, wherein the first version machine is a service with a first version number that is running on a host, and the second version machine is a service with version number different from the first version number that is running on a host.

3. The method of claim 2, wherein the second version is a version number greater than the version number of the first version.

4. The method of claim 2, wherein the second version is a version number less than the version number of the first version.

5. The method of claim 1, further comprising the simulation configuration limiting the second version machine from impacting a computer cluster component in an active configuration.

6. The method of claim 5, wherein the simulation configuration comprises coupling communication to the second version machine with communication to the first version machine.

7. The method of claim 1, wherein the simulation configuration limits output from the second version machine from impacting a machine in an active configuration, wherein limiting output from the second version machine from impacting a machine in an active configuration comprises the second version machine writing to a test database.

8. The method of claim 7, wherein monitoring the simulated operational status comprises analyzing the test database.

9. The method of claim 1,

wherein the telephony platform computing cluster system is a hardware telephony platform computing cluster system,

wherein the telephony platform networked computing cluster is a hardware telephony platform networked computing cluster, the hardware telephony platform networked computing cluster including at least one non-transitory computer-readable storage medium that stores computer-readable instructions that when executed control the computing cluster to perform the monitoring, the determining, and the transitioning,

wherein the configuration controller is a hardware configuration controller, the hardware configuration controller including at least one non-transitory computer-readable storage medium that stores computer-readable instructions that when executed control the configuration controller to perform:

the distributing of the first telephony job;

the active performance of the at least one telephony service operation, and the simulation of the at least one telephony service operation, and

and wherein the hardware configuration controller is communicatively coupled with the hardware computing cluster.

10. The method of claim 9, wherein each call router service includes a call router Application Programming Interface (API) that is used for modifying call router resources to alter state of the call router service and for interacting with media of the call router service, wherein the call router API is a REST API.

11. The method of claim 1, wherein monitoring the simulated operational status comprises monitoring performance of the second version machine during simulation of the at least one telephony service operation.

12. The method of claim 1, wherein monitoring simulated operational status comprises monitoring errors of the second version machine in comparison to the first version machine.

13. The method of claim 1, further comprising the first version machine writing data to a production database and the second version machine writing data to a test database, wherein comparison of the monitored simulated operational status with the monitored active operational status comprises comparing the production database and the test database.

14. The method of claim 1,

wherein simulated operational status includes actions by the second version machine,

wherein actions by the second version machine include at least one of simulated response communications and simulated requests,

wherein active operational status includes actions by the first version machine,

wherein actions by the first version machine include at least one of active response communications and active requests, and

wherein determining whether the simulated operational status satisfies a joining condition comprises at least one of:

comparing a simulated response communication of the second version machine with a corresponding active response communication of the first version machine, and

comparing a simulated request of the second version machine with a corresponding active request of the first version machine.

15. A hardware telephony platform computing cluster system, the telephony platform computing cluster system comprising:

a configuration controller; and

a telephony platform networked computing cluster that includes a plurality of telephony call router services, a plurality of telephony load balancer services, a plurality of first version machines of a first version that are in an active configuration, and at least one second version machine of a second version that is in a simulation configuration,

each call router service being communicatively coupled with a dedicated telephony service load balancer service,

at least one load balancer service distributing telephony jobs of the associated telephony call router service to at least one first version machine in the active configuration and at least one second version machine in the simulation configuration,

the at least one first version machine actively performing telephony service operations of at least one dependent telephony service that is managed by the load balancer service,

the at least one second version machine simulating telephony service operations of the at least one dependent telephony service,

the at least one dependent telephony service including two or more of a media processor service, a routing service, a call queue service, a resource database service, and a proxy service,

wherein the telephony platform computing cluster system includes at least one non-transitory computer-readable storage medium that stores computer-readable instructions that when executed perform processes including;

controlling a first load balancer service to distribute a first telephony job of an associated first telephony call router service to a first version machine in the active configuration and a second version machine in the simulation configuration,

controlling the first version machine to actively perform at least one telephony service operation for the first telephony job,

controlling the second version machine to simulate the at least one telephony service operation for the first telephony job,

controlling the configuration controller to monitor active operational status of the actively performed at least one telephony service operation and simulated operational status of the simulated at least one telephony service operation,

controlling the configuration controller to determine whether the simulated operational status satisfies a joining condition based on a comparison of the monitored simulated operational status with the monitored active operational status, and

responsive to a determination that the simulated operational status satisfies the joining condition, controlling the configuration controller to transition configuration of the second version machine from the simulation configuration to the active configuration, such that the second version machine actively performs the at least one telephony service operation for the computing cluster while the first version machine continues to actively perform the at least one telephony service operation for the computing cluster.

16. A non-transitory computer-readable storage medium that stores computer-readable instructions, the non-transitory computer-readable storage medium comprising:

instructions for managing a hardware telephony platform computing cluster system, the telephony platform computing cluster system including;

a configuration controller; and

a telephony platform networked computing cluster that includes a plurality of telephony call router services, a plurality of telephony load balancer services, a plurality of first version machines of a first version that are in an active configuration, and at least one second version machine of a second version that is in a simulation configuration,

each call router service being communicatively coupled with a dedicated telephony service load balancer service,

at least one load balancer service distributing telephony jobs of the associated telephony call router service to at least one first version machine in the active configuration and at least one second version machine in the simulation configuration,

the at least one first version machine actively performing telephony service operations of at least one dependent telephony service that is managed by the load balancer service,

the at least one second version machine simulating telephony service operations of the at least one dependent telephony service,

the at least one dependent telephony service including two or more of a media processor service, a routing service, a call queue service, a resource database service, and a proxy service,

wherein the instructions for managing the hardware telephony platform computing cluster system include instructions that when executed perform processes including;

controlling a first load balancer service to distribute a first telephony job of an associated first telephony call router service to a first version machine in the active configuration and a second version machine in the simulation configuration,

controlling the first version machine to actively perform at least one telephony service operation for the first telephony job,

controlling the second version machine to simulate the at least one telephony service operation for the first telephony job,

controlling the configuration controller to monitor active operational status of the actively performed at least one telephony service operation and simulated operational status of the simulated at least one telephony service operation,

controlling the configuration controller to determine whether the simulated operational status satisfies a joining condition based on a comparison of the monitored simulated operational status with the monitored active operational status, and

responsive to a determination that the simulated operational status satisfies the joining condition, controlling the configuration controller to transition configuration of the second version machine from the simulation configuration to the active configuration, such that the second version machine actively performs the at least one telephony service operation for the computing cluster while the first version machine continues to actively perform the at least one telephony service operation for the computing cluster.