System for live-migration and automated recovery of applications in a distributed system

• US 9,483,542 B2
• Filed: 12/17/2013
• Issued: 11/01/2016
• Est. Priority Date: 09/23/2011
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Hybrid Logic Ltd.
• Inventors: Marsden, Luke
• Primary Examiner(s): Robinson, Greta
• Application Number:
  US14/109,428
• Publication Number:
  US 20140108341A1
• Time in Patent Office: 1,050 Days
• Field of Search: 707/615, 707/616, 707/623, 707/633, 707/634, 707/635, 707/637, 707/638, 707/639, 707/674, 707/686, 707/E17.01, 707/703, 707/721, 707/726, 707/732, 707/748, 707/749, 707/751, 707/758, 707/766, 707/767, 707/768, 707/771, 707/777, 707/780, 707/782, 707/802, 707/610, 707/636, 714/1, 714/2, 714/6.12, 714/6.3, 718/105, 706/46, 715/753, 709/224, 709/226
• US Class Current: 1/1
• CPC Class Codes: G06F 16/275 Synchronous replication

Abstract

A method and apparatus for distribution of applications amongst a number of servers, ensuring that changes to application data on a master for that application are asynchronously replicated to a number of slaves for that application. Servers may be located in geographically diverse locations; the invention permits data replication over high-latency and lossy network connections and failure-tolerance under hardware and network failure conditions. Access to applications is mediated by a distributed protocol handler which allows any request for any application to be addressed to any server, and which, when working in tandem with the replication system, pauses connections momentarily to allow seamless, consistent live-migration of applications and their state between servers. Additionally, a system which controls the aforementioned live-migration based on dynamic measurement of load generated by each application and the topological preferences of each application, in order to automatically keep servers at an optimum utilization level.

Claims

1. A method of balancing load in a cluster of servers hosting a plurality of applications, the method comprising:

determining the current load of each server;

determining an average load taking into account the loads at the servers in the cluster;

determining for a server whether its load is less or greater than the average load plus a damping factor (Q);

making a decision to migrate an application from the server when its load is greater than the average plus the damping factor.

2. The method according to claim 1, further comprising prior to the decision to migrate an application from the server, replicating changes in data of an application to a configurable number of servers of the cluster.

3. The method according to any preceding claim, further comprising maintaining a record of the server on which an application is currently hosted, and inspecting the record to determine the server to which an incoming application request is to be directed.

4. The method according to claim 1, further comprising exchanging messages with other servers in the cluster to determine the version of a filesystem of the highest centre of mass metric, based on analysis of snapshots of changes in data of the application which have been received, and wherein the application is migrated to that server having the highest centre of mass metric.

5. The method according to claim 1, further comprising pausing incoming requests for the application and terminating current requests for the application after a predetermined time period.

6. A method according to claim 1, further comprising pausing incoming requests for the application for a predetermined time period and releasing the paused requests if current requests for the application have not completed in the predetermined time period.

7. The method according to according to claim 1, further comprising emitting periodic heart beat signals indicating live presence in the cluster.

8. Computer software which, when executed by appropriate processing means, causes the processing means to implement a method of balancing load in a cluster of servers hosting a plurality of applications, the method comprising:

determining the current load of each server;

determining an average load taking into account the loads at the servers in the cluster;

determining for a server whether its load is less or greater than the average load plus a damping factor (Q);

making a decision to migrate an application from the server when its load is greater than the average plus the damping factor.

9. The computer software according to claim 8, wherein the method further comprises:

prior to the decision to migrate an application from the server, replicating changes in data of an application to a configurable number of servers of the cluster.

10. The computer software according to claim 8, wherein the method further comprises:

maintaining a record of the server on which an application is currently hosted, and inspecting the record to determine the server to which an incoming application request is to be directed.

11. The computer software according to claim 8, wherein the method further comprises:

exchanging messages with other servers in the cluster to determine the version of a filesystem of the highest centre of mass metric, based on analysis of snapshots of changes in data of the application which have been received, and wherein the application is migrated to that server having the highest centre of mass metric.

12. The computer software according to claim 8, wherein the method further comprises:

pausing incoming requests for the application and terminating current requests for the application after a predetermined time period.

13. The computer software according to claim 8, wherein the method further comprises:

pausing incoming requests for the application for a predetermined time period and releasing the paused requests if current requests for the application have not completed in the predetermined time period.

14. The computer software according to claim 8, wherein the method further comprises:

emitting periodic heart beat signals indicating live presence in the cluster.