METHOD, APPARATUS AND SYSTEM FOR REAL-TIME OPTIMIZATION OF COMPUTER-IMPLEMENTED APPLICATION OPERATIONS USING MACHINE LEARNING TECHNIQUES

• US 20190312800A1
• Filed: 06/09/2019
• Published: 10/10/2019
• Est. Priority Date: 07/27/2015
• Status: Active Grant

• Current Assignee: Cisco Technology, Inc. (Cisco Systems, Inc.)
• Original Assignee: Datagrid Systems, Inc. (Cisco Systems, Inc.)
• Inventors: Schibler, Ross, Nickolov, Peter, Quintero, Stephen
• Granted Patent: US 10,756,995 B2
• Time in Patent Office: Days
• CPC Class Codes: G06F 2009/45591 Monitoring or debugging support
• CPC Class Codes: G06F 8/65 Updates security arrangements therefor G06F21/57
• CPC Class Codes: G06F 8/658 Incremental updates; Differential updates
• CPC Class Codes: G06F 9/44505 Configuring for program initiating, e.g. using registry, configuration files
• CPC Class Codes: G06F 9/45558 Hypervisor-specific management and integration aspects
• CPC Class Codes: H04L 41/082 the condition being updates or upgrades of network functionality
• CPC Class Codes: H04L 41/0823 characterised by the purposes of a change of settings, e.g. optimising configuration for enhancing reliability for optimising operational conditions of wireless networks H04W24/02
• CPC Class Codes: H04L 41/0869 Validating the configuration within one network element
• CPC Class Codes: H04L 41/16 using machine learning or artificial intelligence
• CPC Class Codes: H04L 43/0817 by checking functioning
• CPC Class Codes: H04L 63/1433 Vulnerability analysis

Abstract

Various aspects described or referenced herein are directed to different methods, systems, and computer program products for facilitating real-time optimization of computer-implemented application operations using machine learning techniques.

Claims

1. A computer implemented method for optimizing a mutable runtime configuration of a first application hosted at a remote networked environment, the remote networked environment being communicatively coupled to a computer network, the computer network including an Optimizer System, the Optimizer System including a first set of optimizer algorithms, the method comprising causing at least one processor to execute instructions stored in non-transient memory to:

cause at least one network device to initiate a first measurement of a first operational metric of the first application while the first application is operating in accordance with a first runtime configuration;

cause the at least one network device to transmit first measurement information to the Optimizer System, where the first measurement information relates to the first measurement of the first operational metric of the first application;

calculate, using the first measurement information, a first score in relation to a first optimization objective, the first score being calculated using a first scoring function;

determine, at the Optimizer System, a first set of updated application settings relating to the mutable runtime configuration of the first application;

cause, using the at least one network device, the first set of updated application settings to be deployed at the first application to thereby cause the first application to operate in accordance with a second runtime configuration;

cause the at least one network device to initiate a second measurement of the first operational metric of the first application while the first application is operating in accordance with the second runtime configuration;

cause the at least one network device to transmit second measurement information to the Optimizer System, where the second measurement information relates to the second measurement of the first operational metric of the first application;

calculate, using the second measurement information, a second score in relation to the first optimization objective, the second score being calculated using the first scoring function;

compute, using at least the second score, a first reward;

update the first set of optimization algorithms using information relating to the first reward;

select, from the first set of optimization algorithms, a first optimization algorithm to be used for determining a second set of updated application settings relating to the mutable runtime configuration of the first application;

determine, using the first optimization algorithm, a second set of updated application settings relating to the mutable runtime configuration of the first application;

cause, using the at least one network device, the second set of updated application settings to be deployed at the first application to thereby cause the first application to operate in accordance with a third runtime configuration;

cause the at least one network device to initiate a third measurement of the first operational metric of the first application while the first application is operating in accordance with the third runtime configuration;

cause the at least one network device to transmit third measurement information to the Optimizer System, where the third measurement information relates to the third measurement of the first operational metric of the first application;

calculate, using the third measurement information, a third score in relation to the first optimization objective, the third score being calculated using the first scoring function;

compute, using at least the third score, a second reward;

update the first set of optimization algorithms using information relating to the second reward;

select, from the first set of optimization algorithms, a second optimization algorithm to be used for determining a third set of updated application settings relating to the mutable runtime configuration of the first application;

determine, using the second optimization algorithm, a third set of updated application settings relating to the mutable runtime configuration of the first application;

cause, using the at least one network device, the third set of updated application settings to be deployed at the first application to thereby cause the first application to operate in accordance with a fourth runtime configuration; and

determine, at the Optimizer System, if additional cycles of optimization adjustment are to be performed for the first application.

2. The computer implemented method of claim 1 further comprising causing, if it is determined that additional cycles of optimization adjustment are to be performed for the first application, the at least one processor to execute additional instructions to:

cause the at least one network device to initiate a fourth measurement of the first operational metric of the first application while the first application is operating in accordance with the fourth runtime configuration;

cause the at least one network device to transmit forth measurement information to the Optimizer System, where the fourth measurement information relates to the fourth measurement of the first operational metric of the first application;

calculate, using the fourth measurement information, a fourth score in relation to the first optimization objective, the fourth score being calculated using the first scoring function;

compute, using at least the fourth score, a third reward;

update the first set of optimization algorithms using information relating to the third reward;

select, from the first set of optimization algorithms, a third optimization algorithm to be used for determining a fourth set of updated application settings relating to the mutable runtime configuration of the first application;

determine, using the third optimization algorithm, a fourth set of updated application settings relating to the mutable runtime configuration of the first application; and

cause, using the at least one network device, the fourth set of updated application settings to be deployed at the first application to thereby cause the first application to operate in accordance with a fifth runtime configuration.

3. The computer implemented method of claim 1 wherein the at least one network component includes a servo component deployed at the remote networked environment and configured or designed to implement instructions received from the Optimizer System, and to initiate interactions with the first application in response to the received instructions.

4. The computer implemented method of claim 1 wherein the at least one network component includes a servo component deployed at the Optimizer System and configured or designed to implement instructions generated by the Optimizer System and to initiate interactions with the first application in response to the instructions.

5. The computer implemented method of claim 1 further comprising causing the at least one processor to execute additional instructions to:

calculate, using the first measurement information, a first performance indicator of the first application, the first performance indicator being representative of a first performance of the first application while operating in accordance with the first runtime configuration;

calculate, using information relating to the first runtime configuration, a first cost indicator of the first application, the first cost indicator being representative of a first cost of resources utilized for operating the first application in accordance with the first runtime configuration;

wherein the first score is calculated using the first performance indicator and first cost indicator;

calculate, using the second measurement information, a second performance indicator of the first application, the second performance indicator being representative of a second performance of the first application while operating in accordance with the second runtime configuration;

calculate, using information relating to the second runtime configuration, a second cost indicator of the first application, the second cost indicator being representative of a second cost of resources utilized for operating the first application in accordance with the second runtime configuration; and

wherein the second score is calculated using the second performance indicator and second cost indicator.

6. The computer implemented method of claim 1 wherein the first reward corresponds to the second score.

7. The computer implemented method of claim 1 wherein the first reward is calculated based on a comparison of the second score and the first score.

8. The computer implemented method of claim 1 further comprising causing the at least one processor to execute additional instructions to:

calculate, using the first measurement information, a first performance measurement of the first application;

calculate, using information relating to the first runtime configuration, a first cost of the application;

wherein the first score is calculated using the first performance measurement and first cost; and

wherein the first scoring function corresponds to a scoring function selected from a group consisting of;

performance measurement/cost;

performance measurement W1/cost, where W1 represents a weighted value;

performance measurement, where cost is represented as constant;

performance measurement bounded by a maximum cost; and

cost while maintaining a minimum performance measurement value.

9. The computer implemented method of claim 1:

wherein at least one set of updated application settings is selected from a group consisting of;

at least one virtual machine associated with the first application;

at least one container associated with the first application;

at least one CPU core associated with the first application;

at least one memory associated with the first application;

network bandwidth associated with the first application;

at least one provisioned disk TOPS associated with the first application;

at least one resource setting associated with the first application; and

number of replicas of a component deployed at the first application.

10. The computer implemented method of claim 1:

wherein at least one set of updated application settings is selected from a group consisting of;

the number of Apache worker threads associated with the first application;

My SQL memory pool size associated with the first application;

kernel tuning parameters associated with the first application;

number of virtualized components of the first application which are co-located on a same physical machine; and

number of virtualized container components of the first application which are co-located on a same host.

11. The computer implemented method of claim 1 wherein at least one selected optimization algorithm corresponds to a reinforced learning algorithm configured or designed to employ Q-learning using a neural network as a Q function.

12. The computer implemented method of claim 1:

wherein the first optimization algorithm corresponds to a first type of optimization algorithm selected from a group consisting of;

a reinforced learning algorithm configured or designed to employ Q-learning using a neural network as a Q function, a Bayesian algorithm, an Evolutionary algorithm, an Ouch heuristic algorithm, a Stochastic algorithm, and a Bruteforce algorithm;

wherein the second optimization algorithm corresponds to a second type of optimization algorithm selected from a group consisting of;

a reinforced learning algorithm configured or designed to employ Q-learning using a neural network as a Q function, a Bayesian algorithm, an Evolutionary algorithm, an Ouch heuristic algorithm, a Stochastic algorithm, and a Bruteforce algorithm; and

wherein the first type of optimization algorithm is different from the second type of optimization algorithm.

13. The computer implemented method of claim 1 further comprising causing the at least one processor to execute instructions to cause at least one set of updated application settings to be deployed at the first application while the first application is running in a live production environment.

14. The computer implemented method of claim 1 further comprising causing the at least one processor to execute instructions to cause at least one set of updated application settings to be deployed at the first application while the first application is running in a test bed environment.

15. The computer implemented method of claim 1 further comprising causing the at least one processor to execute instructions to cause at least one set of updated application settings to be deployed at the first application while the first application is running in a canary environment.

16. The computer implemented method of claim 1 wherein the method is implemented as a SaaS service which is configured or designed to utilize the Optimizer System to remotely and securely optimize the first application.

17. A computer implemented system for optimizing a mutable runtime configuration of a first application hosted at a remote networked environment, the remote networked environment being communicatively coupled to a computer network, the computer network including an Optimizer System, the Optimizer System including a first set of optimizer algorithms, the system comprising:

at least one non-transient memory;

at least one processor operable to execute a plurality of instructions stored in the at least one non-transient memory to;

cause at least one network device to initiate a first measurement of a first operational metric of the first application while the first application is operating in accordance with a first runtime configuration;

cause the at least one network device to transmit first measurement information to the Optimizer System, where the first measurement information relates to the first measurement of the first operational metric of the first application;

calculate, using the first measurement information, a first score in relation to a first optimization objective, the first score being calculated using a first scoring function;

determine, at the Optimizer System, a first set of updated application settings relating to the mutable runtime configuration of the first application;

cause, using the at least one network device, the first set of updated application settings to be deployed at the first application to thereby cause the first application to operate in accordance with a second runtime configuration;

cause the at least one network device to initiate a second measurement of the first operational metric of the first application while the first application is operating in accordance with the second runtime configuration;

cause the at least one network device to transmit second measurement information to the Optimizer System, where the second measurement information relates to the second measurement of the first operational metric of the first application;

calculate, using the second measurement information, a second score in relation to the first optimization objective, the second score being calculated using the first scoring function;

compute, using the second and first scores, a first reward;

update the first set of optimization algorithms using information relating to the first reward;

select, from the first set of optimization algorithms, a first optimization algorithm to be used for determining a second set of updated application settings relating to the mutable runtime configuration of the first application;

determine, using the first optimization algorithm, a second set of updated application settings relating to the mutable runtime configuration of the first application;

cause, using the at least one network device, the second set of updated application settings to be deployed at the first application to thereby cause the first application to operate in accordance with a third runtime configuration;

cause the at least one network device to initiate a third measurement of the first operational metric of the first application while the first application is operating in accordance with the third runtime configuration;

cause the at least one network device to transmit third measurement information to the Optimizer System, where the third measurement information relates to the third measurement of the first operational metric of the first application;

calculate, using the third measurement information, a third score in relation to the first optimization objective, the third score being calculated using the first scoring function;

compute, using the second and third scores, a second reward;

update the first set of optimization algorithms using information relating to the second reward;

select, from the first set of optimization algorithms, a second optimization algorithm to be used for determining a third set of updated application settings relating to the mutable runtime configuration of the first application;

determine, using the second optimization algorithm, a third set of updated application settings relating to the mutable runtime configuration of the first application;

cause, using the at least one network device, the third set of updated application settings to be deployed at the first application to thereby cause the first application to operate in accordance with a fourth runtime configuration; and

determine, at the Optimizer System, if additional cycles of optimization adjustment are to be performed for the first application.

18. The computer implemented system of claim 17 being further operable to cause, if it is determined that additional cycles of optimization adjustment are to be performed for the first application, the at least one processor to execute additional instructions to:

causing the at least one processor to execute additional instructions to;

cause the at least one network device to initiate a fourth measurement of the first operational metric of the first application while the first application is operating in accordance with the fourth runtime configuration;

cause the at least one network device to transmit forth measurement information to the Optimizer System, where the fourth measurement information relates to the fourth measurement of the first operational metric of the first application;

calculate, using the fourth measurement information, a fourth score in relation to the first optimization objective, the fourth score being calculated using the first scoring function;

compute, using the third and fourth scores, a third reward;

update the first set of optimization algorithms using information relating to the third reward;

select, from the first set of optimization algorithms, a third optimization algorithm to be used for determining a fourth set of updated application settings relating to the mutable runtime configuration of the first application;

determine, using the third optimization algorithm, a fourth set of updated application settings relating to the mutable runtime configuration of the first application; and

cause, using the at least one network device, the fourth set of updated application settings to be deployed at the first application to thereby cause the first application to operate in accordance with a fifth runtime configuration.

19. The computer implemented system of claim 17 wherein the at least one network component includes a servo component deployed at the remote networked environment and configured or designed to implement instructions received from the Optimizer System, and to initiate interactions with the first application in response to the received instructions.

20. The computer implemented system of claim 17 wherein the at least one network component includes a servo component deployed at the Optimizer System and configured or designed to implement instructions generated by the Optimizer System and to initiate interactions with the first application in response to the instructions.

21. The computer implemented system of claim 17 being further operable to cause the at least one processor to execute additional instructions to:

calculate, using the first measurement information, a first performance indicator of the first application, the first performance indicator being representative of a first performance of the first application while operating in accordance with the first runtime configuration;

calculate, using information relating to the first runtime configuration, a first cost indicator of the first application, the first cost indicator being representative of a first cost of resources utilized for operating the first application in accordance with the first runtime configuration;

wherein the first score is calculated using the first performance indicator and first cost indicator;

calculate, using the second measurement information, a second performance indicator of the first application, the second performance indicator being representative of a second performance of the first application while operating in accordance with the second runtime configuration;

calculate, using information relating to the second runtime configuration, a second cost indicator of the first application, the second cost indicator being representative of a second cost of resources utilized for operating the first application in accordance with the second runtime configuration; and

wherein the second score is calculated using the second performance indicator and second cost indicator.

22. The computer implemented system of claim 17 wherein the first reward corresponds to the second score.

23. The computer implemented system of claim 17 wherein the first reward is calculated based on a comparison of the second score and the first score.

24. The computer implemented system of claim 17 being further operable to cause the at least one processor to execute additional instructions to:

calculate, using the first measurement information, a first performance measurement of the first application;

calculate, using information relating to the first runtime configuration, a first cost of the application;

wherein the first score is calculated using the first performance measurement and first cost; and

wherein the first scoring function corresponds to a scoring function selected from a group consisting of;

performance measurement/cost;

performance measurement W1/cost, where W1 represents a weighted value;

performance measurement, where cost is represented as constant;

performance measurement bounded by a maximum cost; and

cost while maintaining a minimum performance measurement value.

25. The computer implemented system of claim 17:

wherein at least one set of updated application settings is selected from a group consisting of;

at least one virtual machine associated with the first application;

at least one container associated with the first application;

at least one CPU core associated with the first application;

at least one memory associated with the first application;

network bandwidth associated with the first application;

at least one provisioned disk TOPS associated with the first application;

at least one resource setting associated with the first application; and

number of replicas of a component deployed at the first application.

26. The computer implemented system of claim 17:

wherein at least one set of updated application settings is selected from a group consisting of;

the number of Apache worker threads associated with the first application;

My SQL memory pool size associated with the first application;

kernel tuning parameters associated with the first application;

number of virtualized components of the first application which are co-located on a same physical machine; and

number of virtualized container components of the first application which are co-located on a same host.

27. The computer implemented system of claim 17 wherein at least one selected optimization algorithm corresponds to a reinforced learning algorithm configured or designed to employ Q-learning using a neural network as a Q function.

28. The computer implemented system of claim 17:

wherein the first optimization algorithm corresponds to a first type of optimization algorithm selected from a group consisting of;

a reinforced learning algorithm configured or designed to employ Q-learning using a neural network as a Q function, a Bayesian algorithm, an Evolutionary algorithm, an Ouch heuristic algorithm, a Stochastic algorithm, and a Bruteforce algorithm;

wherein the second optimization algorithm corresponds to a second type of optimization algorithm selected from a group consisting of;

a reinforced learning algorithm configured or designed to employ Q-learning using a neural network as a Q function, a Bayesian algorithm, an Evolutionary algorithm, an Ouch heuristic algorithm, a Stochastic algorithm, and a Bruteforce algorithm; and

wherein the first type of optimization algorithm is different from the second type of optimization algorithm.

29. The computer implemented system of claim 17 being further operable to cause the at least one processor to execute instructions to cause at least one set of updated application settings to be deployed at the first application while the first application is running in a live production environment.

30. The computer implemented system of claim 17 being further operable to cause the at least one processor to execute instructions to cause at least one set of updated application settings to be deployed at the first application while the first application is running in a test bed environment.

31. The computer implemented system of claim 17 being further operable to cause the at least one processor to execute instructions to cause at least one set of updated application settings to be deployed at the first application while the first application is running in a canary environment.

32. The computer implemented system of claim 17 wherein the system is implemented as a SaaS service which is configured or designed to utilize the Optimizer System to remotely and securely optimize the first application.

33. A non-transitory computer usable medium for use in a computer network, the computer network including a first application hosted at a remote networked environment, and including an Optimizer System configured to access a first set of optimizer algorithms, the computer network further including at least one processor, the computer usable medium having computer readable code embodied therein, the computer readable code comprising computer code for causing at least one processor to execute instructions stored in at least one memory to:

cause at least one network device to initiate a first measurement of a first operational metric of the first application while the first application is operating in accordance with a first runtime configuration;

cause the at least one network device to transmit first measurement information to the Optimizer System, where the first measurement information relates to the first measurement of the first operational metric of the first application;

calculate, using the first measurement information, a first score in relation to a first optimization objective, the first score being calculated using a first scoring function;

determine, at the Optimizer System, a first set of updated application settings relating to the mutable runtime configuration of the first application;

cause, using the at least one network device, the first set of updated application settings to be deployed at the first application to thereby cause the first application to operate in accordance with a second runtime configuration;

cause the at least one network device to initiate a second measurement of the first operational metric of the first application while the first application is operating in accordance with the second runtime configuration;

cause the at least one network device to transmit second measurement information to the Optimizer System, where the second measurement information relates to the second measurement of the first operational metric of the first application;

calculate, using the second measurement information, a second score in relation to the first optimization objective, the second score being calculated using the first scoring function;

compute, using the second and first scores, a first reward;

update the first set of optimization algorithms using information relating to the first reward;

select, from the first set of optimization algorithms, a first optimization algorithm to be used for determining a second set of updated application settings relating to the mutable runtime configuration of the first application;

determine, using the first optimization algorithm, a second set of updated application settings relating to the mutable runtime configuration of the first application;

cause, using the at least one network device, the second set of updated application settings to be deployed at the first application to thereby cause the first application to operate in accordance with a third runtime configuration;

cause the at least one network device to initiate a third measurement of the first operational metric of the first application while the first application is operating in accordance with the third runtime configuration;

cause the at least one network device to transmit third measurement information to the Optimizer System, where the third measurement information relates to the third measurement of the first operational metric of the first application;

calculate, using the third measurement information, a third score in relation to the first optimization objective, the third score being calculated using the first scoring function;

compute, using the second and third scores, a second reward;

update the first set of optimization algorithms using information relating to the second reward;

select, from the first set of optimization algorithms, a second optimization algorithm to be used for determining a third set of updated application settings relating to the mutable runtime configuration of the first application;

determine, using the second optimization algorithm, a third set of updated application settings relating to the mutable runtime configuration of the first application;

cause, using the at least one network device, the third set of updated application settings to be deployed at the first application to thereby cause the first application to operate in accordance with a fourth runtime configuration; and

determine, at the Optimizer System, if additional cycles of optimization adjustment are to be performed for the first application.