Optimization of production systems

• US 10,182,128 B1
• Filed: 03/12/2013
• Issued: 01/15/2019
• Est. Priority Date: 02/07/2013
• Status: Active Grant

• Current Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Original Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Inventors: Allocca, William W, Bodley, James Cade, Cherukuri, Ram, Sivakrishnan, Nitin, Telang, Nandan Ulhas
• Primary Examiner(s): Tiv, Backhean
• Assistant Examiner(s): Doan, Tan
• Application Number:
  US13/797,641
• Time in Patent Office: 2,135 Days
• Field of Search: 709201
• CPC Class Codes: H04L 67/564 Enhancement of application control based on intercepted application data
• CPC Class Codes: H04L 67/566 Grouping or aggregating service requests, e.g. for unified processing

Abstract

The techniques described herein provide software testing of a candidate version of software. In some examples, an interceptor may perform an asynchronous and non-blocking intercept of at least one request to a production system and may provide the intercepted at least one request to a shadow proxy service as at least one shadow request. The shadow proxy service may process the shadow request by causing the at least one shadow request to be classified based at least one criteria. The processed shadow requests are logged. A selection may be made from among the logged shadow request to be replayed by the production system based on one or more criteria used to classify the shadow requests. The shadow proxy service may replay the selected at least one logged shadow request to the production system.

Claims

1. A computer-implemented method for use with a production system having first computing resources and second computing resources, the method comprising:

intercepting, asynchronously and without blocking, at least some production requests directed to the production system while the first computing resources are in an on-line mode of operation to generate intercepted production requests, a production request being a request for production system data from the production system;

logging the intercepted production requests as logged shadow requests;

selecting at least some of the logged shadow requests as selected shadow requests;

requesting the second computing resources to process, while in an off-line mode of operation and without modifying the production system data, the selected shadow requests; and

requesting the second computing resources to process additional shadow requests for replay as replayed shadow requests until one or more conditions regarding the replayed shadow requests is satisfied, wherein the replayed shadow requests include a mix of use cases to control a warm-up duration for the second computing resources;

wherein the selected shadow requests prepare a cache of the second computing resources to reduce start-up delay of the second computing resources for a subsequent on-line mode of operation.

2. The computer-implemented method as recited in claim 1, further comprising:

determining an aggregate mix of use cases of software of the production system exercised by the logged shadow requests,

wherein the requesting the second computing resources to process additional shadow requests is performed at least until at least one condition regarding the aggregate mix of use cases of the software exercised by the replayed shadow requests is satisfied.

3. The computer-implemented method as recited in claim 1, wherein the production system comprises a plurality of computing resources, and the computer-implemented method further comprising:

requesting the production system to process the additional shadow requests until an indication regarding a cache of at least one computing resource indicates the at least one computing resource of the plurality of computing resources has cache contents related to production requests received by at least one other computing resource.

4. The computer-implemented method as recited in claim 1, wherein the production system comprises a plurality of computing resources, the computer-implemented method further comprising:

requesting the production system to process the additional shadow requests until an indication regarding a cache of at least one computing resource of the plurality of computing resources indicates the at least one computing resource has cache contents related to predicted production requests related to an upcoming event.

5. The computer-implemented method as recited in claim 1, further comprising:

classifying the logged shadow requests based on one or more criteria, the one or more criteria including at least a time of day criterion; and

wherein selecting the at least some of the logged shadow requests is based on the time of day criterion.

6. The computer-implemented method as recited in claim 1, wherein the logged shadow requests are:

stored in a data store in which each logged shadow request stored in the data store is labeled with a unique identification (ID) and that utilizes an index to the unique ID to retrieve logged shadow requests.

7. The computer-implemented method as recited in claim 1, wherein:

the production system comprises a plurality of computing resources; and

requesting the production system to process the additional shadow requests continues until an indication regarding a cache of at least one of the computing resources indicates the at least one of the computing resources has cache contents related to production requests received at a time of day similar to a previous time of day.

8. The computer-implemented method as recited in claim 1, wherein:

the production system comprises a plurality of computing resources; and

requesting the production system to process the additional shadow requests continues until an indication regarding a cache of at least one of the computing resources indicates the at least one of the computing resources has cache contents related to predicted production requests related to a predicted upcoming surge in a type of production request traffic.

9. The computer-implemented method as recited in claim 1, further comprising:

classifying the logged shadow requests based on one or more criteria, the one or more criteria including at least a type of event criterion; and

wherein selecting the at least some of the logged shadow requests is based at least on the type of event criterion.

10. A computer-implemented method comprising:

intercepting, asynchronously and without blocking, at least some production requests directed to a production system to generate intercepted production requests, the production system comprising a plurality of production machines, while a first production machine of the production system is in an on-line mode of operation, a production request being a request for production system data from the production system;

logging the intercepted production requests as logged shadow requests;

selecting at least some of the logged shadow requests as selected shadow requests;

requesting a second production machine of the production system to process the selected shadow requests while in an off-line mode of operation and without modifying the production system data; and

requesting the second production machine to process additional shadow requests until one or more conditions regarding the additional shadow requests is satisfied, wherein the additional shadow requests include a mix of use cases to control a warm-up duration for the second production machine; and

wherein the selected shadow requests prepare a cache of the second production machine to reduce start-up delay of the second production machine for a subsequent on-line mode of operation.

11. The computer-implemented method as recited in claim 10, further comprising:

determining an aggregate mix of use cases of software exercised by the selected shadow requests;

wherein the requesting the production system to process additional shadow requests is performed at least until at least one condition regarding the aggregate mix of use cases of the software exercised by the selected shadow requests is satisfied.

12. The computer-implemented method as recited in claim 10, wherein:

requesting the second production machine to process the additional shadow requests continues until an indication regarding the cache of the second production machine indicates the second production machine has cache contents related to production requests being received by at least one other production machine.

13. The computer-implemented method as recited in claim 10, wherein:

requesting the second production machine to process the additional shadow requests continues until an indication regarding the cache of the second production machine indicates that the second production machine has cache contents related to predicted production requests related to an upcoming event.

14. The computer-implemented method as recited in claim 10, further comprising:

classifying the logged shadow requests based on one or more classification criteria, wherein the one or more classification criteria include at least a time of day criterion; and

wherein selecting the at least some of the logged shadow requests is based at least on the time of day criterion.

15. The computer-implemented method as recited in claim 10, wherein the logged shadow requests are stored in a data store in which each logged shadow request stored in the data store is labeled with a unique identification (ID) and that utilizes an index to the unique ID to retrieve logged shadow requests.

16. The computer-implemented method as recited in claim 10, wherein:

requesting the second production machine to process the additional shadow requests continues until an indication regarding the cache of the second production machine indicates the second production machine has cache contents related to production requests received at a time of day similar to a previous time of day.

17. The computer-implemented method as recited in claim 10, wherein:

requesting the second production machine to process the additional shadow requests continues until an indication regarding the cache of the second production machine indicates that the second production machine has cache contents related to predicted production requests related to a predicted upcoming surge in a type of production request traffic.

18. The computer-implemented method as recited in claim 10, further comprising:

classifying the logged shadow requests based on one or more criteria, wherein the one or more criteria include at least a type of event criterion; and

wherein selecting the at least some of the logged shadow requests is based on one or more of the one or more criteria used to classify the logged shadow requests.

19. A shadow proxy system for use with a production system having first computing resources and second computing resources, comprising:

an interceptor, comprising a hardware processor and computer-readable media which contains operating instructions for the hardware processor, that;

performs an asynchronous and non-blocking intercept of at least some production requests to the production system while the first computing resources are in an on-line mode of operation to generate intercepted production requests, a production request being a request for data from the production system;

provides the intercepted production requests to the first computing resources while the first computing resources are in the on-line mode of operation; and

provides the intercepted production requests to a shadow proxy service as shadow requests; and

the shadow proxy service, comprising a hardware processor and computer-readable media which contains operating instructions for the processor, the hardware processor and computer-readable media of the shadow proxy service not being the hardware processor and computer-readable media of the interceptor, that;

logs the shadow requests as logged shadow requests;

selects at least some of the logged shadow requests as selected shadow requests; and

sends the selected shadow requests to the second computing resources for replay as replayed shadow requests by the second computing resources while the second computing resources are in an off-line mode of operation;

wherein the shadow proxy service selects and sends additional logged shadow requests to the second computing resources for replay as replayed shadow requests until one or more conditions regarding the replayed shadow requests is satisfied, and the replayed shadow requests include a mix of use cases to control a warm-up duration for the second computing resources; and

wherein the selected shadow requests sent to the second computing resources prepare a cache of the second computing resources to reduce start-up delay of the second computing resources for a subsequent on-line mode of operation.

20. The shadow proxy system as recited in claim 19, wherein:

each logged shadow request includes a time of day;

the shadow proxy service classifies the shadow requests based on classification criteria;

the classification criteria include a time of day criterion; and

the shadow proxy service selects the at least some of the logged shadow requests based at least in part on a time of day of the logged shadow requests.

21. The shadow proxy system as recited in claim 19, wherein:

the shadow proxy service processes the shadow requests by classifying the shadow requests based on classification criteria;

the classification criteria include a type of event criterion; and

the shadow proxy service selects the at least some of the logged shadow requests based at least in part on a type of event of the logged shadow requests.

22. A shadow proxy system for use with a production system comprising a plurality of production machines, the shadow proxy system comprising:

a hardware processor and computer-readable media which contains operating instructions for the hardware processor that causes the shadow proxy system to;

perform an asynchronous and non-blocking intercept of at least some production requests to the production system while a first production machine of the plurality of production machines is in an on-line mode of operation to generate intercepted production requests, a production request being a request for data from the production system;

log the intercepted production requests as logged shadow requests;

select at least some of the logged shadow requests as selected shadow requests;

send the selected shadow requests to the production system for replay as replayed shadow requests by a second production machine of the plurality of production machines while the second production machine is in an off-line mode of operation; and

request the production system to process additional shadow requests for replay as additional replayed shadow requests by the second production machine until one or more conditions regarding the additional replayed shadow requests is satisfied, wherein the additional replayed shadow requests include a mix of use cases to control a warm-up duration for the second production machine;

wherein the selected shadow requests prepare a cache of the second production machine to reduce start-up delay of the second production machine for a subsequent on-line mode of operation.

23. The shadow proxy system as recited in claim 22, wherein:

each logged shadow request has a time of day;

the shadow proxy system classifies the shadow requests based on classification criteria;

the classification criteria include a time of day criterion; and

the shadow proxy system selects the at least some of the logged shadow requests based at least in part on a time of day of the logged shadow requests.

24. The shadow proxy system as recited in claim 22, wherein:

the shadow proxy system classifies the shadow requests based on classification criteria;

the classification criteria include a type of event criterion; and

the shadow proxy system selects the at least some of the logged shadow requests based at least in part on a type of event of the logged shadow requests.

25. The shadow proxy system as recited in claim 22, wherein the selected shadow requests prepare the cache of the second production machine for the on-line mode of operation by a predetermined event.

26. The shadow proxy system as recited in claim 25, wherein the predetermined event is a time of day.

27. The shadow proxy system as recited in claim 25, wherein the predetermined event is an expected next surge of production requests.

28. The shadow proxy system as recited in claim 25, wherein the predetermined event is expected event-specific traffic.