Detecting parameter validity in code including cross-service calls

• US 10,572,375 B1
• Filed: 02/05/2018
• Issued: 02/25/2020
• Est. Priority Date: 02/05/2018
• Status: Active Grant

• Current Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Original Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Inventors: Wagner, Timothy Allen
• Primary Examiner(s): Ko, Chae M
• Application Number:
  US15/888,540
• Time in Patent Office: 750 Days
• Field of Search: 714 381
• CPC Class Codes: G06F 11/3612 by runtime analysis performance monitoring G06F11/3466
• CPC Class Codes: G06F 11/368 for test version control, e.g. updating test cases to a new software version
• CPC Class Codes: G06F 11/3688 for test execution, e.g. scheduling of test suites
• CPC Class Codes: G06F 16/951 Indexing; Web crawling techniques
• CPC Class Codes: G06F 9/547 Remote procedure calls [RPC]; Web services
• CPC Class Codes: G06N 20/00 Machine learning
• CPC Class Codes: G06N 3/08 Learning methods
• CPC Class Codes: G06N 5/022 Knowledge engineering; Knowledge acquisition

Abstract

Systems and methods are described for conducting static analysis of code invoking network-based services to identify, without requiring execution of the code, errors that may be introduced due to the invocations of the network-based services. A system is provided that may analyze code to detect both direct invocations of services, as well as indirect invocations caused by the direct invocations. The system can model inputs and outputs of directly or indirectly invoked services to identify errors in parameters passed to those services, even when the errors are not apparent from an analysis of the code in isolation. In some instances, the system can traverse a “call graph” of all services invoked by code either directly or indirectly to trace parameter errors through multiple levels of indirection.

Claims

1. A system to analyze code executable on an on-demand code execution system to detect potentially invalid parameters passed to an invoked service, the system comprising:

a physical data store storing executable code submitted to the on-demand code execution system by a user device, the executable code including an invocation of a first network-accessible service;

a computing device in communication with the physical data store and configured to;

obtain the executable code;

identify, independent of execution of the executable code, the invocation within the executable code of the first network-accessible service, wherein the invocation of the first service includes a parameter to be passed to the first network-accessible service;

determine an expected output of the first network-accessible service, the expected output including an invocation of one or more second network-accessible services not referenced within the executable code and an expected parameter to be passed to the one or more second network-accessible services;

obtain parameter evaluation criteria for the one or more second network-accessible services;

compare the expected parameter to the parameter evaluation criteria to determine that the expected parameter does not satisfy the parameter evaluation criteria; and

transmit an indication that the expected parameter does not satisfy the parameter evaluation criteria.

2. The system of claim 1, wherein the invocation within the executable code of the first network-accessible service is an application programming interface (API) call.

3. The system of claim 1, wherein the computing device is further configured to:

obtain service information for the first network-accessible service, the service information including a transformation of an input parameter to an output of the first network-accessible service; and

determine the expected output of the first network-accessible service based on applying the transformation to the parameter to be passed to the first network-accessible service.

4. The system of claim 3, wherein the first network-accessible service is implemented as an execution of second executable code on the on-demand code execution system, and wherein the computing device is configured to determine the transformation based on analysis of the second executable code.

5. The system of claim 3, wherein the analysis of the second executable code is a static analysis conducted without requiring execution of the second executable code.

6. The system of claim 1, wherein the parameter validation criteria for the second network-accessible service includes at least one of a permissible data type for the expected parameter or a permissible range of values for the expected parameter.

7. A computer-implemented method comprising:

obtaining executable code from a user device;

identifying, independent of execution of the executable code, an invocation within the executable code of a first network-accessible service and a parameter to be passed to the first network-accessible service;

determining that an expected output of the first network-accessible service corresponds to an invocation of one or more second network-accessible services;

determining an expected parameter to be passed to the one or more second network-accessible services;

obtaining parameter evaluation criteria for the one or more second network-accessible services;

comparing the expected parameter to the parameter evaluation criteria to determine that the expected parameter does not satisfy the parameter evaluation criteria; and

transmitting an indication that the expected parameter does not satisfy the parameter evaluation criteria.

8. The computer-implemented method of claim 7, wherein the one or more second network-accessible services are not referenced within the executable code.

9. The computer-implemented method of claim 7, wherein identifying the invocation within the executable code comprises identifying a code segment within the executable code that is associated with the first network-accessible service.

10. The computer-implemented method of claim 9, wherein identifying the invocation within the executable code further comprises referencing information mapping the code segment to the first network-accessible service.

11. The computer-implemented method of claim 7, wherein determining that the expected output of the first network-accessible service corresponds to the invocation of the one or more second network-accessible services comprises determining that the expected output of the first network-accessible service is the invocation of the one or more second network-accessible services.

12. The computer-implemented method of claim 7, wherein at least one of the one or more second network-accessible services is implemented as an execution of second executable code on an on-demand code execution system, and wherein determining that the expected output of the first network-accessible service corresponds to the invocation the one or more second network-accessible services comprises determining that the expected output of the first network-accessible service satisfies criteria maintained at the on-demand code execution system for execution of the second executable code.

13. The computer-implemented method of claim 7 further comprising:

determining, from the expected parameter to be passed to the one or more second network-accessible services, an expected output of the one or more second network-accessible services, the expected output of the one or more second network-accessible services including an invocation of a third network-accessible service and an expected parameter to be passed to the third network-accessible service;

obtaining parameter validation criteria for the third network-accessible service;

comparing the expected parameter to be passed to the third network-accessible service to the parameter evaluation criteria for the third network-accessible service to determine that the expected parameter to be passed to the third network-accessible service does not satisfy the parameter evaluation criteria for the third network-accessible service; and

transmitting an indication that the expected parameter to be passed to the third network-accessible service does not satisfy the parameter evaluation criteria for the third network-accessible service.

14. Non-transitory computer-readable media comprising executable instructions that, when executed on a computing system, cause the computing system to:

obtain executable code from a user device;

identify, independent of execution of the executable code, an invocation within the executable code of a first network-accessible service and a parameter to be passed to the first network-accessible service;

determine an expected output of the first network-accessible service, the expected output including an invocation of one or more second network-accessible services and an expected parameter to be passed to the one or more second network-accessible services;

obtain parameter evaluation criteria for the one or more second network-accessible services;

compare the expected parameter to the parameter evaluation criteria to determine that the expected parameter does not satisfy the parameter evaluation criteria; and

transmit an indication that the expected parameter does not satisfy the parameter evaluation criteria.

15. The non-transitory computer-readable media of claim 14, wherein the executable instructions further cause the computing system to:

obtain service information for the first network-accessible service, the service information including a transformation of an input parameter to an output of the first network-accessible service; and

determine the expected output of the first network-accessible service based on applying the transformation to the parameter to be passed to the first network-accessible service.

16. The non-transitory computer-readable media of claim 15, wherein the executable instructions further cause the computing system to:

determine the transformation based on a record of one or more prior input parameters to the first network-accessible service and one or more prior outputs of the first network-accessible service corresponding to the one or more prior input parameters.

17. The non-transitory computer-readable media of claim 15, wherein the executable instructions further cause the computing system to determine the transformation based on application of a machine-learning algorithm to the record.

18. The non-transitory computer-readable media of claim 15, wherein the service information is obtained at the computing system in response to transmission of a query to an application programming interface of the first network-accessible service.

19. The non-transitory computer-readable media of claim 15, wherein the first network-accessible service is implemented as an execution of second executable code on an on-demand code execution system, and wherein the executable instructions further cause the computing system to determine the transformation based on analysis of the second executable code.

20. The non-transitory computer-readable media of claim 15, wherein the indication that the expected parameter does not satisfy the parameter evaluation criteria includes an identification of at least one of the invocation, within the executable code, of the first network-accessible service or the expected output of the first network-accessible service.