Automatic verification and anomaly detection in a representational state transfer (REST) application programming interface

• US 8,745,641 B1
• Filed: 07/14/2011
• Issued: 06/03/2014
• Est. Priority Date: 07/14/2011
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Coker, Doug
• Primary Examiner(s): Sough, H S
• Assistant Examiner(s): DORAIS, CRAIG C
• Application Number:
  US13/182,803
• Time in Patent Office: 1,055 Days
• Field of Search: None
• US Class Current: 719/328
• CPC Class Codes: G06F 11/3668 Software testing software testing in telephone exchanges H04M3/242, testing of hardware G06F11/22

Abstract

Methods for automatically detecting an anomaly in an application programming interface (API) having a plurality of invariants is provided. Such an API may conform to a Representational State Transfer (REST) model or style of software architecture. Such methods enable users to build simple test scripts that can be fed into specialized software applications configured to automatically exercise and test the API using the test scripts. Further, such methods allow automatic testing of API functionality regardless of whether or not the expected output value(s) for a given input are specified.

Claims

1. A computer-implemented method for automatically detecting an anomaly in an application programming interface having a plurality of invariants, the method comprising:

generating a first data transfer request based on a first mapping of a first representational state transfer operation to a first uniform resource locator template and a request payload, the request payload associated with one or more first commands written in a domain specific language that enables the first representational state transfer operation to be expressed using the one or more first commands independently of the request payload'"'"'s syntax or escape characters;

submitting the generated first data transfer request to a server that implements the application programming interface for processing;

receiving a first response from the server based on the submitted first data transfer request; and

automatically testing the received first response based on the one or more first commands in order to verify that the first response satisfies a first invariant in the plurality of invariants of the application programming interface,

wherein the generating, the submitting, the receiving, and the testing are performed by one or more computing devices; and

wherein at least one command in the one or more first commands includes one or more variables, the one or more first commands written in the domain specific language include a return command for extracting a value from the first response and storing the value as a global variable, and the global variable may be substituted for the one or more variables in the one or more first commands.

2. The method of claim 1, further comprising:

generating a second data transfer request based on a second mapping of a second representational state transfer operation to a second uniform resource locator template and one or more second commands expressed in the domain specific language;

submitting the generated second data transfer request to the server for processing;

receiving a second response from the server based on the submitted second data transfer request; and

automatically testing the received second response based on the one or more second commands in order to verify that it satisfies a second invariant in the plurality of invariants of the application programming interface.

3. The method of claim 1, wherein the first response from the server includes one or more results, and wherein the automatically testing comprises:

parsing each of the one or more results in the first response; and

automatically testing each of the parsed results based on the one or more first commands in order to verify that it satisfies the first invariant of the application programming interface.

4. The method of claim 1, wherein the first data transfer request is a Hypertext Transfer Protocol request.

5. The method of claim 1, wherein the one or more first commands written in the domain specific language include an operation command for specifying a type of the first representational state transfer operation.

6. The method of claim 1, wherein the one or more first commands written in the domain specific language include an authentication command for specifying authentication information.

7. The method of claim 1, wherein the one or more first commands written in the domain specific language include a verification command for verifying the first response matches a predetermined template response.

8. The method of claim 1, further comprising:

identifying request constraints associated with the first data transfer request;

generating a second data transfer request in accordance with the identified request constraints; and

submitting the generated second data transfer request to the server that implements the application programming interface for processing.

9. The method of claim 8, further comprising:

identifying new request constraints based on the first and second data transfer requests; and

generating a third data transfer request in accordance with the identified new request constraints.

10. The method of claim 8, wherein the identifying comprises:

identifying the request constraints based on the one or more first commands associated with the first data transfer request, the request constraints specified in a user test script that includes the one or more first commands and the request payload.

11. The method of claim 8, further comprising:

receiving a second response from the server based on the submitted second data transfer request; and

automatically testing the received second response based on the first data transfer request in order to verify that the second response satisfies the identified request constraints associated with the first data transfer request.

12. A computer-implemented method for automatically detecting an anomaly in an application programming interface having a plurality of invariants, the method comprising:

generating a first data transfer request based on a first mapping of a first representational state transfer operation to a first uniform resource locator template and a request payload, the first uniform resource locator template comprising one or more variable parameters, the request payload associated with one or more first commands written in a domain specific language that enables the first representational state transfer operation to be expressed using the one or more first commands independently of the request payload'"'"'s syntax or escape characters;

submitting the generated first data transfer request to a server that implements the application programming interface for processing;

receiving a first response from the server based on the submitted first data transfer request; and

automatically testing the received first response based on the one or more first commands in order to verify that the first response satisfies a first invariant in the plurality of invariants of the application programming interface;

wherein the generating, the submitting, the receiving, and the testing are performed by one or more computing devices; and

wherein the generating further comprises;

generating the first data transfer request based on a second mapping between a data transfer method and the first representational state transfer operation;

substituting the first representational state transfer operation with the data transfer method based on the third mapping between the data transfer method and the first representational state transfer operation; and

substituting the one or more variable parameters of the first uniform resource locator template with one or more corresponding predetermined values based on the first mapping of the first representational state transfer operation to the first uniform resource locator template.

13. A computer-implemented method for automatically detecting an anomaly in an application programming interface having a plurality of invariants, the method comprising:

generating a first data transfer request based on a first mapping of a first representational state transfer operation to a first uniform resource locator template and a request payload, the request payload associated with one or more first commands written in a domain specific language that enables the first representational state transfer operation to be expressed using the one or more first commands independently of the request payload'"'"'s syntax or escape characters;

submitting the generated first data transfer request to a server that implements the application programming interface for processing;

receiving a first response from the server based on the submitted first data transfer request; and

automatically testing the received first response based on the one or more first commands in order to verify that the first response satisfies a first invariant in the plurality of invariants of the application programming interface,

wherein the generating, the submitting, the receiving, and the testing are performed by one or more computing devices; and

wherein the plurality of invariants includes one or more of data-typing rules, nesting rules, formatting rules, default-value rules, and content-mismatch rules, wherein each of the data-typing rules, the nesting rules, the formatting rules, the default-value rules, and the content-mismatch rules may be based on variables associated with the first uniform resource locator template.

14. A computer-implemented method for automatically detecting an anomaly in an application programming interface having a plurality of invariants, the method comprising:

identifying request constraints associated with a first data transfer request, the identifying including recognizing data type and formatting patterns associated with the first data transfer request;

generating the first data transfer request based on a first mapping of a first representational state transfer operation to a first uniform resource locator template and a request payload, the request payload associated with one or more first commands written in a domain specific language that enables the first representational state transfer operation to be expressed using the one or more first commands independently of the request payload'"'"'s syntax or escape characters;

submitting the generated first data transfer request to a server that implements the application programming interface for processing;

receiving a first response from the server based on the submitted first data transfer request;

automatically testing the received first response based on the one or more first commands in order to verify that the first response satisfies a first invariant in the plurality of invariants of the application programming interface;

generating a second data transfer request in accordance with the identified request constraints and based on the recognized data type and formatting patterns; and

submitting the generated second data transfer request to the server that implements the application programming interface for processing;

wherein the generating, the submitting, the receiving, and the testing are performed by one or more computing devices.

15. A system for automatically detecting an anomaly in an application programming interface having a plurality of invariants, the system comprising:

non-transitory memory storing instructions; and

a data processing device in communication with the non-transitory memory and executing the instructions to implement;

a request generator configured to generate a first data transfer request based on a first mapping of a first representational state transfer operation to a first uniform resource locator template and a request payload, the request payload associated with one or more first commands written in a domain specific language that enables the first representational state transfer operation to be expressed using the one or more first commands independently of the request payload'"'"'s syntax or escape characters, and to submit the generated first data transfer request to a server that implements the application programming interface for processing; and

a response manager configured to receive a first response from the server based on the submitted first data transfer request, and to automatically test the received first response based on the one or more first commands in order to verify that the first response satisfies a first invariant in the plurality of invariants of the application programming interface;

wherein at least one command in the one or more first commands includes one or more variables, and wherein the one or more first commands written in the domain specific language include a return command for extracting a value from the first response and storing the value as a global variable, wherein the global variable may be substituted for the one or more variables in the one or more first commands.

16. The system of claim 15, wherein the request generator is further configured to generate a second data transfer request based on a second mapping of a second representational state transfer operation to a second uniform resource locator template and one or more second commands expressed in the domain specific language, and to submit the generated second data transfer request to the server for processing, and

wherein the response manager is further configured to receive a second response from the server based on the submitted second data transfer request, and to automatically test the received second response based on the one or more second commands in order to verify that it satisfies a second invariant in the plurality of invariants of the application programming interface.

17. The system of claim 15, wherein the first response from the server includes one or more results, and wherein the response manager is configured to parse each of the one or more results in the first response, and to automatically test each of the parsed results based on the one or more first commands in order to verify that it satisfies the first invariant of the application programming interface.

18. The system of claim 15, wherein the first data transfer request is a Hypertext Transfer Protocol request.

19. The system of claim 15, wherein the one or more first commands written in the domain specific language include an operation command for specifying a type of the first representational state transfer operation.

20. The system of claim 15, wherein the one or more first commands written in the domain specific language include an authentication command for specifying authentication information.

21. The system of claim 15, wherein the one or more first commands written in the domain specific language include a verification command for verifying the first response matches a predetermined template response.

22. The system of claim 15, wherein the request generator is further configured to identify request constraints associated with the first data transfer request, to generate a second data transfer request in accordance with the identified request constraints, and to submit the generated second data transfer request to the server that implements the application programming interface for processing.

23. The system of claim 22, wherein the request generator is further configured to identify new request constraints based on the first and second data transfer requests, and to generate a third data transfer request in accordance with the identified new request constraints.

24. The system of claim 22, wherein the request generator is configured to identify the request constraints based on the one or more first commands associated with the first data transfer request, and the request constraints are specified in a user test script that includes the one or more first commands and the request payload.

25. The system of claim 22, wherein the response manager is further configured to receive a second response from the server based on the submitted second data transfer request, and to automatically test the received second response based on the first data transfer request in order to verify that the second response satisfies the identified request constraints associated with the first data transfer request.

26. A system for automatically detecting an anomaly in an application programming interface having a plurality of invariants, the system comprising:

non-transitory memory storing instructions; and

a data processing device in communication with the non-transitory memory and executing the instructions to provide;

a request generator configured to generate a first data transfer request based on a first mapping of a first representational state transfer operation to a first uniform resource locator template and a request payload, the request payload associated with one or more first commands written in a domain specific language that enables the first representational state transfer operation to be expressed using the one or more first commands independently of the request payload'"'"'s syntax or escape characters, and to submit the generated first data transfer request to a server that implements the application programming interface for processing; and

a response manager configured to receive a first response from the server based on the submitted first data transfer request, and to automatically test the received first response based on the one or more first commands in order to verify that the first response satisfies a first invariant in the plurality of invariants of the application programming interface;

wherein the first uniform resource locator template includes one or more variable parameters, wherein the request generator is further configured to generate the first data transfer request based on a third mapping between a data transfer method and the first representational state transfer operation, to substitute the first representational state transfer operation with the data transfer method based on the third mapping between the data transfer method and the first representational state transfer operation, and to substitute the one or more variable parameters of the first uniform resource locator template with one or more corresponding predetermined values based on the first mapping of the first representational state transfer operation to the first uniform resource locator template.

27. A system for automatically detecting an anomaly in an application programming interface having a plurality of invariants, the system comprising:

non-transitory memory storing instructions; and

a data processing device in communication with the non-transitory memory and executing the instructions to provide;

a request generator configured to generate a first data transfer request based on a first mapping of a first representational state transfer operation to a first uniform resource locator template and a request payload, the request payload associated with one or more first commands written in a domain specific language that enables the first representational state transfer operation to be expressed using the one or more first commands independently of the request payload'"'"'s syntax or escape characters, and to submit the generated first data transfer request to a server that implements the application programming interface for processing; and

a response manager configured to receive a first response from the server based on the submitted first data transfer request, and to automatically test the received first response based on the one or more first commands in order to verify that the first response satisfies a first invariant in the plurality of invariants of the application programming interface;

wherein the plurality of invariants includes one or more of data-typing rules, nesting rules, formatting rules, default-value rules, and content-mismatch rules, wherein each of the data-typing rules, the nesting rules, the formatting rules, the default value rules, and the content-mismatch rules may be based on variables associated with the first uniform resource locator template.

28. A system for automatically detecting an anomaly in an application programming interface having a plurality of invariants, the system comprising:

non-transitory memory storing instructions; and

a data processing device in communication with the non-transitory memory and executing the instructions to provide;

a request generator configured to generate a first data transfer request based on a first mapping of a first representational state transfer operation to a first uniform resource locator template and a request payload, the request payload associated with one or more first commands written in a domain specific language that enables the first representational state transfer operation to be expressed using the one or more first commands independently of the request payload'"'"'s syntax or escape characters, and to submit the generated first data transfer request to a server that implements the application programming interface for processing; and

a response manager configured to receive a first response from the server based on the submitted first data transfer request, and to automatically test the received first response based on the one or more first commands in order to verify that the first response satisfies a first invariant in the plurality of invariants of the application programming interface;

wherein the request generator is further configured to identify request constraints associated with the first data transfer request, to generate a second data transfer request in accordance with the identified request constraints, and to submit the generated second data transfer request to the server that implements the application programming interface for processing; and

wherein the request generator is configured to recognize data type and formatting patterns associated with the first data transfer request, and to generate the second data transfer request based on the recognized data type and formatting patterns.