Analysis of distributed software systems via specification substitution

• US 7,797,669 B1
• Filed: 02/13/2004
• Issued: 09/14/2010
• Est. Priority Date: 02/13/2004
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Fournet, Cédric, Hoare, Charles Antony Richard, Rajamani, Sriram K., Andrews, Anthony D., Rehof, Niels Jakob
• Primary Examiner(s): Zhen; Wei Y
• Assistant Examiner(s): Wu; Junchun
• Application Number:
  US10/779,002
• Time in Patent Office: 2,405 Days
• Field of Search: 717123-129, 717/100, 719/315
• US Class Current: 717/100
• CPC Class Codes: G06F 11/3668 Software testing software testing in telephone exchanges H04M3/242, testing of hardware G06F11/22
• CPC Class Codes: G06F 8/10 Requirements analysis; Specification techniques

Abstract

A distributed software system of communicating software components can be tested for undesirable behavior. A specification of a component can be substituted in place of the component when testing a model of the distributed software system. Thus, the system can be checked to see if it exhibits undesirable behavior without having code for all components of the system. Also, a component can be checked to see if it is in conformance with its specification. If models built with respective components and substituted specifications indicate that the system does not exhibit undesirable behavior, and the components conform to their specifications, then a system assembled from the components will not exhibit the undesirable behavior. Thus, collaborative testing can be achieved, even if no one entity has access to code for the entire distributed system.

Claims

1. One or more computer-readable storage media having computer-executable instructions for causing a computer to perform the following to determine whether a distributed software system having a component and one or more other concurrently executable components operable to communicate via message passing over a network will exhibit undesirable behavior:

reading from one or more computer-readable storage media one or more specifications of externally observable message-passing behavior for the one or more other concurrently executable components;

testing whether the distributed software system will exhibit the undesirable behavior, wherein the testing comprises using the one or more specifications in place of the one or more other concurrently executable components and building a model for the distributed software system, wherein the model incorporates the one or more specifications in place of the one or more other concurrently executable components; and

storing results of the testing in one or more computer-readable storage media;

wherein the undesirable behavior is selected from the group consisting of becoming deadlocked, arriving at a message-passing-based deadlock, reaching an end state for the component while an other concurrently executable component is expecting to send a message to the component or receive a message from the component, waiting by a concurrently executable component for a message that is never to be sent by the other concurrently executable component, sending a message by the concurrently executable component to the other concurrently executable component that is not expecting the message, and becoming stuck.

2. The one or more computer-readable storage media of claim 1 wherein the model incorporates a representation of an implementation of the component.

3. The one or more computer-readable storage media of claim 1 wherein the testing proceeds without access to internal logic of the one or more other concurrently executable components.

4. The one or more computer-readable storage media of claim 1 further comprising computer-executable instructions for performing the following:

determining whether the component complies with a specification of its externally observable message-passing behavior.

5. The one or more computer-readable storage media of claim 1, wherein the concurrently executable components communicate asynchronously.

6. The one or more computer-readable storage media of claim 1, wherein at least one of the components comprises a web service.

7. A computer-implemented method of determining whether a web services system having a plurality of concurrently executable web services will exhibit undesirable behavior, the method comprising:

testing whether a web service and one or more other concurrently executable web services will exhibit undesirable behavior, wherein the testing comprises using at least one specification of externally observable message passing behavior in place of the one or more other concurrently executable web services and building a model for the web service and the one or more other concurrently executable web services, wherein the model incorporates one or more specifications of externally observable message passing behavior in place of the one or more other concurrently executable web services;

testing the model to determine whether the web service complies with the specification of the web service'"'"'s externally observable message-passing behavior; and

storing results of the testing the model in one or more computer-readable storage media;

wherein the undesirable behavior is selected from the group consisting of becoming deadlocked, arriving at a message-passing-based deadlock, reaching an end state for the component while an other concurrently executable component is expecting to send a message to the component or receive a message from the component, waiting by a concurrently executable component for a message that is never to be sent by the other concurrently executable component, sending a message by the concurrently executable component to the other concurrently executable component that is not expecting the message, and becoming stuck.

8. A computer-implemented system for determining whether a distributed software system having a plurality of concurrently executable components will exhibit undesirable behavior, the system comprising:

at least one processing unit configured to execute software; and

one or more computer-readable storage media comprising;

a software representation of a concurrently executable component being tested;

one or more software specifications of externally observable message-passing behavior of one or more others of the concurrently executable components; and

a conformance checker software operable to test whether the distributed software system will exhibit undesirable behavior, wherein the testing comprises using the one or more software specifications in place of the one or more others of the concurrently executable components, wherein the conformance checker software is further operable to build a model for the distributed software system, wherein the model incorporates the software representation in place of the concurrently executable component and the one or more software specifications in place of the one or more others of the concurrently executable components, wherein the undesirable behavior is selected from the group consisting of becoming deadlocked, arriving at a message-passing-based deadlock, reaching an end state for the component while an other concurrently executable component is expecting to send a message to the component or receive a message from the component, waiting by a concurrently executable component for a message that is never to be sent by the other concurrently executable component, sending a message by the concurrently executable component to the other concurrently executable component that is not expecting the message, and becoming stuck.

9. The computer-implemented system of claim 8 wherein the conformance checker software is further operable to determine whether the software representation conforms to a specification of its externally observable message-passing behavior.

10. A computer-implemented method of determining whether a component P of a distributed software system having a plurality of other concurrently executable components conforms with a specification of externally observable message-passing behavior Q of the component P, the method comprising:

reading the specification of externally observable message passing behavior Q;

determining whether the component P conforms with the specification of externally observable message passing behavior Q, wherein the testing comprises using at least one specification of externally observable message-passing behavior in place of one or more of the other concurrently executable components and building a model of the component P and one or more of other of the concurrently executable components, wherein the model incorporates the at least one specification of externally observable message-passing behavior in place of at least one of the other of the concurrently executable components;

determining whether the distributed software system will exhibit undesirable behavior, wherein the undesirable behavior is selected from the group consisting of arriving at a message-passing-based deadlock, reaching an end state for the component while one of the other of the concurrently executable components is expecting to send a message to the component or receive a message from the component, waiting by the component for a message that is never to be sent by one of the other of the concurrently executable components sending a message by the component to one of the other of the concurrently executable components, wherein the one of the other of the concurrently executable components is not expecting the message and becoming stuck; and

storing in one or more computer-readable media results of the determining whether the component P conforms with the specification of externally observable message passing behavior Q and of the determining whether the distributed software system will exhibit undesirable behavior.

11. The computer-implemented method of claim 10, wherein the component P conforms with the specification of the component'"'"'s externally observable message-passing behavior Q when, for a set of action sequences τ

*·

λ

, a component P′ and

a specification of externally observable message-passing behavior Q′

;

If

12. The computer-implemented method of claim 10, wherein the component P conforms with the specification of the component'"'"'s externally observable message-passing behavior Q if and only if there exists a relation R containing a pair (P,Q) such that R satisfies the following two conditions for all pairs in a set of pairs (S,T) in R:

whenever S can transition to S′

by first performing an arbitrary sequence of internal communications and then performing an action, then T can transition to a state T′

by first performing an arbitrary sequence of internal communications and then performing the same action, where a pair (S′

,T′

) is again a member of the relation R; and

if P can refuse a set X while ready on a set Y, then Q can also refuse the set X while ready on the set Y, wherein the set X comprises one or more labeled transitions for a labeled transition system, and wherein the set Y comprises the empty set or one or more names for the labeled transition system.

13. The computer-implemented method of claim 10, wherein the testing proceeds without access to internal logic of the one or more of the other of the concurrently executable components.

14. The computer-implemented method of claim 10, wherein the concurrently executable components communicate asynchronously.

15. The computer-implemented method of claim 10, wherein at least one of the concurrently executable components comprises a web service.

16. One or more computer-readable storage media having computer-executable instructions for performing the computer-implemented method of claim 10.

17. A computer-implemented system for determining whether a distributed software system having a plurality of concurrently executable components will exhibit undesirable behavior, the system comprising:

at least one processing unit configured to execute software; and

one or more computer-readable storage media comprising;

a software representation of a concurrently executable component being tested;

a software specification of externally observable message-passing behavior of the component being tested;

one or more software specifications of externally observable message-passing behavior of one or more others of the concurrently executable components; and

a conformance checker software operable to test whether the distributed software system will exhibit undesirable behavior and whether the concurrently executable component being tested conforms to its specification of externally observable message passing behavior, wherein the testing comprises using the one or more software specifications in place of the one or more others of the concurrently executable components, wherein the conformance checker software is further operable to build a model for the distributed software system, wherein the model incorporates the one or more software specifications in place of the one or more others of the concurrently executable components;

wherein the undesirable behavior is selected from the group consisting of becoming deadlocked, arriving at a message-passing-based deadlock, reaching an end state for the component while an other concurrently executable component is expecting to send a message to the component or receive a message from the component, waiting by a concurrently executable component for a message that is never to be sent by the other concurrently executable component, sending a message by the concurrently executable component to the other concurrently executable component that is not expecting the message, and becoming stuck.

18. A computer-implemented method of conformance checking, comprising:

receiving a specification of externally observable message-passing behavior of a service;

testing a system, the system comprising an implementation of the service in the system, wherein the testing comprises using the specification of externally observable message-passing behavior of the service in place of the implementation of the service in the system and receiving a model for the implementation of the service in the system, wherein the model incorporates the specification of externally observable message-passing behavior of the service in place of the implementation of the service in the system;

determining whether the model exhibits an undesirable behavior, wherein the undesirable behavior is selected from the group consisting of becoming deadlocked, arriving at a message-passing-based deadlock, reaching an end state for the service while a concurrently executable component is expecting to send a message to the service or receive a message from the service, waiting by the service for a message that is never to be sent by the concurrently executable component, sending a message by the service to the concurrently executable component that is not expecting the message, and becoming stuck; and

storing results of the testing and of the determining in one or more computer-readable storage media.

19. The method of claim 18, wherein the service is a component of a message-passing system.

20. A computer-implemented system for conformance checking, comprising:

at least one processing unit configured to execute software; and

one or more computer-readable storage media comprising;

a software interface specification of a component of a message-passing system; and

a conformance checker software, wherein the conformance checker software is operable to test an implementation of the component of the message-passing system, wherein the testing comprises using the software interface specification of the component in place of the component in the implementation of the component of the message-passing system, wherein the test further comprises receiving a model for the implementation of the component of the message-passing system, wherein the model incorporates the software interface specification in place of the component, and wherein the test further comprises determining whether the model exhibits an undesirable behavior, wherein the undesirable behavior is selected from the group consisting of becoming deadlocked, arriving at a message-passing-based deadlock, reaching an end state for the component of the message-passing system while a concurrently executable component is expecting to send a message to the component of the message-passing system or receive a message from the component of the message-passing system, waiting by the component of the message-passing system for a message that is never to be sent by the concurrently executable component, sending a message by the component in the message-passing system to the concurrently executable component that is not expecting the message, and becoming stuck.

21. The computer-implemented system of claim 20, wherein the software interface specification is written in Business Process Execution Language (BPEL).

22. One or more computer-readable storage media having computer-executable instructions for causing a computer to perform a method to determine whether a distributed software system having a component and one or more other concurrently executable components operable to communicate via message passing over a network will exhibit undesirable behavior, the method comprising:

reading one or more specifications of externally observable message-passing behavior for the one or more other concurrently executable components;

testing whether the distributed software system will exhibit the undesirable behavior, wherein the testing comprises using the one or more specifications in place of the one or more other concurrently executable components and building a model for the distributed software system, wherein the model incorporates the one or more specifications in place of the one or more other concurrently executable components;

storing results of the testing in one or more computer-readable storage media; and

determining whether the component complies with a specification of its externally observable message-passing behavior,

wherein the undesirable behavior is selected from the group consisting of becoming deadlocked, arriving at a message-passing-based deadlock, reaching an end state for the component while an other concurrently executable component is expecting to send a message to the component or receive a message from the component, waiting by a concurrently executable component for a message that is never to be sent by the other concurrently executable component, sending a message by the concurrently executable component to the other concurrently executable component that is not expecting the message, and becoming stuck.