Requirement driven interoperability/compliance testing systems and methods

• US 7,757,121 B1
• Filed: 04/23/2007
• Issued: 07/13/2010
• Est. Priority Date: 04/21/2006
• Status: Expired due to Fees

• Current Assignee: Cydone Solutions Incorporated
• Original Assignee: Cydone Solutions Incorporated
• Inventors: Gauthier, Sylvain, Perron, Francois
• Primary Examiner(s): Beausoliel, Robert
• Assistant Examiner(s): Lottich, Joshua P
• Application Number:
  US11/738,812
• Time in Patent Office: 1,177 Days
• Field of Search: 714/25, 714/26, 714/32, 714/37, 714/38
• US Class Current: 714/26
• CPC Class Codes: G06F 11/263 Generation of test inputs, e.g. test vectors, patterns or sequences ; with adaptation of the tested hardware for testability with external testers
• CPC Class Codes: G06F 11/3664 Environments for testing or debugging software
• CPC Class Codes: G06F 11/368 for test version control, e.g. updating test cases to a new software version

Abstract

A system and method for requirement driven interoperability/compliance testing of Hardware and Software in a distributed environment. Three modules are configured as a default system: a test definition, a SUT definition and a test result. A series of tree elements are coupled to represent requirements to test case mapping. Test cases are executed and components in the system are assigned with test results while requirements coverage is computed. Auto-detection of additional parts and test cases enables validity checking of the requirement to test case tree and reports are generated.

Claims

1. An interoperability/compliance test system for performing tests on a system under test (SUT), the test system comprising:

an integrity engine comprising a server; and

a memory accessible by the server and having computer code thereon, the memory implementing the server to provide;

a test definition engine for receiving test requirements, for inserting test objects into a structured format for a test definition, the test definition structured format comprising a test definition default object, and for linking a test script to one of the test requirements;

an SUT definition engine for inserting SUT objects into a structured format for an SUT definition, the SUT definition structured format comprising an SUT default object, components of the SUT, and a state for each of the components; and

a test result engine for inserting test result objects into a structured format for a test result definition, the test result definition structured format comprising at least one of a test event, a measurement and configuration;

wherein when a new object is to be inserted in one of the structured formats and the new object does not belong to any specific parent object it is inserted in one of the default parent objects thereby each one of the engines can be used independently and in any sequence to complete at least a part of an interoperability/compliance test; and

a computing device in communication with the server for implementing a test harness, the test harness for connection to the SUT and the integrity engine; and

for receiving the test script, for using the test script to send stimuli to the SUT, for capturing a resulting response from the SUT, and for forwarding the resulting response to the test result engine.

2. The test system of claim 1, comprising an object database accessible by the server via a communication channel, for storing at least one of:

the test objects, the test definition default object, the SUT default object, the test result objects.

3. The test system of claim 2, comprising a web engine implemented on a server, the web engine having access to the server and the object database, for allowing navigation through data stored on the object database and for allowing access to the test results objects.

4. The test system of claim 1, comprising a history database having access to the server via a communication channel, the history database implementing a revision control system for tracking revisions of data being processed by the server.

5. The test system of claim 1, comprising an application programming interface (API) engine implemented on the server, for interaction with a computing device of a user, to allow the user to enter the test requirements in accordance with a given format.

6. The test system of claim 1, comprising a user computing device in communication with the server via a network, for allowing a user to access the server to at least one of:

input test requirements;

update the test objects;

request that the server generates a test report;

managing test objects and test result objects using the server.

7. The test system of claim 1, wherein the integrity engine is implemented by the computer code to provide the test definition engine for linking the test script to the one of the test requirements using an annotation, the annotation being assigned to at least one of:

the test script and the one of the test requirements.

8. An interoperability/compliance test method performing tests on a system under test (SUT), the test method comprising:

a server;

receiving test requirements;

inserting test objects into a structured format for a test definition, the test definition structured format comprising a test definition default object;

linking a test script to one of the test requirements;

inserting SUT objects into a structured format for an SUT definition, the SUT definition structured format comprising an SUT default object, components of the SUT, and a state for each of the components;

inserting test result objects into a structured format for a test result definition, the test result definition structured format comprising at least one of a test event, a measurement and configuration; and

sending the test script to a test harness connected to the SUT;

wherein when inserting a new object in one of the structured formats and the new object does not belong to any specific parent object it is inserted in one of the default parent objects and stored in a memory of the server;

a computing device implementing the test harness;

receiving the test script;

using the test script to send stimuli to the SUT;

capturing a resulting response from the SUT; and

outputting the resulting response for display on a display device.

9. The test method of claim 8, wherein the linking a test script to one of the test requirements comprises assigning an annotation to at least one of:

the test script and the one of the test requirements.

10. The test method of claim 8, wherein the using the test script to send stimuli to the SUT comprises using the test script to generate a stimuli which comprises at least one of a set of manual operations on the SUT and a set of automated operations on the SUT.

11. The test method of claim 8, wherein the inserting the test results objects comprise inserting the configuration which comprises at least one of:

a name of a user, a start date and an end date associated to an execution of the test, a system name identifier, a sub-system serial number, a sub-system version number.

12. The test method of claim 8, comprising the server generating a report upon receiving a request via a network from an other computing device;

and sending the report to the other computing device for display thereon.

13. The test method of claim 8, wherein the receiving the test requirements comprises retrieving default test requirements;

and wherein the linking, the inserting SUT objects, the inserting test result objects, and the sending the test script are reiterated upon user-entered test requirements are received by the server from a user computing device.