Automated Testing and Control of Networked Devices

• US 20080120521A1
• Filed: 11/21/2006
• Published: 05/22/2008
• Est. Priority Date: 11/21/2006
• Status: Active Grant

• Current Assignee: Etaliq Incorporated
• Original Assignee: Etaliq Incorporated
• Inventors: Gupta, Pankaj, Hill, Paul, Uchimaru, Clifford, Trottier, Jean-Sebastien, Poisson, Kenneth J., Muma, Kimberley J., Binns, Mark, Vobruba, Vladimir, Beverley, Jonathan, Ivan, Chris, St-Laurent, Normand, Phillips, Randall A., Aunger, Nola Michele, Casey, Rick
• Granted Patent: US 7,631,227 B2
• Time in Patent Office: Days
• US Class Current: 714/26
• CPC Class Codes: G06F 11/2252 using fault dictionaries
• CPC Class Codes: G06F 11/2294 by remote test

Abstract

A system, method of automated testing and control of networked devices is provided. One or more test cases are defined in a test plan for execution against a plurality of networked devices. The test cases are created using a command defined grammar comprising verbs which characterize how commands or actions should be performed. Abstraction markers allow for device-specific characteristics to be mapped to a target device, without modification of the test cases and the test plan itself. The verbs and abstraction markers, once mapped to a target device form device-specific commands comprising actions and device-specific characteristics which are executed against the target networked device. The resulting responses are parsed against expected responses and a result or verdict is assigned. By providing command grammar and abstraction capability a common test plan and test cases can be applied across a range of diverse networked devices without require user intervention or modification.

Claims

1. An automation test system, for testing and control of a plurality of networked devices, the system comprising:

a node database comprising device-specific characteristics associated with each of the plurality of networked devices;

an abstraction module for converting verbs identified in a plurality of test cases to a set of actions, selected based on device-specific characteristics mapped from the node database, to be executed in relation to target networked devices, the abstraction module also for mapping abstraction markers used in the test cases to device-specific characteristics, wherein the verbs and abstraction markers form device-specific commands;

an execution module for executing the device-specific commands generated by the abstraction module, against the target networked devices, by sending device-specific commands to the target networked devices; and

a parser module for parsing responses received from the target networked devices during execution of the test cases by the execution module, and for determining a verdict for each test step and for each test case.

2. The system of claim 1 wherein the abstraction markers further comprise hardware abstraction markers and command line interface (CLI) abstraction markers.

3. The system of claim 1 further comprising a system database comprising a command grammar defining the verbs and associated actions to be executed.

4. The system of claim 3 wherein the execution module utilizes at least one execution plan which defines configuration requirements and execution requirements, wherein the execution plan relates the test case resources to the target networked devices in the node database.

5. The system of claim 4 wherein test cases are arranged hierarchically into groups in a test plan comprising tags delineating groups from cases and procedural sections therein, the verdict of each procedural section can alter execution of the remaining procedural sections of the test cases and test groups.

6. The system of claim 5 wherein the abstraction module further comprises a syntax module which verifies test plan and execution plan structure and hierarchy, and the node database referenced hardware and CLI abstraction.

7. The system of claim 6 further comprising a scheduler module for scheduling executions of the test plans against the networked devices, wherein the scheduler dynamically assesses availability of the networked devices and determines the optimal time for execution based on requirements such as those defined in the execution plan.

8. The system of claim 4 wherein the execution requirements identified in the execution plan define control flags to be used within the test cases.

9. The system of claim 1 wherein the node database further comprises node classes, each node class associated with a unique group of networked devices having common characteristics.

10. The system of claim 9 wherein the characteristics are selected from the group comprising:

connection parameters, login and response formats including prompts, unsolicited message formats and recognition of successful and unsuccessful commands.

11. The system of claim 10 wherein the parser module utilizes expected responses defined in the test cases and device-specific characteristics from the node database to determine the verdict of test steps.

12. The system of claim 11 wherein the verdict of execution of the test case is selected from the group comprising:

Pass, Fail, Parent Fail, Child Fail, Not Applicable, Incomplete, Unclean, Error and Abort.

13. The system of claim 6 further comprising a system database containing grammar definition, test plan and execution structure and supported tags utilized by the abstraction module to interpret the test and execution plan validity.

14. The system of claim 1 further comprising a simulation module for simulating behaviour and response of one or more networked devices.

15. The system of claim 1 further comprising:

a results database for storing all execution log files, including responses received from the networked devices; and

a report and analysis module for indexing each line of each execution log file with a relative position pointer (RPP) and a source position pointer (SPP), wherein the RPP is a relative chronological index established during execution and the SPP links to the source test steps defined in the test cases.

16. A method of testing and control of a plurality of networked devices, the method comprising the steps of:

converting verbs identified in test cases to a set of actions to be executed in relation to the plurality of networked devices;

mapping abstraction markers in test cases to device-specific characteristics of the plurality of networked devices;

sending device-specific commands comprising the converted actions and mapped characteristics to the plurality of networked devices;

receiving responses from the plurality of networked devices;

parsing the received response for each of the plurality of networked devices to determine a result for each test step and for each test case; and

storing the result.

17. The method of claim 16 wherein the steps of converting and mapping further comprises retrieving device-specific actions and characteristics from a database for each of the plurality of networked devices.

18. The method of claim 17, prior to the step of sending, further comprises the steps of:

verifying test case structure and grammar against rules defined in a system database;

verifying validity of verbs against grammar mappings in the node database;

verifying validity of abstraction markers against command mappings and node specifications in a node database; and

verifying resulting device-specific commands against node behaviours in the node database.

19. The method of claim 16 wherein the step of parsing further comprises utilizing expected responses defined in the test case and device-specific characteristics retrieved from the database.

20. The method of claim 19 wherein the characteristics are selected from the group comprising:

connection parameters, login and response formats including prompts, unsolicited message formats and recognition of successful and unsuccessful commands.

21. The method of claim 16 wherein the abstraction markers used in the step of mapping further comprises hardware abstraction markers and command line interface (CLI) abstraction markers.

22. The method of claim 16 further comprising a step of scheduling the execution of one or more test cases based upon associated configuration and execution requirements and availability of each of the plurality of networked devices.

23. The method of claim 16 wherein the step of storing further comprises associating each line of each execution log file with a relative position pointer (RPP) and a source position pointer (SPP), wherein the RPP is a relative chronological index established during execution and the SPP links to the source test steps defined in the test cases.

24. The method of claim 16 wherein the result is a verdict selected from the group comprising:

Pass, Fail, Parent Fail, Child Fail, Not Applicable, Incomplete, Unclean, Error and Abort.

25. A computer readable medium, containing one or more instructions executable by a computer for performing a method of test and control of a plurality of networked devices, the method comprising the steps of:

converting verbs identified in test cases to a set of actions to be executed in relation to the plurality of networked devices;

mapping abstraction markers in test cases to device-specific characteristics of the plurality of networked devices;

sending device-specific commands comprising the converted actions and mapped characteristics to the plurality of networked devices;

receiving responses from the plurality of networked devices;

parsing the received response for each of the plurality of networked devices to determine a result for each test step and for each test case; and

storing the result.