Software test environment for regression testing ground combat vehicle software
First Claim
1. A verification system for combat vehicle software application comprising:
- A first processor communicatively coupled to at least one combat vehicle software application (CVSA) and a second operating system environment;
a second processor communicatively coupled to a set of extensible simulation primitives and a second operating system environment;
a CVSA configured for operation on the first processor under the first operating system environment, the CVSA communicatively coupled to and configured for communication with a plurality of combat vehicle subsystems (CVSUB)s by exchanging a plurality of messages according to a first predefined protocol, each CVSUB characterized by at least one expected behavior triggered responsive to an exchange of at least one of the plurality of messages with the CVSA, wherein the CVSA is operative to generate at least a first event representing a resultant of one or more interdependent expected behaviors of a first plurality of CVSUBs triggered responsive to the exchange, between the CVSA and the first plurality of CVSUBs, of a first ordered sequence of messages from among the plurality of messages;
a set of extensible simulation primitives comprising computer executable instructions suitable for execution on the second processor under the second operating system environment, the set of extensible simulation primitives designed so that at least one subset of extensible simulation primitives is capable of cooperating to simulate the at least one expected behavior of a selected CVSUB from among the plurality of CVSUBS;
a plurality of simulated subsystems (SIMSUB)s corresponding to the plurality of CVSUIBs, each SIMSUB comprising the at least one subset of extensible simulation primitives capable of cooperating to simulate the at least one expected behavior of the CVSUB that corresponds to the SIMSUB by exchanging the at least one of the plurality of messages with the CVSA sufficient to trigger the at least one expected behavior of the CVSUB;
a graphical user interface (GUI) comprising at least one GUI panel corresponding to each of the SIMSUBs, each GUI panel associated with a selected SIMSUB adapted to enable a user to inject input to and receive output from the selected SIMSUB wherein the input and the output affect the generation of the first event;
a record-replay module executing on the second processor and communicatively coupled to the first plurality of SIMSUBs to selectively store the injected input and the received output from each GUI panel associated with the first plurality of SIMSUBs involved in the exchange of the first ordered sequence of messages with the CVSA in a test vector file, and responsive to user input, replay the test vector file to recreate the exchange of the first ordered sequence of messages between the first plurality of SIMSUBs and an altered CVSA, the record-play module comprising;
a test vector file accessible using the second processor, the test vector file configured to store the injected input and the received output from each GUI panel associated with the first plurality of SIMSUBs involved in the exchange of the first ordered sequence of messages with the CVSA to affect generation of the first event by the CVSA; and
seven components, wherein each component includes computer executable code defining a logic structure, and wherein;
the first component is configured with indicia to indicate and confirm activation of a RECORD mode and to open the test vector file responsive to the RECORD mode being activated;
the second component is configured with storage logic to cause storage of the injected input and the received output in the test vector file;
the third component is configured with indicators to indicate and confirm activation of a REPLAY mode;
the fourth component is provided with an array, the fourth component operative to retrieve the test vector file in which the second component has stored the injected input and transfer the received output as entries in the array before transferring the array to the fifth component;
the fifth component configured with control logic adapted to send the received output received from the fourth component to a comparator module;
the sixth component configured with analysis logic adapted to compare the entries in the array received from the fourth component with the responses received from the SSM to determine a whether a disagreement indicative of a regression failure exists, generate pass/fail results for the test file based on the existence of a disagreement, and store the entries of the array; and
the seventh component configured with report logic adapted to generate a report using the pass/fail results from the sixth component; and
a comparator module communicating with the record-replay module and the GUI, the comparator module operative to capture and dynamically compare the received output from the attempted regeneration of the first event using the altered CVSA to the received output from the generation of the first event by the unaltered CVSA and output indicia indicative of each disagreement between the two;
wherein a user interfacing with the GUI selectively controls the operation of the verification system in a first operating system environment to selectively perform the regression testing of the CVSA operating in a second operating system environment by simulating external CVSUBs and selectively recording and replaying simulated subsystem commands and subsystem responses to generate a pass/fail report from the comparator module to indicate the results of the regression test on the CVSA.
1 Assignment
0 Petitions
Accused Products
Abstract
A STE for automated testing of ground combat vehicle software application to validate vehicle software logic provides a system and method to conduct interactive (manual) testing of the vehicle software while under development, record information related to the interactive testing activities, including but not limited to tester inputs and expected outcomes, and perform automated testing of the combat vehicle software application using the recorded information. Preferably, reconfiguration of the STE to support changes that arise due to the evolution of the combat vehicle software application system and the subsystems under control of the evolving software system is provided.
-
Citations
18 Claims
-
1. A verification system for combat vehicle software application comprising:
-
A first processor communicatively coupled to at least one combat vehicle software application (CVSA) and a second operating system environment; a second processor communicatively coupled to a set of extensible simulation primitives and a second operating system environment; a CVSA configured for operation on the first processor under the first operating system environment, the CVSA communicatively coupled to and configured for communication with a plurality of combat vehicle subsystems (CVSUB)s by exchanging a plurality of messages according to a first predefined protocol, each CVSUB characterized by at least one expected behavior triggered responsive to an exchange of at least one of the plurality of messages with the CVSA, wherein the CVSA is operative to generate at least a first event representing a resultant of one or more interdependent expected behaviors of a first plurality of CVSUBs triggered responsive to the exchange, between the CVSA and the first plurality of CVSUBs, of a first ordered sequence of messages from among the plurality of messages; a set of extensible simulation primitives comprising computer executable instructions suitable for execution on the second processor under the second operating system environment, the set of extensible simulation primitives designed so that at least one subset of extensible simulation primitives is capable of cooperating to simulate the at least one expected behavior of a selected CVSUB from among the plurality of CVSUBS; a plurality of simulated subsystems (SIMSUB)s corresponding to the plurality of CVSUIBs, each SIMSUB comprising the at least one subset of extensible simulation primitives capable of cooperating to simulate the at least one expected behavior of the CVSUB that corresponds to the SIMSUB by exchanging the at least one of the plurality of messages with the CVSA sufficient to trigger the at least one expected behavior of the CVSUB; a graphical user interface (GUI) comprising at least one GUI panel corresponding to each of the SIMSUBs, each GUI panel associated with a selected SIMSUB adapted to enable a user to inject input to and receive output from the selected SIMSUB wherein the input and the output affect the generation of the first event; a record-replay module executing on the second processor and communicatively coupled to the first plurality of SIMSUBs to selectively store the injected input and the received output from each GUI panel associated with the first plurality of SIMSUBs involved in the exchange of the first ordered sequence of messages with the CVSA in a test vector file, and responsive to user input, replay the test vector file to recreate the exchange of the first ordered sequence of messages between the first plurality of SIMSUBs and an altered CVSA, the record-play module comprising; a test vector file accessible using the second processor, the test vector file configured to store the injected input and the received output from each GUI panel associated with the first plurality of SIMSUBs involved in the exchange of the first ordered sequence of messages with the CVSA to affect generation of the first event by the CVSA; and seven components, wherein each component includes computer executable code defining a logic structure, and wherein; the first component is configured with indicia to indicate and confirm activation of a RECORD mode and to open the test vector file responsive to the RECORD mode being activated; the second component is configured with storage logic to cause storage of the injected input and the received output in the test vector file; the third component is configured with indicators to indicate and confirm activation of a REPLAY mode; the fourth component is provided with an array, the fourth component operative to retrieve the test vector file in which the second component has stored the injected input and transfer the received output as entries in the array before transferring the array to the fifth component; the fifth component configured with control logic adapted to send the received output received from the fourth component to a comparator module; the sixth component configured with analysis logic adapted to compare the entries in the array received from the fourth component with the responses received from the SSM to determine a whether a disagreement indicative of a regression failure exists, generate pass/fail results for the test file based on the existence of a disagreement, and store the entries of the array; and the seventh component configured with report logic adapted to generate a report using the pass/fail results from the sixth component; and a comparator module communicating with the record-replay module and the GUI, the comparator module operative to capture and dynamically compare the received output from the attempted regeneration of the first event using the altered CVSA to the received output from the generation of the first event by the unaltered CVSA and output indicia indicative of each disagreement between the two; wherein a user interfacing with the GUI selectively controls the operation of the verification system in a first operating system environment to selectively perform the regression testing of the CVSA operating in a second operating system environment by simulating external CVSUBs and selectively recording and replaying simulated subsystem commands and subsystem responses to generate a pass/fail report from the comparator module to indicate the results of the regression test on the CVSA. - View Dependent Claims (2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12)
-
-
13. A software test environment (STE) for conducting a regression test of a combat vehicle software application (CVSA) running on a first processor under a first operating system environment, the CVSA configured to interact with combat vehicle software subsystems applications (CVSUBAs) by encoded messages communicated over a data bus to generate a set of events, each event resulting from an interaction of one or more CVSUBAs with the CVSA, the STE comprising:
-
a first processor communicatively coupled to a CVSA and a first operating system environment; a second processor communicatively coupled to one or more simulated system modules (SSMs); a simulated subsystem module (SSM) having computer readable instructions defining a separate logic structure for each of the CVSUBAs, the logic structures configured for operation on the second processor under a second operating system environment wherein each separate logic structure simulates at least one operation of at least one of the CVSUBAs, wherein a first event results from one or more simulated operations representing the interaction of the at least one CVSUBA or a combination of CVSUBAs with the CVSA using a first set of electronic message interchanges; a bi-directional communication link communicatively coupled to the first and second processors to allow an exchange of electronic messages between the CVSA and the SSM using a predetermined communications protocol; a translator module coupled intermediate the bi-directional communication link and each of the first and second processors, the translator module configured to provide a communication interface between the CVSA and the SSM; a graphical user interface (GUI) coupled to the SSM and having a plurality of control panels for data input so as to initiate and to continue the generation of the first event; a record-replay module communicatively coupled to the SSM to selectively store the electronic message interchanges associated with the generation of the first event in a first test file and responsive to user input, replay the first test file to selectively supply at least one of the first set of electronic message interchanges stored therein to the SSM, in lieu of the data input, to attempt to initiate and automatically continue the generation of the first event, the record-replay module effective to dynamically compare the resulting second set of electronic message interactions with the first set of electronic message interactions to determine a disagreement between them wherein the disagreement is indicative of the CVSA failing the regression test, the record-play module comprising; a first test file comprising the electronic message interchanges that include subsystem commands and subsystem responses; and seven components, wherein each component includes computer executable code defining a logic structure, and wherein; the first component is configured with indicia to indicate and confirm activation of a RECORD mode and to open the at least one test file responsive to the RECORD mode being activated; the second component is configured with storage logic to cause storage of the subsystem commands and subsystem responses in the test file; the third component is configured with indicators to indicate and confirm activation of a REPLAY mode; the fourth component is provided with an array, the fourth component operative to retrieve the test file in which the second component has stored the simulated subsystem commands and subsystem responses and transfer the stored subsystem commands and subsystem responses as entries in the array before transferring the array to the fifth and the sixth component; the fifth component configured with control logic adapted to send the stored subsystem commands and subsystem responses received from the fourth component to the SSM; the sixth component configured with analysis logic adapted to compare the entries in the array received from the fourth component with the responses received from the SSM to determine a whether a disagreement indicative of a regression failure exists, generate pass/fail results for the test file based on the existence of a disagreement, and store the entries of the array; and the seventh component configured with report logic adapted to generate a report using the pass/fail results from the sixth component; and wherein a user interfacing with the GUI selectively controls the operation of the STE in a first operating system environment to selectively perform the regression testing of the CVSA operating in a second operating system environment by simulating external combat vehicle subsystems and selectively recording and replaying simulated subsystem commands and subsystem responses to generate a pass/fail report to indicate the results of the regression test on the CVSA. - View Dependent Claims (14, 15, 16, 17)
-
-
18. A system for regression testing combat vehicle software application configured for cooperative operation with at least one combat vehicle subsystem, the system comprising:
-
a first processor operating under the control of a first operating system; at least one second processor operating under the control of a second operating system; at least one data store communicatively coupled to at least one of the first and the second processors; a combat vehicle software application (CVSA) configured to execute on the first processor, the CVSA operative to receive regression testing related information defining at least one stimulus and to selectively generate regression testing related information defining at least one signal command responsive to the at least one stimulus; and at least one reconfigurable logic module including instructions executable on the second processor, the instructions operative to generate regression testing related information defining a signal response responsive to the at least one signal command and regression testing related information defining a time for response, the signal response indicative of a predefined behavior of the at least one combat vehicle subsystem in response to the at least one signal command, the time for response indicative of an anomalous delay between the at least one stimulus and the signal response; a bi-directional communication link operative to communicatively couple the first and the second processors for transmission of regression testing related information between the combat vehicle software and the at least one reconfigurable logic module; a translator module configured to interoperate with the bi-directional communication link and the combat vehicle software at the first processor and with the bi-directional communication link and the at least one reconfigurable logic module at the second processor, the translator module operative to reformat regression testing related information for each passage of the regression testing related information to and from the bi-directional communication link; an input device configured to receive user input and to generate regression testing related information defining an input signal; and a record-replay module in data communication with the at least one reconfigurable logic module and the at least one data store, the record-replay module comprising; at least one test file that receives the input signal from the input device, including the subsystem commands and subsystem responses; and seven components, wherein each component includes computer executable code defining a logic structure, and wherein; the first component is configured with indicia to indicate and confirm activation of a RECORD mode and to open the at least one test file responsive to the RECORD mode being activated; the second component is configured with storage logic to cause storage of the subsystem commands and subsystem responses in the at least one test file; the third component is configured with indicators to indicate and confirm activation of a REPLAY mode; the fourth component is provided with an array, the fourth component operative to retrieve the at least one test file in which the second component has stored the simulated subsystem commands and subsystem responses and transfer the stored subsystem commands and subsystem responses as entries in the array before transferring the array to the fifth and the sixth component; the fifth component configured with control logic adapted to send the stored subsystem commands and subsystem responses received from the fourth component to the SSM; the sixth component configured with analysis logic adapted to compare the entries in the array received from the fourth component with the responses received from the SSM to determine whether a disagreement indicative of a regression failure exists, generate pass/fail results for the test file based on the existence of a disagreement, and store the entries of the array; and the seventh component configured with report logic adapted to generate a report using the pass/fail results from the sixth component; such that the record-replay module is operative to record the regression testing related information corresponding to each test case wherein each test case is characterized by the regression testing related information flowing through the system beginning with the regression testing related information defining at least one stimulus and ending with the regression testing related information defining the at least one signal response wherein a difference between regression testing related information corresponding to two test cases is indicative of a regression of the system.
-
Specification