Method and apparatus for regression testing of application logic

• US 5,758,062 A
• Filed: 04/30/1996
• Issued: 05/26/1998
• Est. Priority Date: 04/30/1996
• Status: Expired due to Term

• Current Assignee: Oracle International Corporation (Oracle Corporation)
• Original Assignee: Oracle Corporation
• Inventors: McMahon, Douglas James, Buzsaki, George
• Primary Examiner(s): Beausoliel, Jr., Robert W.
• Assistant Examiner(s): Baderman, Scott T.
• Application Number:
  US08/640,247
• Time in Patent Office: 756 Days
• Field of Search: 395/183.14, 395/183.13, 395/183.15, 395/185.02, 395/183.21, 395/183.09, 364/193
• US Class Current: 714/38.11
• CPC Class Codes: G06F 11/3414 Workload generation, e.g. scripts, playback
• CPC Class Codes: G06F 11/3688 for test execution, e.g. scheduling of test suites
• CPC Class Codes: G06F 2201/86 Event-based monitoring
• CPC Class Codes: G06F 2201/865 Monitoring of software

Abstract

A method and system for regression testing of application logic within a software system. Software architecture can be separated into a presentation layer (e.g., user interface), an application layer (e.g., business logic), and a data/storage layer. Within the novel testing system, communication interfaces between the presentation layer and the application layer are identified such that user initiated commands ("events") passed from the presentation layer to the application layer are recorded into a test playback event file. During test playback event recordation, differences in the state of the application layer of a reference program are collected into a reference state log file. To test a program, the recorded events of the test playback event file are played back and applied directly to its application layer. After each event, the differences in the state of the program are collected into a test state log file. The reference and test state log files are compared and differences are stored in an error or summary difference file. By recording events that enter the application layer, in lieu of the user entered commands that enter the presentation layer, the novel testing system is insensitive to changes in the presentation layer and allows more direct testing and error detection of the application layer. By recording differences in the state of the application layer on an event by event basis, in lieu of recording differences in information presented to the user, the novel system allows more effective error detection and error isolation of the application layer.

Claims

1. In a computer system having a processor and a computer readable memory, a method of performing a regression test comprising the computer implemented steps of:

a) recording detected application events into a playback event file, said application events originating from a presentation layer and passed to an application layer of a first version of a software program;

b) recording a first response of said first version of said software program to said application events and storing said first response in said computer readable memory;

c) replaying said playback file and directly applying said application events stored therein to an application layer of a second version of said software program;

d) recording a second response, said second response being a response of said application layer of said second version of said software program to said application events recorded by step a); and

e) determining differences between said first and second versions of said software program by comparing said first response and said second response and determining differences therein.

2. A method as described in claim 1 wherein said step b) comprises the steps of:

sequentially applying said application events received from said presentation layer to said application layer of said first version of said software program;

recording a respective previous state of said first version of said software program before application of each application event thereto;

recording a respective current state of said first version of said software program after application of each application event thereto;

generating a respective state difference record for each application event by comparing each current state with each previous state and recording differences therein; and

storing each state difference record into a reference state log file as said first response.

3. A method as described in claim 2 wherein said step d) comprises the steps of:

sequentially applying said application events of said playback event file to said application layer of said second version of said software program;

recording a respective previous state of said second version of said software program before application of each application event thereto;

recording a respective current state of said second version of said software program after application of each application event thereto;

generating a respective state difference record for each application event by comparing each current state of said second version of said software program with each previous state of said second version of said software program and recording differences therein; and

storing each state difference record into a test state log file as said second response.

4. A method as described in claim 1 wherein said step a) comprises the steps of:

monitoring identified communication pathways that couple said presentation layer to said application layer of said first version of said software program to detect application events communicated to said application layer; and

recording said application events detected within said communication pathways.

5. A method as described in claim 4 wherein said step c) comprises the steps of:

accessing said playback event file to retrieve application events recorded therein; and

applying said application events through said communication pathways to said application layer of said second version of said software program without interaction with a presentation layer of said second version of said software program.

6. A method as described in claim 1 wherein said step b) comprises the steps of:

generating a plurality of state difference records representative of said first response; and

representing information of each state difference record in a highly structured format that closely resembles a format used by said first version of said software program to internally represent information.

7. A method as described in claim 1 wherein said step d) comprises the steps of:

generating a plurality of state difference records representative of said second response; and

representing information of each state difference record in a highly structured format that closely resembles a format used by said second version of said software program to internally represent information.

8. A method as described in claim 1

wherein said step b) comprises the step of recording a plurality of state difference records as said first response, a respective state difference record of said first response recorded for each application event originating from said presentation layer, and

wherein said step d) comprises the step of recording a plurality of state difference records as said second response, a respective state difference record of said second response recorded for each application event originating from said playback event file, and

wherein said step e) comprises the step of individually comparing respective state difference records of same application events to determine differences between said first response and said second response.

9. A method as described in claim 8 wherein said step e) comprises the steps of:

receiving an ignore file comprising a sequence of ignore commands which identify information that produces irrelevant mismatches between said first response and said second response; and

ignoring differences between said respective state difference records of same application events provided said differences are identified within said ignore file.

10. In a computer system, a method of performing a regression test comprising the computer implemented steps of:

a) recording application events into a playback event file, said application events originating from a presentation layer of a first version of a software program and passed to an application layer of said first version of said software program;

b) recording a response of said first version of said software program to said application events and storing said response as a reference state log file in a computer readable memory wherein said reference state log file comprises a plurality of state difference records, a respective state difference record recorded for each application event originating from said presentation layer;

c) replaying said playback event file and directly applying said application events stored therein to an application layer of a second version of said software program;

d) in response to step c), recording a response of said application layer of said second version of said software program to said application events and storing said response as a test state log file in said computer readable memory, wherein said test state log file comprises a plurality of state difference records, a respective state difference record recorded for each application event; and

e) determining differences between said first and second version of said software program by comparing said reference state log file and said test state log file and determining differences therein.

11. A method as described in claim 10 wherein said step b) comprises the steps of:

sequentially applying said application events from said presentation layer to said application layer of said first version of said software program;

determining a respective previous internal state of said first version of said software program before application of each application event thereto;

determining a respective current internal state of said first version of said software program after application of each application event thereto; and

generating a respective state difference record with respect to each application event by comparing each current internal state with each previous internal state and recording differences therein.

12. A method as described in claim 11 wherein said step d) comprises the steps of:

sequentially applying said application events from said playback event file to said application layer of said second version of said software program;

determining a respective previous internal state of said second version of said software program before application of each application event thereto;

determining a respective current internal state of said second version of said software program after application of each application event thereto; and

generating a respective state difference record with respect to each application event by comparing each current internal state of said second version of said software program with each previous internal state of said second version of said software program and recording differences therein.

13. A method as described in claim 10 wherein said step a) comprises the steps of:

monitoring identified communication pathways that couple said presentation layer to said application layer of said first version of said software program to detect application events communicated to said application layer of said first version of said software program; and

recording said application events detected within said communication pathways.

14. A method as described in claim 13 wherein said step c) comprises the steps of:

accessing said playback event file to retrieve application events recorded therein; and

applying said application events through said communication pathways to said application layer of said second version of said software program without interaction with a presentation layer of said second version of said software program.

15. A method as described in claim 10 wherein information within each state difference record is stored in a highly structured format that closely resembles a format used by said first and second versions of said software program to internally represent information.

16. A method as described in claim 10 wherein said step e) comprises the steps of:

individually comparing respective state difference records of same application events to determine differences between said reference state log file and said test state log file;

receiving an ignore file comprising a sequence of ignore commands which identify information that produces irrelevant mismatches between said reference state log file and said test state log file; and

ignoring differences between said respective state difference records of same application events provided said differences are identified within said ignore file.

17. A computer system comprising:

a processor;

an address/data bus coupled to said processor;

a computer readable memory coupled to communicate with said processor;

said processor for performing step a) of recording application events to a playback event file, said application events originating from a presentation layer of a first version of a software program and passed over a communication pathway to an application layer of said first version of said software program;

said processor for performing step b) of recording a first response of said first version of said software program to said application events and storing said first response in said computer readable memory;

said processor for performing step c) of replaying said playback file and directly applying said application events stored therein to an application layer of a second version of said software program;

said processor for performing step d) of recording a second response, said second response being a response of said application layer of said version of said second software program to said application events; and

said processor for also performing step e) of determining differences between said first and second versions of said software program by comparing said first response and said second response and determining differences therein.

18. A computer system as described in claim 17 wherein said processor performs said step b) by:

said processor sequentially applying said application events to said application layer of said first version of said software program;

said processor recording a respective previous internal state of said first version of said software program before application of each application event thereto;

said processor recording a respective current internal state of said first version of said software program after application of each application event thereto;

said processor generating a respective state difference record for each application event by comparing each current internal state with each previous internal state and recording differences therein; and

said processor storing each state difference record into a reference state log file as said first response.

19. A computer system as described in claim 18 wherein said processor performs said step d) by:

said processor sequentially applying said application events of said playback event file to said application layer of said second version of said software program;

said processor recording a respective previous internal state of said second version of said software program before application of each application event thereto;

said processor recording a respective current internal state of said second version of said software program after application of each application event thereto;

said processor generating a respective state difference record for each application event by comparing each current internal state of said second version of said software program with each previous internal state of said second version of said software program and recording differences therein; and

said processor storing each state difference record into a test state log file as said second response.

20. A computer system as described in claim 17 wherein said processor performs said step c) by:

said processor accessing said playback event file to retrieve application events recorded therein; and

said processor applying said application events through said communication pathways to said application layer of said second version of said software program without interaction with a presentation layer of said second version of said software program.

21. A computer system as described in claim 17 wherein said processor performs said step b) by said processor generating a plurality of state difference records representative of said first response, and wherein said processor performs said step d) by generating a plurality of state difference records representative of said second response, and wherein information within each state difference record is represented in a highly structured format that closely resembles a format used by said first and second versions of said software program to internally represent information.

22. A computer system as described in claim 17

wherein said processor performs said step b) by recording a plurality of state difference records as said first response, a respective state difference record of said first response recorded for each application event originating from said presentation layer,

wherein said processor performs said step d) by recording a plurality of state difference records as said second response, a respective state difference record of said second response recorded for each application event originating from said playback event file, and

wherein said processor performs said step e) by:

said processor individually comparing respective state difference records of same application events to determine differences between said first response and said second response;

said processor receiving an ignore file comprising a sequence of ignore commands which identify information that produces irrelevant mismatches between said first response and said second response; and

said processor ignoring differences between said respective state difference records of same application events provided said differences are identified within said ignore file.

23. In a computer system having a processor coupled to a bus, a computer readable memory for communicating with said processor and containing a sequence of instructions for causing said computer system to perform a regression test, said regression test comprising the steps of:

a) recording application events to a playback event file, said application events originating from a presentation layer of a first version of said software program and passed to an application layer of said first version of said software program;

b) recording a first response of said first version of said software program to said application events and storing said first response;

c) replaying said playback file and directly applying said application events stored therein to an application layer of a second version of said software program;

d) in response to step c), recording a response of said application layer of said second version of said software program to said application events as a second response; and

e) determining differences between said first and second versions of said software program by comparing said first response and said second response to determine differences therein.

24. A computer readable memory as described in claim 23 wherein said step b) comprises the steps of:

sequentially applying said application events to said application layer of said first version of said software program;

recording a respective previous internal state of said first version of said software program before application of each application event thereto;

recording a respective current internal state of said first version of said software program after application of each application event thereto;

generating a respective state difference record for each application event by comparing each current internal state with each previous internal state and recording differences therein; and

storing each state difference record into a reference state log file as said first response.

25. A computer readable memory as described in claim 24 wherein said step d) comprises the steps of:

sequentially applying said application events of said playback event file to said application layer of said second version of said software program;

recording a respective previous internal state of said second version of said software program before application of each application event thereto;

recording a respective current internal state of said second version of said software program after application of each application event thereto;

generating a respective state difference record for each application event by comparing each current internal state of said second version of said software program with each previous internal state of said second version of said software program and recording differences therein; and

storing each state difference record into a test state log file as said second response.

26. A computer readable memory as described in claim 23 wherein said step a) comprises the steps of:

monitoring identified communication pathways that couple said presentation layer to said application layer of said first version of said software program to detect application events communicated to said application layer of said first version of said software program; and

recording said application events detected within said communication pathways.

27. A computer readable memory as described in claim 26 wherein said step c) comprises the steps of:

accessing said playback event file to retrieve application events recorded therein; and

applying said application events through said communication pathways to said application layer of said second version of said software program without interaction with a presentation layer of said second version of said software program.

28. A computer readable memory as described in claim 23 wherein said step b) comprises steps of:

generating a plurality of state difference records representative of said first response; and

representing information of each state difference record in a highly structured format that closely resembles a format used by said first version of said software program to internally represent information.

29. A computer readable memory as described in claim 23 wherein said step d) comprises the steps of:

generating a plurality of state difference records representative of said second response; and

representing information of each state difference record in a highly structured format that closely resembles a format used by said second version of said software program to internally represent information.

30. A computer readable memory as described in claim 23

wherein said step b) comprises the step of recording a plurality of state difference records as said first response, a respective state difference record of said first response recorded for each application event originating from said presentation layer, and

wherein said step d) comprises the step of recording a plurality of state difference records as said second response, a respective state difference record of said second response recorded for each application event originating from said playback file, and

wherein said step e) comprises the steps of:

individually comparing respective state difference records of same application events to determine differences between said first response and said second response;

receiving an ignore file comprising a sequence of ignore commands which identify information that produces irrelevant mismatches between said first response and said second response; and

ignoring differences between said respective state difference records of same application events provided said differences are identified within said ignore file.