Method and apparatus for automatically generating test data for code testing purposes

• US 20060064570A1
• Filed: 09/07/2004
• Published: 03/23/2006
• Est. Priority Date: 09/07/2004
• Status: Abandoned Application

• Current Assignee: Agitar Software Incorporated (Oridium LLC)
• Original Assignee: Agitar Software Incorporated (Oridium LLC)
• Inventors: di Savoia, Luigi Alberto Pio
• Application Number:
  US10/935,766
• Publication Number:
  US 20060064570A1
• Time in Patent Office: Days
• US Class Current: 712/227
• CPC Class Codes: G06F 11/3684 for test design, e.g. generating new test cases

Abstract

One embodiment of the present invention provides a system that automatically generates test data for code testing purposes. During operation, the system receives code under test (CUT). The system then determines type information for one or more parameters for methods of the CUT. Next, the system automatically selects, based on the type information, one or more test data factories (TDFs) to generate test data for parameters of the CUT.

Claims

1. A method for automatically generating test data for code testing purposes, comprising:

receiving code under test (CUT);

determining type information for one or more parameters for methods of the CUT; and

automatically selecting, based on the type information, one or more test data factories (TDFs) to generate test data for parameters of the CUT.

2. The method of claim 1,

wherein automatically selecting one or more TDFs involves automatically selecting one or more test-data directives (TDDs);

and

wherein a TDD may specify one or more TDFs to be used to generate test data and may specify the manner in which a TDF is used.

3. The method of claim 2, wherein automatically selecting the TDDs involves:

applying a number of generalized test data rules (GTDRs) to the CUT, wherein a GTDR specifies a test data condition (TDC) and specifies one or more TDDs to be used if the CUT satisfies the TDC, and wherein a TDC includes at least one predicate; and

if the CUT satisfies the TDC specified in a GTDR, automatically selecting the TDD(s) specified by the GTDR.

4. The method of claim 3, further comprising evaluating how frequently a TDD has been selected by a user to generate test data for CUT which satisfies a predicate;

and

wherein if a TDD has been selected by a user sufficiently frequently to generate test data for CUT which satisfies a predicate, the method further comprises constructing a GTDR which includes the predicate in a TDC and which specifies the TDD.

5. The method of claim 4, wherein evaluating how frequently a TDD has been selected by a user to generate test data for CUT which satisfies a predicate involves computing a user-selection ratio for this predicate-TDD combination, which is the ratio of

the number of times a user has selected this TDD to generate test data for CUT which satisfies this predicate, to

the number of times CUT satisfies this predicate.

6. The method of claim 5, further comprising:

obtaining a new predicate from CUT, wherein one or more TDDs have been confirmed, selected, or provided by a user for this CUT; and

computing an updated user-selection ratio for a combination of this predicate and a TDD.

7. The method of claim 6, wherein obtaining the predicate from CUT involves applying one or more generic predicates without specific parameters to the CUT to obtain one or more predicates with specific parameters.

8. The method of claim 6, further comprising ranking GTDRs based on the user-selection ratio of the predicate-TDD combination included in each GTDR.

9. The method of claim 3, wherein if the user-selection ratio of a predicate-TDD combination falls below a given threshold, the method further comprises deleting a corresponding GTDR which includes this predicate and this TDD.

10. The method of claim 1, further comprising presenting the automatically selected TDFs to a user, and allowing the user to choose TDFs from the presented TDFs.

11. The method of claim 10, wherein presenting the automatically selected TDFs to the user involves presenting the TDFs on a host which is different from the host where the TDFs reside.

12. The method of claim 1, further comprising allowing a user to choose TDFs from a set of additional TDFs which are not automatically selected.

13. The method of claim 1, further comprising allowing a user to provide new TDDs and/or new TDFs.

14. The method of claim 13, wherein if the user provides one or more new TDDs and/or TDFs, the method further comprises storing the user-provided TDDs and/or TDFs so that these TDDs and/or TDFs may be used for future tests.

15. A computer-readable storage medium storing instructions that when executed by a computer cause the computer to perform a method for automatically generating test data for code testing purposes, the method comprising:

receiving CUT;

determining type information for one or more parameters for methods of the CUT; and

automatically selecting, based on the type information, one or more TDFs to generate test data for parameters of the CUT.

16. The computer-readable storage medium of claim 15,

wherein automatically selecting one or more TDFs involves automatically selecting one or more TDDs;

and

wherein a TDD may specify one or more TDFs to be used to generate test data and may specify the manner in which a TDF is used.

17. The computer-readable storage medium of claim 16, wherein automatically selecting the TDDs involves:

applying a number of generalized test data rules (GTDRs) to the CUT, wherein a GTDR specifies a test data condition (TDC) and specifies one or more TDDs to be used if the CUT satisfies the TDC, and wherein a TDC includes at least one predicate; and

if the CUT satisfies the TDC specified in a GTDR, automatically selecting the TDD(s) specified by the GTDR.

18. The computer-readable storage medium of claim 17, wherein the method further comprises evaluating how frequently a TDD has been selected by a user to generate test data for CUT which satisfies a predicate;

and

wherein if a TDD has been selected by a user sufficiently frequently to generate test data for CUT which satisfies a predicate, the method further comprises constructing a GTDR which includes the predicate in a TDC and which specifies the TDD.

19. The computer-readable storage medium of claim 18, wherein evaluating how frequently a TDD has been selected by a user to generate test data for CUT which satisfies a predicate involves computing a user-selection ratio for this predicate-TDD combination, which is the ratio of

the number of times a user has selected this TDD to generate test data for CUT which satisfies this predicate, to

the number of times CUT satisfies this predicate.

20. The computer-readable storage medium of claim 19, wherein the method further comprises:

obtaining a new predicate from CUT, wherein one or more TDDs have been confirmed, selected, or provided by a user for this CUT; and

computing an updated user-selection ratio for a combination of this predicate and a TDD.

21. The computer-readable storage medium of claim 20, wherein obtaining the predicate from CUT involves applying one or more generic predicates without specific parameters to the CUT to obtain one or more predicates with specific parameters.

22. The computer-readable storage medium of claim 20, wherein the method further comprises ranking GTDRs based on the user-selection ratio of the predicate-TDD combination included in each GTDR.

23. The computer-readable storage medium of claim 17, wherein if the user-selection ratio of a predicate-TDD combination falls below a given threshold, the method further comprises deleting a corresponding GTDR which includes this predicate and this TDD.

24. The computer-readable storage medium of claim 15, wherein the method further comprises presenting the automatically selected TDFs to a user, and allowing the user to choose TDFs from the presented TDFs.

25. The computer-readable storage medium of claim 24, wherein presenting the automatically selected TDFs to the user involves presenting the TDFs on a host which is different from the host where the TDFs reside.

26. The computer-readable storage medium of claim 15, wherein the method further comprises allowing a user to choose TDFs from a set of additional TDFs which are not automatically selected.

27. The computer-readable storage medium of claim 15, wherein the method further comprises allowing a user to provide new TDDs and/or new TDFs.

28. The computer-readable storage medium of claim 27, wherein if the user provides one or more new TDDs and/or TDFs, the method further comprises storing the user-provided TDDs and/or TDFs so that these TDDs and/or TDFs may be used for future tests.

29. An apparatus for automatically generating test data for code testing purposes, comprising:

a receiving mechanism configured to receive CUT; and

a selection mechanism configured to;

determine type information for one or more parameters for methods of the CUT; and

to automatically select, based on the type information, one or more TDFs to generate test data for parameters of the CUT.

30. The apparatus of claim 29,

wherein while automatically selecting one or more TDFs, the selection mechanism is configured to automatically select one or TDDs;

and

wherein a TDD may specify one or more TDFs to be used to generate test data and may specify the manner in which a TDF is used.

31. The apparatus of claim 30, wherein while automatically selecting the TDDs, the selection mechanism is configured to:

apply a number of generalized GTDRs to the CUT, wherein a GTDR specifies a test data condition (TDC) and specifies one or more TDDs to be used if the CUT satisfies the TDC, and wherein a TDC includes at least one predicate; and

if the CUT satisfies the TDC specified in a GTDR, to automatically select the TDD(s) specified by the GTDR.

32. The apparatus of claim 31, wherein the selection mechanism is further configured to evaluate how frequently a TDD has been selected by a user to generate test data for CUT which satisfies a predicate;

and

wherein if a TDD has been selected by a user sufficiently frequently to generate test data for CUT which satisfies a predicate, the selection mechanism is further configured to construct a GTDR which includes the predicate in a TDC and which specifies the TDD.

33. The apparatus of claim 32, wherein while evaluating how frequently a TDD has been selected by a user to generate test data for CUT which satisfies a predicate, the selection mechanism is configured to compute a user-selection ratio for this predicate-TDD combination, which is the ratio of

the number of times a user has selected this TDD to generate test data for CUT which satisfies this predicate, to

the number of times CUT satisfies this predicate.

34. The apparatus of claim 33, wherein the selection mechanism is further configured to:

obtain a new predicate from CUT, wherein one or more TDDs have been confirmed, selected, or provided by a user for this CUT; and

to compute an updated user-selection ratio for a combination of this predicate and a TDD.

35. The apparatus of claim 34, wherein while obtaining the predicate from CUT, the selection mechanism is configured to apply one or more generic predicates without specific parameters to the CUT to obtain one or more predicates with specific parameters.

36. The apparatus of claim 34, wherein the selection mechanism is further configured to rank GTDRs based on the user-selection ratio of the predicate-TDD combination included in each GTDR.

37. The apparatus of claim 31, wherein if the user-selection ratio of a predicate-TDD combination falls below a given threshold, the selection mechanism is further configured to delete a corresponding GTDR which includes this predicate and this TDD.

38. The apparatus of claim 29, further comprising a user interface configured to present the automatically selected TDFs to a user, and to allow the user to choose TDFs from the presented TDFs.

39. The apparatus of claim 38, wherein wile presenting the automatically selected TDFs to the user, the user interface is configured to present the TDFs on a host which is different from the host where the TDFs reside.

40. The apparatus of claim 29, further comprising a user interface configured to allow a user to choose TDFs from a set of additional TDFs which are not automatically selected.

41. The apparatus of claim 29, further comprising a user interface configured to allow a user to provide new TDDs and/or new TDFs.

42. The apparatus of claim 41, wherein if the user provides one or more new TDDs and/or TDFs, the apparatus further comprises a storage mechanism configured to store the user-provided TDDs and/or TDFs so that these TDDs and/or TDFs may be used for future tests.