Recursive use of model based test generation for middlevare validation

• US 20030196191A1
• Filed: 04/16/2002
• Published: 10/16/2003
• Est. Priority Date: 04/16/2002
• Status: Active Grant

• Current Assignee: LinkedIn Corporation (Microsoft Corporation)
• Original Assignee: International Business Machines Corporation
• Inventors: Sharon, Guy, Nagin, Kenneth, Hartman, Alan
• Granted Patent: US 7,117,484 B2
• Time in Patent Office: Days
• US Class Current: 717/126
• CPC Class Codes: G06F 11/3684 for test design, e.g. generating new test cases

Abstract

Middleware is validated using a two layered modeling strategy for the generation of test suites. A high level model of the syntax of the programming language used to program the middleware generates a series of high level abstract programs. These high level abstract programs then undergo two separate translations. First they are translated directly into actual test programs, which are expressed in the programming language of the middleware. Second, they are translated into second models of the high level abstract programs. Running the model based test generator again on the second models produces a set of suites of test cases, each test case consisting of data inputs and predicted results for one of the actual test programs. The actual test programs are run repetitively using the middleware execution engine. The results are compared with the predicted results, using a generic validation engine.

Claims

1. A method of validating a computer application, comprising the steps of:

generating an abstract program in accordance with syntactic requirements of a computer application being validated;

performing a first translation of said abstract program into meta-data that is an acceptable input for said computer application;

performing a second translation of said abstract program into a behavioral model of said abstract program;

generating a test case from said behavioral model, said test case comprising data and predicted results;

stimulating said computer application with instructions of said meta-data and said data;

observing actual results produced by said computer application responsive to said step of stimulating; and

comparing said actual results with said predicted results.

2. The method according to claim 1, wherein said step of performing said second translation of said abstract program further comprises automatically generating a coverage criterion for said test case.

3. The method according to claim 1, wherein said test case comprises a plurality of test cases.

4. The method according to claim 1, wherein said meta-data comprises a plurality of test programs.

5. The method according to claim 1, wherein said behavioral model comprises a plurality of behavioral models.

6. The method according to claim 1, wherein said meta-data is expressed in a first language, and said behavioral model is expressed in a second language.

7. The method according to claim 1, wherein said syntactic requirements comprise an event definition.

8. The method according to claim 1, wherein said syntactic requirements comprise a lifespan definition.

9. The method according to claim 1, wherein said syntactic requirements comprise a situation definition.

10. The method according to claim 1, wherein said abstract program comprises a plurality of situations.

11. The method according to claim 1, wherein said step of performing a first translation is performed using a translation table.

12. The method according to claim 1, wherein said behavioral model comprises a state variable.

13. The method according to claim 1, wherein said behavioral model comprises a suite of behavioral models and a common set of rules.

14. The method according to claim 1, wherein said test case comprises a sequence of events.

15. The method according to claim 1, wherein said test case further comprises an indicator of a passage of time.

16. A computer software product, comprising a computer-readable medium in which computer program instructions are stored, which instructions, when read by a computer, cause the computer to perform a method of validating a computer application, comprising the steps of:

generating an abstract program in accordance with syntactic requirements of a computer application being validated;

performing a first translation of said abstract program into meta-data that is an acceptable input for said computer application;

performing a second translation of said abstract program into a behavioral model of said abstract program;

generating a test case from said behavioral model, said test case comprising data and predicted results;

stimulating said computer application with instructions of said meta-data and said data;

observing actual results produced by said computer application responsive to said step of stimulating; and

comparing said actual results with said predicted results.

17. The computer software product according to claim 16, wherein said step of performing said second translation of said abstract program further comprises automatically generating a coverage criterion for said test case.

18. The computer software product according to claim 16, wherein said test case comprises a plurality of test cases.

19. The computer software product according to claim 16, wherein said meta-data comprises a plurality of test programs.

20. The computer software product according to claim 16, wherein said behavioral model comprises a plurality of behavioral models.

21. The computer software product according to claim 16, wherein said meta-data is expressed in a first language, and said behavioral model is expressed in a second language.

22. The computer software product according to claim 16, wherein said syntactic requirements comprise an event definition.

23. The computer software product according to claim 16, wherein said syntactic requirements comprise a lifespan definition.

24. The computer software product according to claim 16, wherein said syntactic requirements comprise a situation definition.

25. The computer software product according to claim 16, wherein said abstract program comprises a plurality of situations.

26. The computer software product according to claim 16, wherein said step of performing a first translation is performed using a translation table.

27. The computer software product according to claim 16, wherein said behavioral model comprises a state variable.

28. The computer software product according to claim 16, wherein said behavioral model comprises a suite of behavioral models and a common set of rules.

29. The computer software product according to claim 16, wherein said test case comprises a sequence of events.

30. The computer software product according to claim 16, wherein said test case further comprises an indicator of a passage of time.

31. A method of validating middleware, comprising the steps of:

generating an abstract program in accordance with syntactic requirements of a middleware system application being validated;

performing a first translation of said abstract program into meta-data that is an acceptable input for said middleware system;

performing a second translation of said abstract program into a behavioral model of said abstract program;

generating a test case from said behavioral model, said test case comprising data and predicted results;

stimulating said middleware system with instructions of said meta-data and said data;

observing actual results produced by said middleware system responsive to said step of stimulating; and

comparing said actual results with said predicted results.

32. The method according to claim 31, wherein said step of performing said second translation of said abstract program further comprises automatically generating a coverage criterion for said test case.

33. The method according to claim 31, wherein said test case comprises a plurality of test cases.

34. The method according to claim 31, wherein said meta-data comprises a plurality of test programs.

35. The method according to claim 31, wherein said behavioral model comprises a plurality of behavioral models.

36. The method according to claim 31, wherein said meta-data is expressed in a first language, and said behavioral model is expressed in a second language.

37. The method according to claim 31, wherein said syntactic requirements comprise an event definition.

38. The method according to claim 31, wherein said syntactic requirements comprise a lifespan definition.

39. The method according to claim 31, wherein said syntactic requirements comprise a situation definition.

40. The method according to claim 31, wherein said abstract program comprises a plurality of situations.

41. The method according to claim 31, wherein said step of performing a first translation is performed using a translation table.

42. The method according to claim 31, wherein said behavioral model comprises a state variable.

43. The method according to claim 31, wherein said behavioral model comprises a suite of behavioral models and a common set of rules.

44. The method according to claim 31, wherein said test case comprises a sequence of events.

45. The method according to claim 31, wherein said test case further comprises an indicator of a passage of time.

46. A computer software product, comprising a computer-readable medium in which computer program instructions are stored, which instructions, when read by a computer, cause the computer to perform a method of validating middleware, comprising the steps of:

generating an abstract program in accordance with syntactic requirements of a middleware system being validated;

performing a first translation of said abstract program into meta-data that is an acceptable input for said middleware system;

performing a second translation of said abstract program into a behavioral model of said abstract program;

generating a test case from said behavioral model, said test case comprising data and predicted results;

stimulating said middleware system with instructions of said meta-data and said data;

observing actual results produced by said middleware system responsive to said step of stimulating; and

comparing said actual results with said predicted results.

47. The computer software product according to claim 46, wherein said step of performing said second translation of said abstract program further comprises automatically generating a coverage criterion for said test case.

48. The computer software product according to claim 46, wherein said test case comprises a plurality of test cases.

49. The computer software product according to claim 46, wherein said meta-data comprises a plurality of test programs.

50. The computer software product according to claim 46, wherein said behavioral model comprises a plurality of behavioral models.

51. The computer software product according to claim 46, wherein said meta-data is expressed in a first language, and said behavioral model is expressed in a second language.

52. The computer software product according to claim 46, wherein said syntactic requirements comprise an event definition.

53. The computer software product according to claim 46, wherein said syntactic requirements comprise a lifespan definition.

54. The computer software product according to claim 46, wherein said syntactic requirements comprise a situation definition.

55. The computer software product according to claim 46, wherein said abstract program comprises a plurality of situations.

56. The computer software product according to claim 46, wherein said step of performing a first translation is performed using a translation table.

57. The computer software product according to claim 46, wherein said behavioral model comprises a state variable.

58. The computer software product according to claim 46, wherein said behavioral model comprises a suite of behavioral models and a common set of rules.

59. The computer software product according to claim 46, wherein said test case comprises a sequence of events.

60. The computer software product according to claim 46, wherein said test case further comprises an indicator of a passage of time.

61. An apparatus for validating a computer application, comprising:

a test generator for generating an abstract program in accordance with syntactic requirements of a computer application being validated;

a translation module for performing a first translation of said abstract program into meta-data that is an acceptable input for said computer application, and for performing a second translation of said abstract program into a behavioral model of said abstract program, said test generator being adapted for generating a test case from said behavioral model, said test case comprising data and predicted results;

an execution engine for stimulating said computer application with instructions of said meta-data and said data, and for observing actual results produced by said computer application; and

a validation engine for comparing said actual results with said predicted results.

62. The apparatus according to claim 61, wherein said translation module automatically generates a coverage criterion for said test case.

63. The apparatus according to claim 61, wherein said test case comprises a plurality of test cases.

64. The apparatus according to claim 61, wherein said meta-data comprises a plurality of test programs.

65. The apparatus according to claim 61, wherein said behavioral model comprises a plurality of behavioral models.

66. The apparatus according to claim 61, wherein said meta-data is expressed in a first language, and said behavioral model is expressed in a second language.

67. The apparatus according to claim 61, wherein said syntactic requirements comprise an event definition.

68. The apparatus according to claim 61, wherein said syntactic requirements comprise a lifespan definition.

69. The apparatus according to claim 61, wherein said syntactic requirements comprise a situation definition.

70. The apparatus according to claim 61, wherein said abstract program comprises a plurality of situations.

71. The apparatus according to claim 61, wherein said behavioral model comprises a state variable.

72. The apparatus according to claim 61, wherein said behavioral model comprises a suite of behavioral models and a common set of rules.

73. The apparatus according to claim 61, wherein said test case comprises a sequence of events.

74. The apparatus according to claim 61, wherein said test case further comprises an indicator of a passage of time.

75. An apparatus for validating middleware, comprising:

a test generator for generating an abstract program in accordance with syntactic requirements of a middleware system being validated;

a translation module for performing a first translation of said abstract program into meta-data that is an acceptable input for said middleware system, and for performing a second translation of said abstract program into a behavioral model of said abstract program, said test generator being adapted for generating a test case from said behavioral model, said test case comprising data and predicted results;

an execution engine for stimulating said middleware system with instructions of said meta-data and said data, and for observing actual results produced by said middleware system; and

a validation engine for comparing said actual results with said predicted results.

76. The apparatus according to claim 75, wherein said translation module automatically generates a coverage criterion for said test case.

77. The apparatus according to claim 75, wherein said test case comprises a plurality of test cases.

78. The apparatus according to claim 75, wherein said meta-data comprises a plurality of test programs.

79. The apparatus according to claim 75, wherein said behavioral model comprises a plurality of behavioral models.

80. The apparatus according to claim 75, wherein said meta-data is expressed in a first language, and said behavioral model is expressed in a second language.

81. The apparatus according to claim 75, wherein said syntactic requirements comprise an event definition.

82. The apparatus according to claim 75, wherein said syntactic requirements comprise a lifespan definition.

83. The apparatus according to claim 75, wherein said syntactic requirements comprise a situation definition.

84. The apparatus according to claim 75, wherein said abstract program comprises a plurality of situations.

85. The apparatus according to claim 75, wherein said behavioral model comprises a state variable.

86. The apparatus according to claim 75, wherein said behavioral model comprises a suite of behavioral models and a common set of rules.

87. The apparatus according to claim 75, wherein said test case comprises a sequence of events.

88. The apparatus according to claim 75, wherein said test case further comprises an indicator of a passage of time.