Testing measurements

• US 7,299,453 B2
• Filed: 11/14/2002
• Issued: 11/20/2007
• Est. Priority Date: 12/20/2001
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Sluiman, Harm, Paternostro, Marcelo
• Primary Examiner(s): Zhen; Wei
• Assistant Examiner(s): DENG, ANNA CHEN
• Application Number:
  US10/294,945
• Publication Number:
  US 20030135843A1
• Time in Patent Office: 1,832 Days
• Field of Search: 717/115, 717124-129, 717/135, 714 46- 48, 714/38, 714/2, 714/4, 702/186, 703/22, 707/200, 709/224
• US Class Current: 717/124
• CPC Class Codes: G06F 11/3684 for test design, e.g. generating new test cases

Abstract

Embodiments of the invention include a testing measurement which output a non-binary test result for one or more steps or processes performed during execution of a test of a subject. The non-binary test result, which may be output by a verification point or a test script, provides a developer with an improved gauge of the subject under test. In embodiments of the present invention, a test script or verification point will, rather than simply generate a “pass” or “fail” (i.e., “successful” or “unsuccessful”) output, generate a non-boolean value providing data with improved correlation between the test output and the relative success of a test script. In some embodiments, each verification point or test script can be associated with a rule for determining the test output which can be easily modified. This aspect provides users with an ability to carefully develop and implement a test result rule which provides output which enhances test output data analysis.

Claims

1. A verification point for use in a test script, said verification point tangibly embodied in a tangible computer readable medium and adapted to output a non-binary value which indicates an actual result resulting from execution of a test command within the test scrip that is associated with the verification point, wherein the verification point comprises a verification rule, said verification rule generating said non-binary value from calculations performed by the verification rule on input received from at least one process;

wherein said verification rule defines (i) how output from said verification point is generated, (ii) at least one input used to generate said non-binary value, and (iii) a data type, wherein said non-binary value output by said verification point is defined by said data type.

2. A method for generating a test script, said method comprising:

creating a verification point for receiving data responsive to execution of an instruction, wherein said verification point is adapted to output a non-binary value representative of actual execution results of said instruction; and

inserting said verification point in said test script, wherein said verification point comprises a verification rule, said verification rule generating said non-binary value from calculations performed by said verification rule on input received from at least one process, wherein said verification rule defines (i) how output from said verification point is generated, (ii) at least one input used to generate said non-binary value, and (iii) a data type, wherein said non-binary value output by said verification point is defined by said data type.

3. A computer program product containing data and instructions tangibly embodied in a tangible computer readable medium, said data and instructions when executed by a computer system adapt said computer system to:

generate a test script for testing a computer application in whole or in part, wherein said test script comprises one or more verification points, wherein at least one of said one or more verification points is adapted to output a non-binary value which indicates an actual result resulting from execution of a test command within the test script that is associated with the at least one of said one or more verification points, wherein said test script outputs a plurality of non-boolean values, and wherein the at least one of said one or more verification points comprises a verification rule that defines (i) how output from said verification point is generated, (ii) at least one input used to generate said non-binary value, and (iii) at least one data type, wherein at least one of said plurality of non-binary values output by said verification point is defined by said at least one data type.

4. A computer program product containing data and instructions tangibly embodied in a tangible computer readable medium, said data and instructions when executed by a computer system adapt said computer system to:

create a verification point for receiving data responsive to execution of an instruction, wherein said verification point is adapted to output a non-binary value representative of actual execution results of said instruction; and

insert said verification point in said test script, wherein said verification point comprises a verification rule is further adapted to output a plurality of non-binary values as specified by the verification rule, said verification rule generating said non-binary value from calculations performed by said verification rule on input received from at least one process, wherein said verification rule defines (i) how output from said verification point is generated, (ii) at least one input used to generate said non-binary value, and (iii) a data type, wherein said non-binary value output by said verification point is defined by said data type.

5. A verification point for use in a test script, said verification point tangibly embodied in a tangible computer readable medium and comprising a means for output a non-binary value which indicates an actual result resulting from execution of a test command within the test scrip that is associated with the verification point, wherein the verification point further comprises a verification rule, said verification rule generating said non-binary value from calculations performed by the verification rule on input received from at least one process;

wherein said verification rule defines (i) how output from said verification point is generated, (ii) at least one input used to generate said non-binary value, and (iii) a data type, wherein means for outputting said non-binary value outputs data conforming to said data type.

6. A data processing apparatus, including a processor, for generating a test script for testing a computer application in whole or in part, said data processing apparatus adapted to generate a test script, wherein said test script comprises one or more verification points, wherein at least one of said one or more verification points comprises means for outputting a non-binary value which indicates an actual result resulting from execution of a test command within the test script that is associated with the at least one of said one or more verification points, wherein the at least one of said one or more verification points comprises means for outputting a plurality of non-binary values, and wherein the at least one of said one or more verification points comprises a verification rule that defines (i) how output from said verification point is generated, (ii) at least one input used to generate said non-binary value, and (iii) at least one data type, wherein at least one of said plurality of non-binary values output by said verification point is defined by said at least one data type.