System and method for facilitating programmable coverage domains for a testcase generator

• US 7,240,243 B2
• Filed: 03/28/2002
• Issued: 07/03/2007
• Est. Priority Date: 03/28/2002
• Status: Expired due to Fees

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: International Business Machines Corporation
• Inventors: Decker, Mark H.
• Primary Examiner(s): Beausoliel; Robert W
• Assistant Examiner(s): Ehne; Charles
• Application Number:
  US10/113,319
• Publication Number:
  US 20030188224A1
• Time in Patent Office: 1,923 Days
• Field of Search: 714/32, 714/33, 714/37, 714/41, 714/741, 717/143
• US Class Current: 714/33
• CPC Class Codes: G06F 11/261 by simulating additional hardware, e.g. fault simulation
• CPC Class Codes: G06F 11/263 Generation of test inputs, e.g. test vectors, patterns or sequences ; with adaptation of the tested hardware for testability with external testers

Abstract

The invention relates to a system and method for facilitating programmable coverage domains for test case generation, feedback, and measurement. The system comprises a domain definition input file; user-defined coverage domain data entered into the domain definition input file; and a parser operable for translating the user-defined coverage domain data into machine-readable computer program code. The system further includes an internal coverage domain comprising: a union of enabled coverage domains extracted from the user-defined coverage domain data; a session component comprising a session update count for each domain element; and a history component comprising a history update count for each domain element. The system further comprises a testcase generator including an internal coverage feedback and measurement system. The testcase generator comprises an instruction generator operable for generating instructions for use in a testcase and an instruction simulator operable for simulating execution of the instructions. The testcase generator accesses information in the internal coverage domain via the internal coverage feedback and measurement system, generates testcases for a current generation session, and updates the internal coverage domain according to the results of the testcase execution. The invention also comprises a method and storage medium.

Claims

1. A system for facilitating programmable coverage domains, comprising:

a domain definition input file;

user-defined coverage domain data entered into said domain definition input file;

a parser operable for translating said user-defined coverage domain data into machine-readable computer program code;

an internal coverage domain comprising;

a union of enabled coverage domains extracted from said user-defined coverage domain data;

a session component comprising a session update count for each domain element; and

a history component comprising a history update count for each domain element; and

a testcase generator including an internal coverage feedback and measurement system, said testcase generator comprising;

an instruction generator operable for generating instructions for use in a testcase; and

an instruction simulator operable for simulating execution of said instructions;

wherein said testcase generator accesses information in said internal coverage domain via said internal coverage feedback and measurement system, said internal coverage domain guiding testcase selections and updating said internal coverage domain according to results of testcase executions.

2. The system of claim 1, further comprising an instruction template including features operable for specifying instructions for generation;

wherein said features comprise;

a meta-mnemonic operable for specifying a coverage selected instruction;

a constraint operable for specifying a coverage to select initial state of machine; and

a coverage policy.

3. The system of claim 2, wherein said meta-mnemomc for a coverage selected instruction causes said coverage feedback system to select a next instruction, wherein a meta-mnemonic ID is dynamically resolved at each generation cycle based on a current state of said internal coverage domain.

4. The system of claim 2, wherein said constraint for coverage to select initial state of machine causes said coverage feedback system to select initial settings of machine in order to set same attributes of a selected domain element operable for huffing uncovered domain elements in different modes.

5. The system of claim 2, wherein said coverage policy refers to a constraint for determining which coverage policy to utilize and includes:

uncovered operable for selecting uncovered domain elements including;

session; and

cumulative;

uniform operable for selecting domain elements with a uniform distribution including;

session; and

cumulative;

uncovered-then-switch-to-uniform operable for selecting uncovered domain elements until all have been covered, then switches to uniform distribution; and

user supplied probability operable for applying a user-supplied probability for a domain element.

6. The system of claim 1, wherein said user-defined coverage domain data is entered via features provided by the domain definition input file, said features including:

a verb item operable for specifying a manner of handling a domain, said verb item comprising at least one of;

enable;

disable; and

display;

an instruction identifier;

a branch resolution type;

an addressing mode;

an architecture mode;

a translation mode;

a level identifier mode;

a type of domain; and

a sequence identifier.

7. The system of claim 6, wherein:

enable causes an associated coverage domain to be added to a list of valid coverage domains;

disable causes an associated coverage domain to be added to a list of invalid coverage domains; and

display causes an associated coverage domain to be added to a list of coverage domains requiring measurement reports.

8. The system of claim 6, wherein said instruction identifier comprises classifications including:

an opcode classification operable for specifying a hexadecimal representation;

a mnemonic classification operable for specifying at least one of an alphanumeric representation and a meta-mnemonic representation;

a class field operable for specifying an instruction'"'"'s membership in an instruction class, said instruction class specified as at least one of;

a general;

a control;

a floating point;

a hex floating point;

a binary floating point;

an input/output value;

a CC setting;

a PSW reference;

a store type; and

a privileged type; and

a format classification operable for specifying an instruction format type.

9. The system of claim 6, wherein:

said branch resolution type enables a user to specify branch conditions applicable to branch instructions;

said translation mode enables a user to specify at least one of;

a real;

a primary;

a secondary;

a home; and

an access register;

said level identifier mode enables a user to specify at least one of;

a host; and

a guest;

said type of domain enables a user to specify at least one of;

an exception;

a resulting condition code; and

an instruction side effect including trace area entry, target addressing mode, cases, and sequence; and

said sequence identifier enables a user to specify one of;

an instruction sequence; and

no sequence wherein a time order of elements in the domain is identified.

10. The system of claim 1, wherein said session update count records a number of successful occurrences of said each domain element for a current generation session.

11. The system of claim 1, wherein said history update count records a number of successful occurrences of said each domain element from user-selected testcases executed in at least one prior generation session.

12. The system of claim 1, further comprising a prior cumulative history database operable for storing a collective history of specified domain data from previously-generated testcases.

13. A method for facilitating programmable coverage domains, comprising:

entering user-defined coverage domain data by at least one user into a domain definition input file;

translating said user-defined coverage domain data into machine-readable computer program code; and

creating an internal coverage domain comprising;

a union of enabled coverage domains extracted from said user-defined coverage domain data;

a session update count for each domain element; and

a history update count for each domain element;

wherein a testcase generator generates instructions for use in a testcase and a simulator simulates execution of the instructions; and

wherein further, the testcase generator accesses information in said internal coverage domain, said internal coverage domain guiding testcase selections and updating said internal coverage domain according to results of the testcase executions.

14. The method of claim 13, further comprising:

specifying at least one instruction for generation via features provided by an instruction template, said features comprising;

a meta-mnemonic operable for specifying a coverage selected instruction;

a constraint operable for specifying a coverage to select initial state of machine; and

a coverage policy.

15. The method of claim 14, wherein said meta-mnemonic for a coverage selected instruction causes a coverage feedback system to select a next instruction, and wherein further, a meta-mnemonic ID is dynamically resolved at each generation cycle based on a current state of said internal coverage domain.

16. The method of claim 14, wherein said constraint for coverage to select initial state of machine causes a coverage feedback system to select initial settings of machine in order to set same attributes of a selected domain element operable for hitting uncovered domain elements in different modes.

17. The method of claim 14, wherein said coverage policy refers to a constraint for determining which coverage policy to utilize and includes:

uncovered operable for selecting uncovered domain elements including;

session; and

cumulative;

uniform operable for selecting domain elements with a uniform distribution including;

session; and

cumulative;

uncovered-then-switch-to-uniform operable for selecting uncovered domain elements until all have been covered, then switches to uniform distribution; and

user supplied probability operable for applying a user-supplied probability for a domain element.

18. The method of claim 13, wherein entering said user-defined coverage domain data includes specifying:

a verb item operable for specifying a manner of handling a domain, said verb item comprising at least one of;

enable;

disable; and

display;

an instruction identifier;

a branch resolution type;

an addressing mode;

an architecture mode;

a translation mode;

a level identifier mode;

a type of domain; and

a sequence identifier.

19. A storage medium encoded with machine-readable computer program code for facilitating programmable coverage domains, the storage medium including instructions for causing a computer to implement a method comprising:

entering user-defined coverage domain data by at least one user into a domain definition input file;

translating said user-defined coverage domain data into machine-readable computer program code; and

creating an internal coverage domain comprising;

a union of enabled coverage domains extracted from said user-defined coverage domain data;

a session update count for each domain element; and

a history update count for each domain element;

wherein a testcase generator generates instructions for use in a testcase and a simulator simulates execution of the instructions; and

wherein further, the testcase generator accesses information in said internal coverage domain, said internal coverage domain guiding testcase selections and updating said internal coverage domain according to results of the testcase executions.

20. The storage medium of claim 19, further comprising instructions for causing said computer to implement:

specifying at least one instruction for generation via features provided by an instruction template, said features comprising;

a meta-mnemonic operable for specifying a coverage selected instruction;

a constraint operable for specifying a coverage to select initial state of machine; and

a coverage policy.

21. The storage medium of claim 20, wherein said meta-mnemonic for a coverage selected instruction causes a coverage feedback system to select a next instruction, and wherein farther, a meta-mnemomc ID is dynamically resolved at each generation cycle based on a current state of said internal coverage domain.

22. The storage medium of claim 20, wherein said constraint for coverage to select initial state of machine causes a coverage feedback system to select initial settings of machine in order to set same attributes of a selected domain element operable for hitting uncovered domain elements in different modes.

23. The storage medium of claim 20, wherein said coverage policy refers to a constraint for determining which coverage policy to utilize and includes:

uncovered operable for selecting uncovered domain elements including;

session; and

cumulative;

uniform operable for selecting domain elements with a uniform distribution including;

session; and

cumulative;

uncovered-then-switch-to-uniform operable for selecting uncovered domain elements until all have been covered, then switches to uniform distribution; and

user supplied probability operable for applying a user-supplied probability for a domain element.

24. The storage medium of claim 19, wherein entering said user-defined coverage domain data includes specifying:

a verb item operable for specifying a manner of handling a domain, said verb item comprising at least one of;

enable;

disable; and

display;

an instruction identifier;

a branch resolution type;

an addressing mode;

an architecture mode;

a translation mode;

a level identifier mode;

a type of domain; and

a sequence identifier.