Automated regression failure management system

• US 8,145,949 B2
• Filed: 06/16/2010
• Issued: 03/27/2012
• Est. Priority Date: 06/16/2010
• Status: Expired due to Fees

• Current Assignee: Avago Technologies International Sales Pte Limited (Broadcom, Inc.)
• Original Assignee: PLX Technology Incorporated (Broadcom, Inc.)
• Inventors: Silver, Jordan
• Primary Examiner(s): Beausoliel, Jr., Robert
• Assistant Examiner(s): Lottich, Joshua P
• Application Number:
  US12/816,573
• Publication Number:
  US 20110314334A1
• Time in Patent Office: 650 Days
• Field of Search: 714/25, 714/26, 714/37, 714/38.1, 716/106, 716/136
• US Class Current: 714/26
• CPC Class Codes: G06F 11/3688 for test execution, e.g. scheduling of test suites

Abstract

In a first embodiment of the present invention, a method for performing regression testing on a simulated hardware is provided, the method comprising: scanning a defect database for fixed signatures; retrieving all tests in a failing instance database that correspond to the fixed signatures from the defect database; running one or more of the retrieved tests; determining if any of the retrieved tests failed during running; and for any retrieved test that failed during running, refiling the failed retrieved tests in the failing instance database and placing one or more generalized signatures for the failed retrieved tests in the defect database.

Claims

1. A method for performing regression testing on a simulated hardware, the method comprising:

scanning a defect database for fixed signatures;

retrieving all tests in a failing instance database that correspond to the fixed signatures from the defect database;

running one or more of the retrieved tests;

determining if any of the retrieved tests failed during running; and

for any retrieved test that failed during running, refiling the failed retrieved tests in the failing instance database and placing one or more generalized signatures for the failed retrieved tests in the defect database.

2. The method of claim 1, wherein the defect database and the failing instance database are stored in the same physical database.

3. The method of claim 1, wherein the method is performed at periodic intervals.

4. The method of claim 1, wherein the method is performed dynamically as signatures are indicated as fixed.

5. The method of claim 1, wherein at least one entry in the failing instance database includes a seed value.

6. The method of claim 5, wherein if an entry corresponding to a retrieved test from the failing instance database contains a seed value, then the running one or more of the retrieved tests includes running the retrieved test corresponding to the entry containing a seed value using all seed values within a certain threshold value of the seed value identified in the entry.

7. The method of claim 1, wherein at least one entry in the failing instance database includes an address.

8. The method of claim 7, wherein if an entry corresponding to the retrieved test from the failing instance database contains an address, then the running one or more of the retrieved tests includes running the retrieved test corresponding to the entry containing an address using all addresses within a certain threshold value of the address identified in the entry.

9. The method of claim 1, wherein the running one or more of the retrieved tests includes running all of the retrieved tests.

10. A regression test suite including:

a processor;

a memory storing a defect database containing generalized signatures of failed tests;

a failing instance database containing an entry for each unique failed test;

a defect database scanner coupled to the defect database and configured to scan the defect database for fixed signatures;

a failing instance runner coupled to the defect database scanner and to the failing instance database and configured to retrieve all tests in the failing instance database that correspond to the fixed signatures from the defect database, and run one or more of the retrieved tests; and

a failed test refiler coupled to the failing instance runner, the defect database, and the failing instance database and configured to determine if any of the retrieved tests failed during running, and, for any retrieved test that failed during running, refile the failed retrieved tests in the failing instance database and place one or more generalized signatures for the failed retrieved tests in the defect database.

11. The regression test suite of claim 10, wherein the regression test suite operates to test a simulation of an integrated circuit.

12. The regression test suite of claim 11, wherein a user operates the regression test suite after making a fix to the simulation of the integrated circuit.

13. A hardware apparatus for performing regression testing on a simulated hardware, the

apparatus comprising:

means for scanning a defect database for fixed signatures;

means for retrieving all tests in a failing instance database that correspond to the fixed signatures from the defect database;

means for running one or more of the retrieved tests;

means for determining if any of the retrieved tests failed during running; and

means for, for any retrieved test that failed during running, refiling the failed retrieved tests in the failing instance database and placing one or more generalized signatures for the failed retrieved tests in the defect database.

14. The apparatus of claim 13, wherein at least one entry in the failing instance database includes a seed value.

15. The apparatus of claim 14, wherein if an entry corresponding to a retrieved

test from the failing instance database contains a seed value, then the means for running one or more of the retrieved tests includes means for running the retrieved test corresponding to the entry containing a seed value using all seed values within a certain threshold value of the seed value identified in the entry.

16. The apparatus of claim 13, wherein at least one entry in the failing instance database includes an address.

17. The apparatus of claim 16, wherein if an entry corresponding to the retrieved test from the failing instance database contains an address, then the means for running one or more of the retrieved tests includes means for running the retrieved test corresponding to the entry containing an address using all addresses within a certain threshold value of the address identified in the entry.

18. The apparatus of claim 13, wherein the running one or more of the retrieved tests includes running all of the retrieved tests.

19. A non-transitory program storage device readable by a machine tangibly embodying a program of instructions executable by the machine to perform a method for performing regression testing on a simulated hardware, the method comprising:

scanning a defect database for fixed signatures;

retrieving all tests in a failing instance database that correspond to the fixed signatures from the defect database;

running one or more of the retrieved tests;

determining if any of the retrieved tests failed during running; and

for any retrieved test that failed during running, refiling the failed retrieved tests in the failing instance database and placing one or more generalized signatures for the failed retrieved tests in the defect database.