Testing a software application used in a database system

• US 8,510,602 B2
• Filed: 07/23/2010
• Issued: 08/13/2013
• Est. Priority Date: 07/23/2010
• Status: Active Grant

• Current Assignee: Salesforce.com, Inc.
• Original Assignee: Salesforce.com, Inc.
• Inventors: Chen, Zhiqiang
• Primary Examiner(s): Lottich, Joshua P
• Application Number:
  US12/842,988
• Publication Number:
  US 20120023373A1
• Time in Patent Office: 1,117 Days
• Field of Search: 714/25, 714/38.1, 717124-127
• US Class Current: 714/38.1
• CPC Class Codes: G06F 11/368 for test version control, e.g. updating test cases to a new software version
• CPC Class Codes: G06F 11/3692 for test results analysis

Abstract

A method for testing a software application used in a database system. The method includes receiving multiple changes to the software application, and running a plurality of tests on the software application. The method further includes determining if any of the tests fail, and if any of the tests fail, identifying which changes caused the failures.

Claims

1. A method for testing a software application used in a database system, the method comprising:

receiving multiple successive versions of the software application where each successive version includes a single new change to the software application and changes from one or more prior versions of the software application;

running a plurality of tests on a version of the successive versions of the software application to identify one or more test failures; and

responsive to identifying the one or more test failures,

running a subset of the plurality of tests, where the subset comprises one or more tests that failed, on a first successive version of the software application;

responsive to determining that the first successive version fails the subset of the plurality of tests, determining that the first successive version caused the one or more test failures of the plurality of tests; and

responsive to determining that the first successive version passes the subset of the plurality of tests, running the subset of the plurality of tests on one or more remaining successive versions of the software application until a particular successive version fails the subset of the plurality of tests, where the particular successive version is determined to have caused the one or more test failures of the plurality of tests.

2. The method of claim 1, wherein the running of the plurality of tests occurs at predefined intervals.

3. The method of claim 1, wherein the subset of the plurality of tests is a subset of a full run, wherein the full run includes passing tests and failing tests.

4. The method of claim 1, wherein the running the plurality of tests is on a latest version of the successive versions of the software application .

5. The method of claim 4, wherein the running the subset of the plurality of tests comprises:

running a subset of the plurality of tests on each of the successive versions of the software application based on the test failures to identify which changes caused the test failure.

6. A non-transitory computer-readable storage medium having one or more instructions thereon for testing a software application used in a database system in a multi-tenant database system, the instructions when executed by one or more processors causing the one or more processors to:

receiving multiple successive versions of the software application where each successive version includes a single new change to the software application and changes from one or more prior versions of the software application;

running a plurality of tests on a version of the successive versions of the software application to identify one or more test failures; and

responsive to identifying the one or more test failures,

running a subset of the plurality of tests, where the subset comprises one or more tests that failed, on a first successive version of the software application;

responsive to determining that the first successive version fails the subset of the plurality of tests, determining that the first successive version caused the one or more test failures of the plurality of tests; and

responsive to determining that the first successive version passes the subset of the plurality of tests, running the subset of the plurality of tests on one or more remaining successive versions of the software application until a particular successive version fails the subset of the plurality of tests, where the particular successive version is determined to have caused the one or more test failures of the plurality of tests.

7. The computer-readable storage medium of claim 6, wherein the running of the plurality of tests occurs at predefined intervals.

8. The computer-readable storage medium of claim 6, wherein the subset of the plurality of tests is a subset of a full run, wherein the full run includes passing tests and failing tests.

9. The computer-readable storage medium of claim 6 wherein the running the subset of the plurality of tests is performed when any tests fail.

10. An apparatus for testing a software application used in a database system in a multi-tenant database, the apparatus comprising:

a hardware processor; and

one or more stored sequences of instructions which, when executed by the processor, cause the processor to;

receive multiple successive versions of the software application where each successive version includes a single new change to the software application and changes from one or more prior versions of the software application;

run a plurality of tests on a version of the successive versions of the software application to identify one or more test failures; and

responsive to identifying the one or more test failures,

run a subset of the plurality of tests, where the subset comprises one or more tests that failed, on a first successive version of the software application;

responsive to determining that the first successive version fails the subset of the plurality of tests, determine that the first successive version caused the one or more test failures of the plurality of tests; and

responsive to determining that the first successive version passes the subset of the plurality of tests, run the subset of the plurality of tests on one or more remaining successive versions of the software application until a particular successive version fails the subset of the plurality of tests, where the particular successive version is determined to have caused the one or more test failures of the plurality of tests.

11. The apparatus of claim 10, wherein the running of the plurality of tests occurs at predefined intervals.

12. The apparatus of claim 10, wherein subset of the plurality of tests is a subset of a full run, wherein the full run includes passing tests and failing tests.