Method and system for software application testing recommendations

• US 9,727,448 B1
• Filed: 04/22/2015
• Issued: 08/08/2017
• Est. Priority Date: 04/22/2015
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Seibert, Jr., Jeffrey, Chang, Wayne
• Primary Examiner(s): Guyton, Philip
• Application Number:
  US14/693,438
• Time in Patent Office: 839 Days
• CPC Class Codes: G06F 11/3676 for coverage analysis
• CPC Class Codes: G06F 11/368 for test version control, e.g. updating test cases to a new software version

Abstract

A method for application testing recommendation includes deploying a software application to first testers having first system configurations, and receiving testing coverage data describing (i) a first subset of features of the software application being tested by the first testers and (ii) information describing the first system configurations. The method further includes deploying the software application to a marketplace, and receiving market coverage data describing (i) a second subset of features being used by market users and (ii) second system configurations of the market users. The method further includes comparing the market coverage data and the testing coverage data to identify a coverage discrepancy, selecting second testers to test the software application based on the coverage discrepancy, and deploying the software application to the second testers.

Claims

1. A method for application testing recommendation, comprising:

deploying a software application to a first plurality of testers having a first plurality of system configurations;

receiving testing coverage data describing (i) a first subset of a plurality of features of the software application being tested by the first plurality of testers and (ii) information describing the first plurality of system configurations;

deploying the software application to a marketplace;

receiving, from an analytics software development kit (SDK) embedded in the software application, market coverage data describing (i) a second subset of the plurality of features of the software application being used by a plurality of market users and (ii) a second plurality of system configurations of the plurality of market users;

comparing the market coverage data and the testing coverage data to identify a coverage discrepancy;

selecting a second plurality of testers to test the software application based on the coverage discrepancy; and

deploying the software application to the second plurality of testers.

2. The method of claim 1, wherein selecting the second plurality of testers comprises obtaining a unique identifier of each tester in the second plurality of testers.

3. The method of claim 1, wherein comparing the market coverage data and the testing coverage data comprises determining an overlap between the first plurality of system configurations and the second plurality of system configurations.

4. The method of claim 1, wherein comparing the market coverage data and the testing coverage data comprises:

for each system configuration in the second plurality of system configurations;

determining a first proportion of the system configuration used by the plurality of market users;

determining a second proportion of the system configuration used by the plurality of testers; and

determining whether the first proportion is within a threshold difference to the second proportion.

5. The method of claim 1, wherein comparing the market coverage data and the testing coverage data comprises determining whether the first subset of the plurality of features comprises the second subset of the plurality of features.

6. The method of claim 1, wherein comparing the market coverage data and the testing coverage data comprises:

for each feature in the second subset of the plurality of features;

determining a first proportion of the feature used by the plurality of market users to a total usage of the second subset of the plurality of features;

determining a second proportion of the feature used by the plurality of testers to a total usage of the first subset of the plurality of features; and

determining whether the first proportion is within a threshold difference to the second proportion.

7. The method of claim 1, further comprising:

generating a dashboard for displaying a comparison of the market coverage data and the testing coverage data; and

displaying the dashboard.

8. The method of claim 7, further comprising:

generating a rating based on comparing the market coverage data and the testing coverage data; and

incorporating the rating into the dashboard.

9. A system for software application testing recommendation, comprising:

a computer processsor;

a data repository for storing information describing a plurality of system configurations and testing coverage data; and

a testing service configured to execute on the computer processor for;

deploying a software application to a first plurality of testers having a first plurality of system configurations,

an analytics service configured to execute on the computer processor for;

receiving testing coverage data describing (i) a first subset of a plurality of features of the software application being tested by the first plurality of testers and (ii) information describing the first plurality of system configurations, and

receiving, from an analytics software development kit (SDK) embedded in the software application, market coverage data describing (i) a second subset of the plurality of features of the software application being used by a plurality of market users and (ii) a second plurality of system configurations of the plurality of market users, and

a recommendation module configured to execute on the computer processor for;

comparing the market coverage data and the testing coverage data to identify a coverage discrepancy, and

selecting a second plurality of testers to test the software application based on the coverage discrepancy.

10. The system of claim 9, further comprising:

a developer user interface configured to display a dashboard and the second plurality of testers.

11. The system of claim 9, wherein comparing the market coverage data and the testing coverage data comprises determining whether the first plurality of system configurations comprises the second plurality of system configurations.

12. The system of claim 9, wherein comparing the market coverage data and the testing coverage data comprises:

for each system configuration in the second plurality of system configurations;

determining a first proportion of the system configuration used by the plurality of market users;

determining a second proportion of the system configuration used by the plurality of testers; and

determining whether the first proportion is within a threshold difference to the second proportion.

13. The system of claim 9, wherein the testing service is further for:

receiving defect information from a testing software development kit (SDK) embedded in the software application deployed to the plurality of testers, and

adding the defect information to the testing coverage data.

14. The system of claim 9, wherein the testing coverage data received by the analytics service is received from the analytics SDK embedded in the software application.

15. A non-transitory computer readable medium for software application testing recommendation comprising computer readable program code, the computer readable program code for:

deploying a software application to a first plurality of testers having a first plurality of system configurations;

receiving testing coverage data describing (i) a first subset of a plurality of features of the software application being tested by the first plurality of testers and (ii) information describing the first plurality of system configurations;

deploying the software application to a marketplace;

receiving, from an analytics software development kit (SDK) embedded in the software application, market coverage data describing (i) a second subset of the plurality of features of the software application being used by a plurality of market users and (ii) a second plurality of system configurations of the plurality of market users;

comparing the market coverage data and the testing coverage data to identify a coverage discrepancy;

selecting a second plurality of testers to test the software application based on the coverage discrepancy; and

deploying the software application to the second plurality of testers.

16. The non-transitory computer readable medium of claim 15, wherein comparing the market coverage data and the testing coverage data comprises determining whether the first subset of the plurality of features comprises the second subset of the plurality of features.

17. The non-transitory computer readable medium of claim 15, wherein comparing the market coverage data and the testing coverage data comprises:

for each feature in the second subset of the plurality of features;

determining a first proportion of the feature used by the plurality of market users to a total usage of the second subset of the plurality of features;

determining a second proportion of the feature used by the plurality of testers to a total usage of the first subset of the plurality of features; and

determining whether the first proportion is within a threshold difference to the second proportion.

18. The non-transitory computer readable medium of claim 15, further comprising computer readable program code for:

generating a dashboard describing comparing the market coverage data and the testing coverage data; and

displaying the dashboard.

19. The non-transitory computer readable medium of claim 18, further comprising computer readable program code for:

generating a rating based on comparing the market coverage data and the testing coverage data; and

incorporating the rating into the dashboard.