Methods and systems for computing code coverage using grouped/filtered source classes during testing of an application

• US 10,114,737 B2
• Filed: 04/28/2016
• Issued: 10/30/2018
• Est. Priority Date: 09/14/2015
• Status: Active Grant

• Current Assignee: Salesforce.com, Inc.
• Original Assignee: Salesforce.com, Inc.
• Inventors: Ramakrishna, Adarsh, Glaser, Scott, Gandham, Vamshidhar Rao, Vengareddy, Ashwin Simha Reddy, Morales, Jonathan Rico
• Primary Examiner(s): Miller, Viva
• Application Number:
  US15/140,771
• Publication Number:
  US 20170075794A1
• Time in Patent Office: 915 Days
• CPC Class Codes: G06F 11/3676 for coverage analysis
• CPC Class Codes: G06F 11/3692 for test results analysis

Abstract

Computer implemented methods and systems are provided for computing code coverage in a system that includes a user system and a cloud-based computing platform. The user system includes a processing system, memory and an input system that receives input parameters specified by a user of the user system. The memory can store a source class filter module executable by the processing system. Upon being executed by the processing system, the source class filter module can, based on one or more of the input parameters, group and filter source class identifiers to generate a unique source class identifier array of filtered source class identifiers that correspond to a particular subset of source classes that targeted code coverage metrics are to be computed for during code coverage computations when testing an application. A code coverage computation unit can then compute the targeted code coverage metrics for the particular subset of source classes corresponding to the filtered source class identifiers of the unique source class identifier array.

Claims

1. A system, comprising:

a user system, comprising;

an input system configured to receive input parameters specified by a user of the user system, wherein the input parameters comprise;

one or more regular expressions specified by the user of the user system;

a processing system; and

memory configured to store;

a source class filter module executable by the processing system, wherein the source class filter module, upon being executed by the processing system;

is configured to;

group and filter source class identifiers, based on one or more of the input parameters, to generate a unique source class identifier array of filtered source class identifiers that correspond to a particular subset of source classes that targeted code coverage metrics are to be computed for during code coverage computations when testing an application, wherein each regular expression is a pattern that is used to perform a pattern match to identify a particular group of one or more source class identifiers associated with that regular expression that each correspond to a source class name of a source class, and wherein the source class filter module comprises;

a first source class fetcher comprising;

a regular expression processor configured to receive the regular expressions input by the user of the user system; and

a query builder configured to;

generate a query associated with each regular expression; and

a cloud-based computing platform, communicatively coupled to the user system, and comprising;

an interface that interfaces with the source class filter module, wherein the quay builder is configured to;

send each query associated with each regular expression to the interface, wherein the interface is configured to receive each query, contact a data store of the cloud-based computing platform and find matching source class names for each regular expression, and return a source class identifier for each of the matching source class names to the query builder, and wherein the query builder is further configured to receive the source class identifiers for each of the matching source class names, and group the source class identifiers corresponding to each of the regular expressions into a first source class identifier array of the unique source class identifier array; and

a code coverage computation unit that is configured to compute the targeted code coverage metrics for the particular subset of source classes corresponding to the filtered source class identifiers of the unique source class identifier array, wherein the particular subset of source classes correspond to a particular subset of source code of the application.

2. The system according to claim 1, wherein the user system further comprises:

a validation module configured to;

evaluate each of the input parameters to determine whether each input parameter is valid, generate an error message when the validation module determines that any of the input parameters are not valid, and communicate the error message to a user interface of the user system.

3. The system according to claim 1, wherein the input parameters comprise:

a manifest file specified by the user of the user system, and wherein the source class filter module further comprises;

a second source class fetcher comprising;

a manifest file processor configured to;

retrieve the manifest file, wherein the manifest file comprises a plurality of source class names that can be used to fetch a particular group of source class identifiers from the data store of the cloud-based computing platform; and

a manifest file reader and query builder configured to;

read the source class names from the manifest file, send the source class names from the manifest file to the interface, receive the source class identifiers corresponding to each of the source class names from the manifest file from the interface, and group the source class identifiers into a second source class identifier array of the unique source class identifier array.

4. The system according to claim 3, wherein the manifest file has a name, and wherein the plurality of source class names in the manifest file comprises:

a statically defined set of source class names, where each of the source class names is associated with a source class and has a corresponding source class identifier that identifies that source class.

5. The system according to claim 4 wherein the application is a multi-tenant application,

wherein the user selects the manifest file to specify the statically defined set of source class names that are of interest to a sub-tenant of the multi-tenant application during code coverage computations, and

wherein the source classes associated with the statically defined set of source class names collectively correspond to particular source code of the multi-tenant application that is of interest to the sub-tenant of multi-tenant application.

6. The system according to claim 1, wherein the application is a multi-tenant application, and wherein the user selects the one or more regular expressions to specify the particular group of one or more source class names corresponding to source classes that are of interest to a sub-tenant of the multi-tenant application during code coverage computations, and

wherein the source classes associated with the particular group of one or more source class names collectively correspond to particular source code of the multi-tenant application that is of interest to the sub-tenant of multi-tenant application.

7. The system according to claim 1, wherein the source class filter module further comprises:

a unique class ID filter configured to;

receive the second source class identifier array and the second source class identifier array; and

filter the source class identifiers from the second source class identifier array and the source class identifiers from the second source class identifier array to generate the unique source class identifier array of filtered source class identifiers by;

combining the source class identifiers from the first source class identifier array and the source class identifiers from the second source class identifier array into a combined set of source class identifiers;

determining duplicate source class identifiers that are in the combined set;

determining invalid source class identifiers that are in the combined set; and

eliminating the duplicate source class identifiers and the invalid source class identifiers from the combined set to generate the unique source class identifier array of filtered source class identifiers that correspond to the particular subset of source classes that the targeted code coverage metrics are to be computed for during code coverage computations.

8. The system according to claim 7, wherein the unique source class identifier array of filtered source class identifiers correspond to:

a particular subset of all possible source classes that correspond to the particular subset of source code of the application that is of interest to the user and that the targeted code coverage metrics are to be computed for during code coverage computations.

9. The system according to claim 8, wherein the application is a multi-tenant application, wherein the user is a sub-tenant of the multi-tenant application, and wherein the unique source class identifier array of filtered source class identifiers correspond to:

a subset of all possible source classes that are of interest to the sub-tenant during the code coverage computations, and

wherein the subset of all possible source classes corresponds to a particular subset of source code of the multi-tenant application that is of interest to the sub-tenant during the code coverage computations.

10. The system according to claim 6, wherein the memory is further configured to store:

a tooling API invoker and an OAuth generator executable by the processing system, wherein the tooling API invoker, upon being executed by the processing system, is configured to;

receive the unique source class identifier array of the filtered source class identifiers from the source class filter module;

communicate a request to an OAuth generator;

receive a response from the OAuth generator, wherein the response comprises;

an token that is used to establish a session with a tooling API; and

communicate the unique source class identifier array of the filtered source class identifiers to the tooling API, wherein the tooling API communicates the filtered source class identifiers to the code coverage computation unit; and

a code coverage report generator executable by the processing system, wherein the code coverage report generator, upon being executed by the processing system, is configured to;

receive the targeted code coverage metrics for the filtered set of source class identifiers from the code coverage computation unit; and

generate a code coverage report comprising;

the targeted code coverage metrics for the particular subset of source classes corresponding to the filtered set of source class identifiers, wherein the code coverage report is displayed at a user interface of the user system.

11. The system according to claim 10, wherein the code coverage computation unit is further configured to compute organization wide code coverage metrics that indicate code coverage for an organization, and wherein the targeted code coverage metrics are one or more subsets of the organization wide code coverage metrics.

12. The system according to claim 10, wherein the input parameters comprise:

a code coverage threshold specified by the user of the user system, wherein the memory is further configured to store;

a quality metrics verification module executable by the processing system, wherein the quality metrics verification module, upon being executed by the processing system, is configured to;

determine if any tests that were to be executed have failed;

generate an error message when any test is determined to have failed, and communicate the error message to a user interface of the user system;

when none of the tests that were to be executed have failed, determine based on the targeted code coverage metrics whether code coverage is greater than or equal to the code coverage threshold; and

when the code coverage is determined to be less than the code coverage threshold, generate an error message, and communicate the error message to the user interface of the user system.

13. A method for generating targeted code coverage metrics in a cloud-based computing system comprising a user system and a cloud-based computing platform communicatively coupled to the user system, the method comprising:

receiving input parameters specified by a user from an input system of the user system, wherein the input parameters comprise;

one or more regular expressions specified by the user of the user system;

at a processing system of the user system;

grouping and filtering source class identifiers, based on one or more of the input parameters, to generate a unique source class identifier array of filtered source class identifiers that correspond to a particular subset of source classes that targeted code coverage metrics are to be computed for during code coverage computations when testing an application, wherein each regular expression is a pattern that is used to perform a pattern match to identify a particular group of one or more source class identifiers associated with that regular expression that each correspond to a source class name of a source class;

retrieving, via a regular expression processor, the regular expressions;

generating, at a query builder, a query associated with each regular expression;

at an interface of the cloud-based computing platform that interfaces with the user system;

receiving each query associated with each regular expression from the query builder;

contacting a data store of the cloud-based computing platform and finding matching source class names for each regular expression;

returning a source class identifier for each of the matching source class names to the query builder;

receiving, at the query builder, the source class identifiers for each of the matching source class names, and grouping the source class identifiers corresponding to each of the regular expressions into a first source class identifier array of the unique source class identifier array; and

computing, at a code coverage computation unit of the cloud-based computing platform, the targeted code coverage metrics for the particular subset of source classes corresponding to the filtered source class identifiers of the unique source class identifier array, wherein the particular subset of source classes correspond to a particular subset of source code of the application.

14. The method according to claim 13, wherein the input parameters further comprise:

a manifest file specified by the user of the user system, and wherein the method further comprises;

retrieving, via a manifest file processor, the manifest file, wherein the manifest file comprises a plurality of source class names, wherein each of the source class names is associated with a source class and has a corresponding source class identifier that identifies that source class; and

at a manifest file reader and query builder;

reading the source class names from the manifest file;

sending the source class names read from the manifest file to the interface;

receiving, from the interface, source class identifiers corresponding to each of the source class names from the manifest file; and

grouping the source class identifiers into a second source class identifier array.

15. The method according to claim 13, wherein the method further comprises:

filtering, at a unique class ID filter, the source class identifiers from the second source class identifier array and the source class identifiers from the second source class identifier array to generate the unique source class identifier array of filtered source class identifiers, wherein the unique source class identifier array of filtered source class identifiers includes a subset of source classes that have been defined based on one or more of the input parameters,

wherein the unique source class identifier array of filtered source class identifiers correspond to a particular subset of source code of the application that is of interest to the user and that the targeted code coverage metrics are to be computed for during code coverage computations.

16. The method according to claim 15, wherein filtering comprises:

combining the source class identifiers from the first source class identifier array and the source class identifiers from the second source class identifier array into a combined set of source class identifiers;

determining duplicate source class identifiers that are in the combined set;

determining invalid source class identifiers that are in the combined set; and

eliminating the duplicate source class identifiers and the invalid source class identifiers from the combined set to generate the unique source class identifier array of filtered source class identifiers that correspond to the particular subset of source classes that the targeted code coverage metrics are to be computed for during code coverage computations.

17. The method according to claim 16, wherein the application is a multi-tenant application, wherein the user selects one or more of the manifest files and the regular expressions to generate the unique source class identifier array of filtered source class identifiers that correspond to:

a subset of all possible source classes that are of interest to a sub-tenant of the multi-tenant application during the code coverage computations, and

wherein the subset of all possible source classes corresponds to a particular subset of source code of the multi-tenant application that is of interest to the sub-tenant.

18. The method according to claim 13, wherein the method further comprises:

at a tooling API invoker;

receiving the unique source class identifier array of the filtered source class identifiers;

communicating a request to an OAuth generator;

receiving a response from the OAuth generator, wherein the response comprises;

a token that is used to establish a session with a tooling API; and

communicating the unique source class identifier array of the filtered source class identifiers to the tooling API, wherein the tooling API communicates the filtered source class identifiers to the code coverage computation unit.

19. The method according to claim 13, wherein the input parameters comprise:

a code coverage threshold specified by the user of the user system, and wherein the method further comprises;

at the code coverage computation unit;

computing the targeted code coverage metrics for the particular subset of source classes based on the filtered source class identifiers received via the tooling API, and computing organization wide code coverage metrics that indicate code coverage for an organization, wherein the targeted code coverage metrics are one or more subsets of the organization wide code coverage metrics;

at a code coverage report generator;

receiving the targeted code coverage metrics for the filtered set of source class identifiers from the code coverage computation unit; and

generating a code coverage report comprising;

the targeted code coverage metrics for the particular subset of source classes corresponding to the filtered set of source class identifiers, wherein the code coverage report is displayed at a user interface of the user system; and

at a quality metrics verification module;

determining if any tests that were to be executed have failed;

generating an error message when any test is determined to have failed, and communicating the error message to a user interface of the user system;

when none of the tests that were to be executed have failed, determining based on the targeted code coverage metrics whether code coverage is greater than or equal to the code coverage threshold; and

when the code coverage is determined to be less than the code coverage threshold, generating an error message and communicating the error message to the user interface of the user system.

20. A computer program product, comprising:

a non-transitory computer readable medium having a computer readable program code embodied therein that is readable by a processing circuit and storing instructions for execution by the processing circuit to perform a method, the method comprising;

receiving input parameters specified by a user from an input system of a user system, wherein the input parameters comprise;

one or more regular expressions specified by the user of the user system;

grouping and filtering source class identifiers, based on the input parameters, to generate a unique source class identifier array of filtered source class identifiers that correspond to a particular subset of all possible source classes that targeted code coverage metrics are computed for during code coverage computations when testing an application, wherein each regular expression is a pattern that is used to perform a pattern match to identify a particular group of one or more source class identifiers associated with that regular expression that each correspond to a source class name of a source class;

retrieving, via a regular expression processor, the regular expressions;

generating, at a query builder, a query associated with each regular expression;

at an interface of the cloud-based computing platform that interfaces with the user system;

receiving each query associated with each regular expression from the query builder;

contacting a data store of the cloud-based computing platform and finding matching source class names for each regular expression;

returning a source class identifier for each of the matching source class names to the query builder;

receiving, at the query builder, the source class identifiers for each of the matching source class names, and grouping the source class identifiers corresponding to each of the regular expressions into a first source class identifier array of the unique source class identifier array; and

computing, at a code coverage computation unit of the cloud-based computing platform, the targeted code coverage metrics for the particular subset of source classes corresponding to the filtered source class identifiers of the unique source class identifier array, wherein the particular subset of source classes correspond to a particular subset of source code of the application.