Assessing quality of code in an open platform environment

• US 9,110,770 B1
• Filed: 03/04/2014
• Issued: 08/18/2015
• Est. Priority Date: 03/04/2014
• Status: Expired due to Fees

• Current Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Original Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Inventors: Crosley, Jay Austin, Raju, Gurinder, Pistolet, Oleg Oleg, Kinast, Seth Bradon, Austgen, Cameron Dailey, Shekhar, Shashank, Kumar Eerpini, Satish, Lau, Teresa Shuk Kwan
• Primary Examiner(s): COYER, RYAN D
• Application Number:
  US14/197,131
• Time in Patent Office: 532 Days
• Field of Search: 717100-101
• US Class Current: 1/1
• CPC Class Codes: G06F 11/3616 using software metrics
• CPC Class Codes: G06F 8/10 Requirements analysis; Specification techniques
• CPC Class Codes: G06F 8/71 Version control security arrangements therefor G06F21/57; Configuration management
• CPC Class Codes: G06F 8/77 Software metrics
• CPC Class Codes: G06N 5/022 Knowledge engineering; Knowledge acquisition
• CPC Class Codes: G06Q 10/06395 Quality analysis or management

Abstract

Techniques for driving higher quality of code may be provided. For example, code may be received for hosting on a computing resource. A computing service may be implemented to analyze the code prior to the hosting. The analysis may include assessing components of the code against criteria associated with code performance. Based on the analysis, the computing service may determine a quality of the code, provide recommended changes to the code that may improve the quality, and identify a proper computing resource for hosting the code at the quality.

Claims

1. A computer-implemented method, comprising:

generating, by a computer system of an open platform, a development rule for evaluating a development level of code and for recommending a modification to the code based at least in part on the evaluating, the code associated with an item provider and configured to provide a web site hosted at the open platform on behalf of the item provider, the development rule included in an electronic tool provided to a computing device of the item provider;

receiving the code, the development level, and a request for implementing the code from the computing device of the item provider, the development level being based at least in part on the development rule;

determining, by the computer system, a validation rule based at least in part on the development level;

validating, by the computer system, the development level to generate a validation level associated with the code based at least in part on the validation rule;

providing a recommendation associated with the code to the computing device of the item provider based at least in part on the validation level;

identifying, by the computer system, a particular computing resource for hosting the code based at least in part on the validation level, the particular computing resource identified from a plurality of different computing resources of the open platform;

monitoring one or more operational metrics associated with the hosting of the code on the particular computing resource;

generating an operational level based at least in part on the one or more operational metrics; and

in response to determining that the validation level exceeds the operational level, identifying a second computing resource for hosting the code based at least in part on the operational level.

2. The computer-implemented of claim 1, wherein the open platform is a part of an electronic marketplace configured to offer an item of the item provider, wherein the development rule is selected from a plurality of development rules based at least in part on the development level and a selection of a desired development level.

3. The computer-implemented of claim 1, wherein the validation rule and the development rule comprise parameters for evaluating a computational cost of hosting the code, wherein the parameters comprise one or more of:

a semantic check, a syntactic check, a code coverage measure, an application programming interface (API) version, a number of API calls, a code complexity measure, a size of data associated with API calls, a latency measure, a simulated operational metric, or an emulated operational metric.

4. A computer-implemented method, comprising:

receiving, by a development tool, a user selection of a desired grade of code for providing network content associated with a user, the network content to be hosted on an open platform associated with a provider, the code being configured for implementation by a computing system of the open platform;

determining, by the development tool, a rule for evaluating an actual grade of the code based at least in part on a set of parameters for estimating a computational cost for hosting the code on the open platform;

evaluating, by the development tool, the actual grade of the code based at least in part on the rule;

in response to a determination that the actual grade meets or exceeds the desired grade, allowing a submission of the code to the open platform at the desired grade; and

in response to a determination that the desired grade exceeds the actual grade;

allowing a modification of the user selection such that a revised grade is selected, the actual grade meeting or exceeding the revised grade;

estimating the computational cost based at least in part on a subset of the parameters; and

allowing a submission of the code to the open platform at the revised grade based at least in part on the estimated computational cost.

5. The computer-implemented method of claim 4, wherein the development tool is configured to store the rule.

6. The computer-implemented method of claim 4, wherein the rule is determined based at least in part on the desired grade.

7. The computer-implemented method of claim 4, wherein evaluating the actual grade of the code comprises:

associating a weight with one or more particular parameters of the set of parameters;

allocating a score for a parameter of the one or more-particular parameters based at least in part on a corresponding weight and a determination that the code satisfies the parameter; and

combining scores of the one or more particular parameters to generate the actual grade.

8. The computer-implemented method of claim 4, further comprising:

receiving a modification to the code, wherein the modification allows the desired grade to meet or exceed the actual grade; and

allowing a submission of the code updated with the modification to the open platform at the desired grade.

9. The computer-implemented method of claim 4, further comprising providing a user interface for inputting the user selection of the desired grade.

10. A system, comprising:

a memory that stores computer-executable instructions; and

a processor configured to access the memory, wherein the processor is configured to execute the computer-executable instructions to collectively at least;

identify a first code quality associated with code for providing network content, the network content to be hosted on an open platform, the code being configured for implementation by the open platform;

determine a second code quality for the code based at least in part on a checklist for evaluating the code for quality, the checklist comprising parameters and respective weights for estimating a cost of running the code on a computing resource of the open platform, the weights based at least in part on potential contributions of the respective parameters to the cost, wherein determining the second code quality for the code comprises selecting a subset of the parameters and weights based at least in part on the first code quality;

in response to determining that the second code quality meets or exceeds the first code quality, allow a submission of the code to the open platform at the second code quality; and

in response to determining that that the first code quality exceeds the second code quality;

recommend a modification to the code based at least in part on the first code quality;

receive modified code;

determine that the modified code is associated with a third code quality that meets or exceeds the first code quality; and

allow a submission of the modified code to the open platform at the third code quality.

11. The system of claim 10, wherein the recommended modification identifies changes to the code, the changes allowing the code to have a particular code quality that meets or exceeds the first code quality.

12. The system of claim 10, wherein determining that the modified code is associated with a third code quality that meets or exceeds the first code quality comprises:

identifying changes made to the code; and

determining that the changes improve the code quality to meet or exceed the first code quality.

13. The system of claim 10, wherein determining the second code quality for the code further comprises:

generating scores based at least in part on using the parameters to analyze the code;

weighing the scores based at least in part on the respective weights; and

combining the weighted scores to generate the code quality for the code.

14. One or more non-transitory computer-readable storage media storing computer-executable instructions that, when executed by one or more computer systems, configure the one or more computer systems to perform operations comprising:

receiving code for providing network content, the network content to be hosted on an open platform associated with the one or more computer systems, the code being configured for implementation on the open platform;

determining a first quality indicator for the code based at least in part on a criteria for determining a cost associated with implementing the code on the open platform;

recommending a modification to the code based at least in part on the first quality indicator;

in response to receiving a request for implementing the code with the first quality indicator, identifying a first computing resource of the open platform for running the code by at least;

determining, based at least on the first quality indicator, a set of parameters comprising at least one of;

a security level, a computing power, a memory size, or a routing requirement on a network of the open platform; and

determining that a configuration of the first computing resource meets the set of parameters; and

in response to receiving a request for implementing a modified code;

determining a second quality indicator for the modified code based at least in part on the criteria; and

identifying a second computing resource of the open platform for running the modified code based at least in part on the second quality indicator.

15. The one or more non-transitory computer-readable storage media of claim 14, further comprising:

setting a priority for hosting a compute instance of the code based at least in part on the first quality indicator; and

causing the first computing resource to host the compute instance of the code based at least in part on the priority, wherein the first computing resource hosts a plurality of compute instances at a plurality of priorities.

16. The one or more non-transitory computer-readable storage media of claim 14, wherein the code is associated with a user, and further comprising:

analyzing historical quality indicators for received codes associated with the user;

determining that the historical quality indicators meet a quality level; and

providing an incentive to the user based at least in part on the quality level.

17. The one or more non-transitory computer-readable storage media of claim 16, wherein the incentive comprises at least one of a reduced fee for running the code on the open platform, an access to a beta release of a functionality of the open platform, or a reduced flow for deploying the code on the open platform.

18. The one or more non-transitory computer-readable storage media of claim 14, wherein the code is associated with a first user, wherein the open platform is associated with a provider, and further comprising:

analyzing historical quality indicators for received codes associated with the first user and a second user;

providing a recommendation to the provider for scaling the platform based at least in part on the historical quality indicators;

identifying, based at least in part on the historical quality indicators, a documentation issue associated with a document of the provider for developing the code;

providing a recommendation to the first user for a best practice to develop the code based at least in part on the historical quality indicators; and

identifying a list of preferred solution providers based at least in part on the historical quality indicators, the preferred solution providers having developed codes for the first user or the second user.

19. The one or more non-transitory computer-readable storage media of claim 14, wherein the criteria is derived from a checklist for evaluating the code, the checklist being associated with the first quality indicator, and wherein the code is associated with a first user, and further comprising:

analyzing historical quality indicators for received code associated with the first user and a second user; and

updating the checklist based at least in part on the historical quality indicators.