Software testing with feedback acquisition

• US 9,396,092 B1
• Filed: 03/26/2014
• Issued: 07/19/2016
• Est. Priority Date: 03/26/2014
• Status: Active Grant

• Current Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Original Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Inventors: Bender, James William, Badescu, Gabriel, Voorheis, Spencer Lyle, Weise, Jedd, Kuo, Calvin Yue-Ren
• Primary Examiner(s): Nahar, Qamrun
• Application Number:
  US14/226,619
• Time in Patent Office: 846 Days
• Field of Search: None
• US Class Current: 1/1
• CPC Class Codes: G06F 11/3664 Environments for testing or debugging software
• CPC Class Codes: G06F 8/00 Arrangements for software engineering testing or debugging G06F11/36; administrative, planning or organisation aspects of software project management G06Q10/06
• CPC Class Codes: G06F 8/38 for implementing user interfaces
• CPC Class Codes: G06F 8/65 Updates security arrangements therefor G06F21/57

Abstract

Methods and systems for software testing with feedback acquisition are disclosed. Modified program code is generated based on programmatic analysis of original program code. The modified program code includes the original program code and a plurality of additional instructions. The additional instructions implement user interface prompts soliciting answers to user feedback questions. The modified program code is sent to one or more client devices for execution. User input responsive to the user interface prompts is then received from the one or more client devices.

Claims

1. A system, comprising:

at least one processor;

a memory coupled to the at least one processor, wherein the memory stores program instructions, wherein the program instructions are executable by the at least one processor to;

receive, from a software developer, original application program code and a plurality of user feedback questions;

generate modified application program code based on programmatic analysis of the original application program code, wherein the modified application program code comprises the original application program code and a plurality of additional user interface instructions, wherein the plurality of additional user interface instructions implement user interface prompts soliciting one or more answers to individual ones of the plurality of user feedback questions, and wherein, in generating the modified application program code based on the programmatic analysis of the original application program code, the program instructions are further executable by the at least one processor to;

determine a plurality of modification hints in the original application program code;

determine individual ones of the user feedback questions matching individual ones of the plurality of modification hints; and

insert the plurality of additional user interface instructions based on locations of the plurality of modification hints;

send the modified application program code to one or more client devices; and

receive, from individual ones of the one or more client devices, user input responsive to individual ones of the user interface prompts.

2. The system as recited in claim 1, wherein the additional user interface instructions are executable to cause a marketplace application to display one or more user interface elements, wherein the user input is received using the marketplace application on the one or more client devices.

3. The system as recited in claim 1, wherein the program instructions are further executable by the at least one processor to:

store the user input in a feedback repository, wherein the feedback repository is accessible to the software developer.

4. A computer-implemented method, comprising:

receiving modified program code, wherein the modified program code is generated based on programmatic analysis of original program code, wherein the modified program code comprises a plurality of additional instructions based on a plurality of user feedback questions, and wherein, in the programmatic analysis of the original program code, the plurality of additional instructions are inserted based on locations of a plurality of modification hints in the original program code;

initiating execution of the modified program code; and

executing individual ones of the additional instructions during the execution of the modified program code, comprising soliciting user input from a user based on individual ones of the user feedback questions.

5. The method as recited in claim 4, wherein the plurality of modification hints comprise one or more annotations in the original program code.

6. The method as recited in claim 4, wherein the plurality of modification hints comprise one or more variable declarations in the original program code.

7. The method as recited in claim 4, wherein the plurality of modification hints comprise one or more instances of application programming interface (API) usage in the original program code.

8. The method as recited in claim 4, wherein soliciting the user input from the user based on the individual ones of the user feedback questions comprises:

causing a marketplace application to display one or more user interface elements, wherein one or more answers to the individual ones of the user feedback questions are received from the user via the one or more user interface elements.

9. The method as recited in claim 4, further comprising:

receiving, from the user, one or more answers to the individual ones of the user feedback questions; and

sending the one or more answers to a feedback repository, wherein the feedback repository is accessible to a developer of the original program code.

10. The method as recited in claim 4, wherein the user input is solicited from the user at a plurality of points in time during the execution of the modified program code.

11. A computer-readable storage medium storing program instructions computer-executable to perform:

generating modified program code based on programmatic analysis of original application program code, wherein the modified application program code comprises a plurality of additional instructions, wherein the plurality of additional instructions implement user interface prompts for a plurality of user feedback questions at a plurality of locations in the modified application program code, and wherein, in generating the modified application program code based on the programmatic analysis of the original program code, the program instructions are computer-executable to perform;

determining a plurality of modification hints in the original application program code;

determining individual ones of the user feedback questions matching individual ones of the plurality of modification hints; and

inserting the plurality of additional instructions based on the plurality of modification hints;

sending the modified application program code to a client device, wherein individual ones of the additional instructions are executed on the client device, comprising causing a display of the user interface prompts for individual ones of the user feedback questions; and

receiving, from the client device, one or more answers to the individual ones of the user feedback questions.

12. The computer-readable storage medium as recited in claim 11, wherein the plurality of modification hints comprise one or more comments in the original application program code.

13. The computer-readable storage medium as recited in claim 11, wherein the plurality of modification hints comprise one or more method names in the original application program code.

14. The computer-readable storage medium as recited in claim 11, wherein the original application program code comprises bytecode, and wherein, in generating the modified application program code based on the programmatic analysis of the original program code, the program instructions are computer-executable to perform:

using bytecode injection to add the plurality of additional instructions to the modified application program code.

15. The computer-readable storage medium as recited in claim 11, wherein the user interface prompts are displayed on the client device by a marketplace application.

16. The computer-readable storage medium as recited in claim 11, wherein the program instructions are computer-executable to perform:

storing the one or more answers in a feedback repository, wherein the feedback repository is accessible to a developer of the original application program code.

17. The computer-readable storage medium as recited in claim 11, wherein the program instructions are computer-executable to perform:

selecting the client device from a plurality of client devices based on one or more hardware characteristics of the selected client device.