Generating application models based on discovery based machine learning

• US 9,053,435 B2
• Filed: 09/17/2013
• Issued: 06/09/2015
• Est. Priority Date: 09/17/2013
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Xamarin Incorporated (Microsoft Corporation)
• Inventors: Krukow, Karl, Larsen, Jonas Maturana, Siegel, David, Friedman, Nathaniel Dourif, Smith, Jason Adam, Depizzol, Vinicius Scopel
• Primary Examiner(s): Chaki, Kakali
• Assistant Examiner(s): Tran, Mai T
• Application Number:
  US14/029,670
• Publication Number:
  US 20150081598A1
• Time in Patent Office: 630 Days
• Field of Search: 706/12, 717/125
• US Class Current: 1/1
• CPC Class Codes: G06F 11/3664 Environments for testing or debugging software
• CPC Class Codes: G06F 8/35 model driven
• CPC Class Codes: G06N 20/00 Machine learning
• CPC Class Codes: G06N 5/04 Inference or reasoning models

Abstract

Embodiments are directed towards generating application models based on discovery based machine learning. A mobile application may be uploaded to a computer that may be part of a testing platform. A reference mobile computer may be selected and the mobile application maybe installed onto the reference mobile computer. Also, the testing platform may generate an initial application model based on the mobile application. The current active window of the mobile application may be determined and the application model may be updated accordingly. Screenshots may be generated that correspond to each current active window of the mobile application. Also, each user-interface control in the active window may be activated. The results of activating each control may be observed and added to the model. If the activation causes navigation, another active window may be determined. The application model may be used for testing other mobile computers.

Claims

1. A method for testing mobile applications over a network using a computer that is operative to perform actions, comprising:

determining a reference mobile computer based on at least one characteristic of a provided mobile application;

installing the mobile application on a physical instance having the same hardware and software as the reference mobile computer;

generating an application model based on the mobile application and the physical instance of the reference mobile computer, by employing one or more machine learning techniques to perform further actions on the physical instance of the reference mobile computer, including;

determining each active window of the mobile application executing on the physical instance of the reference mobile computer;

generating at least one screenshot on the physical instance of the reference mobile computer that corresponds to each active window, wherein the at least one screenshot is stored by the computer;

determining each user-interface (UI) control for the mobile application that is included in each active window based on at least iterating over each UI control that is in each active window, wherein each determined UI control is classified as to its type and class in each active window and associated with at least one UI control profile;

separately activating each UI control in each active window executing on the physical instance of the reference mobile computer based on the at least one UI control profile, wherein at least one action is selected from the at least one UI control profile based on at least one of the determined type or determined class for the activated UI control;

generating at least one screenshot on the physical instance of the reference mobile computer that corresponds to each result for separately activating each UI control for the mobile application executing on the physical instance of the reference mobile computer;

adding to the application model information that includes at least one of an active window, a screenshot, or a result for separately activating a UI control for the mobile application executing on the physical instance of the reference mobile computer; and

automatically generating a test script for executing the mobile application on at least one other mobile computer based on the application model, wherein the at least one other mobile computer is determined to be eligible based on one or more of the at least one characteristic of the provided mobile application and one or more of criteria for the physical instance of the reference mobile computer.

2. The method of claim 1, further comprising, classifying at least one active window based on one or more heuristic tests, wherein the classification is employed to determine activation of at least one UI control in the at least one active window.

3. The method of claim 1, wherein separately activating each UI control, further comprises, determining another active window if activation results in navigation away from a current active window that includes a currently activated UI control.

4. The method of claim 1, wherein separately activating each UI control, further comprises, adding information to the application model that corresponds to activation of each UI control.

5. The method of claim 1, wherein automatically generating the test script, further comprises corresponding each active window to at least one portion of the test script for the mobile application executing on at least the one other mobile computer.

6. The method of claim 1, further comprising, installing the test script on each other mobile computer, wherein the test script is executed when the mobile application executes on each other mobile computer.

7. The method of claim 1, further comprising:

highlighting a portion of the application model as a corresponding portion of the test script is stepped through its execution; and

generating an interactive display of the highlighted portion of the application model and the test script for a user.

8. The method of claim 1, further comprising, determining the physical instance of the reference mobile computer based on criteria including one or more of a screen size, a screen resolution, an operating system, an input method, a processor speed, a processor type, a memory capacity, a browser, a manufacturer, a storage media capacity, or a market share for sales in at least one country.

9. A system for testing mobile applications, comprising:

a network computer, including;

a transceiver for communicating over the network;

a memory for storing at least instructions;

a processor device that is operative to execute instructions that enable actions, including;

determining a reference mobile computer based on at least one characteristic of a provided mobile application;

installing the mobile application on a physical instance having the same hardware and software as the reference mobile computer;

generating an application model based on the mobile application and the physical instance of the reference mobile computer, by employing one or more machine learning techniques to perform further actions on the physical instance of the reference mobile computer, including;

determining each active window of the mobile application executing on the physical instance of the reference mobile computer;

generating at least one screenshot on the physical instance of the reference mobile computer that corresponds to each active window, wherein the at least one screenshot is stored by the computer;

determining each user-interface (UI) control for the mobile application that is included in each active window based on at least iterating over each UI control that is in each active window, wherein each determined UI control is classified as to its type and class in each active window and associated with at least one UI control profile;

separately activating each UI control in each active window executing on the physical instance of the reference mobile computer based on the at least one UI control profile, wherein at least one action is selected from the at least one UI control profile based on at least one of the determined type or determined class for the activated UI control;

generating at least one screenshot on the physical instance of the reference mobile computer that corresponds to each result for separately activating each UI control for the mobile application executing on the physical instance of the reference mobile computer;

adding to the application model information that includes at least one of an active window, a screenshot, or a result for separately activating a UI control for the mobile application executing on the physical instance of the reference mobile computer; and

automatically generating a test script for executing the mobile application on at least one other mobile computer based on the application model, wherein the at least one other mobile computer is determined to be eligible based on one or more of the at least one characteristic of the provided mobile application and one or more of criteria for the physical instance of the reference mobile computer; and

a client network computer, including;

a transceiver for communicating over the network;

a memory for storing at least instructions;

a processor device that is operative to execute instructions that enable actions, including;

providing the mobile application to the network computer.

10. The system of claim 9, further comprising, classifying at least one active window based on one or more heuristic tests, wherein the classification is employed to determine activation of at least one UI control in the at least one active window.

11. The system of claim 9, wherein separately activating each UI control, further comprises, determining another active window if activation results in navigation away from a current active window that includes a currently activated UI control.

12. The system of claim 9, wherein separately activating each UI control, further comprises, adding information to the application model that corresponds to activation of each UI control.

13. The system of claim 9, wherein automatically generating the test script, further comprises corresponding each active window to at least one portion of the test script for the mobile application executing on at least the one other mobile computer.

14. The system of claim 9, further comprising, installing the test script on each other mobile computer, wherein the test script is executed when the mobile application executes on each other mobile computer.

15. The system of claim 9, further comprising:

highlighting a portion of the application model as a corresponding portion of the test script is stepped through its execution; and

generating an interactive display of the highlighted portion of the application model and the test script for a user.

16. The system of claim 9, further comprising, determining the physical instance of the reference mobile computer based on criteria including one or more of a screen size, a screen resolution, an operating system, an input method, a processor speed, a processor type, a memory capacity, a browser, a manufacturer, a storage media capacity, or a market share for sales in at least one country.

17. A network computer for testing mobile applications, comprising:

a transceiver for communicating over the network;

a memory for storing at least instructions;

a processor device that is operative to execute instructions that enable actions, including;

determining a reference mobile computer based on at least one characteristic of a provided mobile application;

installing the mobile application on a physical instance having the same hardware and software as the reference mobile computer;

generating an application model based on the mobile application and the physical instance of the reference mobile computer, by employing one or more machine learning techniques to perform further actions on the physical instance of the reference mobile computer, including;

determining each active window of the mobile application executing on the physical instance of the reference mobile computer;

generating at least one screenshot on the physical instance of the reference mobile computer that corresponds to each active window, wherein the at least one screenshot is stored by the computer;

determining each user-inter ice (UI) control for the mobile application that is included in each active window based on at least iterating over each UI control that is in each active window, wherein each determined UI control is classified as to its type and class in each active window and associated with at least one UI control profile;

separately activating each UI control in each active window executing on the physical instance of the reference mobile computer based on the at least one UI control profile, wherein al least one action is selected from the at least one UI control profile based on at least one of the determined type or determined class for the activated UI control;

generating at least one screenshot on the reference mobile computer that corresponds to each result for separately activating each UI control for the mobile application executing on the physical instance of the reference mobile computer;

adding to the application model information that includes at least one of an active window, a screenshot, or a result for separately activating a UI control for the mobile application executing on the physical instance of the reference mobile computer; and

automatically generating a test script for executing the mobile application on at least one other mobile computer based on the application model, wherein the at least one other mobile computer is determined to be eligible based on one or more of the at least one characteristic of the provided mobile application and one or more of criteria for the physical instance of the reference mobile computer.

18. The network computer of claim 17, further comprising, classifying at least one active window based on one or more heuristic tests, wherein the classification is employed to determine activation of at least one UI control in the at least one active window.

19. The network computer of claim 17, wherein separately activating each UI control, further comprises, determining another active window if activation results in navigation away from a current active window that includes a currently activated UI control.

20. The network computer of claim 17, wherein separately activating each UI control, further comprises, adding information to the application model that corresponds to activation of each UI control.

21. The network computer of claim 17, wherein automatically generating the test script, further comprises corresponding each active window to at least one portion of the test script for the mobile application executing on at least the one other mobile computer.

22. The network computer of claim 17, further comprising, installing the test script on each other mobile computer, wherein the test script is executed when the mobile application executes on each other mobile computer.

23. The network computer of claim 17, further comprising, determining the physical instance of the reference mobile computer based on criteria including one or more of a screen size, a screen resolution, an operating system, an input method, a processor speed, a processor type, a memory capacity, a browser, a manufacturer, a storage media capacity, or a market share for sales in at least one country.

24. A processor readable non-transitory storage media that includes instructions for testing mobile applications, wherein a computer that executes at least a portion of the instructions is enabled to perform actions, comprising:

determining a reference mobile computer based on at least one characteristic of a provided mobile application;

installing the mobile application on a physical instance having the same hardware and software as the reference mobile computer;

generating an application model based on the mobile application and the physical instance of the reference mobile computer, by employing one or more machine learning techniques to perform further actions on the physical instance of the reference mobile computer, including;

determining each active window of the mobile application executing on the physical instance of the reference mobile computer;

generating at least one screenshot on the physical instance of the reference mobile computer that corresponds to each active window, wherein the at least one screenshot is stored by the computer;

determining each user-interface (UI) control for the mobile application that is included in each active window based on at least iterating over each UI control that is in each active window, wherein each determined UI control is classified as to its type and class in each active window and associated with at least one UI control profile;

separately activating each UI control in each active window executing on the physical instance of the reference mobile computer based on the at least one UI control profile, wherein at least one action is selected from the at least one UI control profile based on at least one of the determined type or determined class for the activated UI control;

generating at least one screenshot on the physical instance of the reference mobile computer that corresponds to each result for separately activating each UI control for the mobile application executing on the reference mobile computer;

adding to the application model information that includes at least one of an active window, a screenshot, or a result for separately activating a UI control for the mobile application executing on the physical instance of the reference mobile computer; and

automatically generating a test script for executing the mobile application on at least one other mobile computer based on the application model, wherein the at least one other mobile computer is determined to be eligible based on one or more of the at least one characteristic of the provided mobile application and one or more of criteria for the physical instance of the reference mobile computer.

25. The media of claim 24, further comprising, classifying at least one active window based on one or more heuristic tests, wherein the classification is employed to determine activation of at least one UI control in the at least one active window.

26. The media of claim 24, wherein separately activating each UI control, further comprises, determining another active window if activation results in navigation away from a current active window that includes a currently activated UI control.

27. The media of claim 24, wherein separately activating each UI control, further comprises, adding information to the application model that corresponds to activation of each UI control.

28. The media of claim 24, wherein automatically generating the test script, further comprises corresponding each active window to at least one portion of the test script for the mobile application executing on at least the one other mobile computer.

29. The media of claim 24, further comprising, installing the test script on each other mobile computer, wherein the test script is executed when the mobile application executes on each other mobile computer.

30. The media of claim 24, further comprising:

highlighting a portion of the application model as a corresponding portion of the test script is stepped through its execution; and

generating an interactive display of the highlighted portion of the application model and the test script for a user.