Systems and methods for automated on-device performance testing of mobile applications

• US 9,277,432 B2
• Filed: 10/09/2014
• Issued: 03/01/2016
• Est. Priority Date: 10/09/2013
• Status: Active Grant

• Current Assignee: Salesforce.com, Inc.
• Original Assignee: Salesforce.com, Inc.
• Inventors: Mathur, Ashraya Raj, Xia, Jingshu, Vaughan, Tom, Katari, Venkatesh, Tiwari, Prashant, Prasad, Sudatta Gautham Sharada
• Primary Examiner(s): Vo, Don N
• Application Number:
  US14/511,054
• Publication Number:
  US 20150099493A1
• Time in Patent Office: 509 Days
• Field of Search: 455/414.1, 717/124, 717/127
• US Class Current: 1/1
• CPC Class Codes: G06F 11/3419 by assessing time
• CPC Class Codes: G06F 11/3495 for systems
• CPC Class Codes: G06F 21/6227 where protection concerns the structure of data, e.g. records, types, queries
• CPC Class Codes: G06F 2201/865 Monitoring of software
• CPC Class Codes: G06F 2201/875 Monitoring of systems including the internet
• CPC Class Codes: H04L 43/0858 One way delays
• CPC Class Codes: H04L 67/02 based on web technology, e.g. hypertext transfer protocol [HTTP]
• CPC Class Codes: H04W 24/08 Testing, supervising or monitoring using real traffic
• CPC Class Codes: H04W 4/60 Subscription-based services using application servers or record carriers, e.g. SIM application toolkits

Abstract

Methods and systems for measuring the performance of a mobile application and its associated user page flows include a communication link configured to connect a plurality of mobile client devices to an application server running the mobile application. The system includes a first module configured to implement a plurality of mobile network protocols; a second module configured to implement a plurality of mobile carrier protocols; a third module configured to implement a plurality of mobile operating systems; and a processing module configured to download the application and user pages from the application server and measure performance on each of the plurality of mobile client devices using different combinations of the plurality of mobile network protocols, the plurality of mobile carrier protocols, and the plurality of mobile operating systems.

Claims

1. An automated test system for measuring the performance of a mobile application including a user page, comprising:

a communication link configured to connect a plurality of mobile client devices to an application server running the mobile application;

a first module configured to implement a plurality of mobile network protocols;

a second module configured to implement a plurality of mobile carrier protocols;

a third module configured to implement a plurality of mobile operating systems; and

a processing module configured to download the user page from the application server to each of the plurality of mobile client devices using different combinations of the plurality of mobile network protocols, the plurality of mobile carrier protocols, and the plurality of mobile operating systems.

2. The system of claim 1, wherein the plurality of mobile network protocols comprises at least two of WiFi, LTE, WiMAX, 3G, 4G, Edge, UMTS, EV-DO, iBurst, and HSPA.

3. The system of claim 1, wherein the plurality of mobile carrier protocols comprises at least two of Verizon™

, Leap Wireless™

, Cricket™

, Virgin Mobile™

, AT&

T™

, Sprint™

, US Cellular™

, Net10™

, Clearwire™

, and T-Mobile™

.

4. The system of claim 1, wherein the plurality of mobile operating systems comprises at least two of iOS™

, Android™

, Windows™ and

Blackberry™

.

5. The system of claim 1, further comprising a fourth module configured to implement a plurality of mobile browsers, and wherein the processing system is further configured to download the user page from the application server to each of the plurality of mobile client devices using different combinations of the plurality of mobile network protocols, the plurality of mobile carrier protocols, the plurality of mobile operating systems, and the plurality of mobile browsers.

6. The system of claim 5, wherein the plurality of mobile browsers comprises at least two of Android™

, Chrome™

, Internet Explorer Mobile™

, Firefox™

, Opera™

, Skyfire™

, Safari™

, Bolt™

, Teashark™

, Blazer™

, Minimo™

, Thunderhawk™

, and Amazon™

.

7. The system of claim 5, wherein the processing module comprises:

a first processor module configured to correlate the plurality of mobile network protocols, the plurality of mobile carrier protocols, the plurality of mobile operating systems, and the plurality of mobile browsers; and

a second processor module configured to schedule the downloading of the user page from the application server to each of the plurality of mobile client devices using the correlated network protocols carrier protocols, mobile operating systems, and mobile browsers.

8. The system of claim 7, further comprising a user interface configured to facilitate user interaction with and configuration of the first and second processor modules.

9. The system of claim 1, further comprising a database and a display module, and wherein the processing module is further configured to record performance data to the database and display the recorded data on the display module.

10. The system of claim 9, wherein the performance data comprises, for each of the plurality of mobile client devices:

a first time period between a download request by the mobile client device and receipt of the request by the application server;

a second time period representing processing of the request by the application server;

a third time period between transmitting the user page by the application server and receipt of the user page by the mobile client device; and

a fourth time period between receipt of the user page by the mobile client device and display of the user page on the mobile client device.

11. The system of claim 9, wherein the processing module is configured to record performance data using a Selenium-based script.

12. A method of measuring the performance of a mobile application including a user page, the method comprising:

connecting a plurality of mobile client devices to an application server running the mobile application;

configuring a first module to implement a plurality of mobile network protocols;

configuring a second module to implement a plurality of mobile carrier protocols;

configuring a third module to implement a plurality of mobile operating systems; and

downloading the user page from the application server to each of the plurality of mobile client devices using different combinations of the plurality of mobile network protocols, the plurality of mobile carrier protocols, and the plurality of mobile operating systems.

13. The method of claim 12, wherein the plurality of mobile network protocols comprises at least two of WiFi, LTE, WiMAX, 3G, 4G, Edge, UMTS, EV-DO, iBurst, and HSPA.

14. The method of claim 12, wherein:

the plurality of mobile network protocols comprises at least two of WiFi, LTE, WiMAX, 3G, 4G, Edge, UMTS, EV-DO, iBurst, and HSPA;

the plurality of mobile carrier protocols comprises at least two of Verizon™

, Leap Wireless™

, Cricket™

, Virgin Mobile™

, AT&

T™

, Sprint™

, US Cellular™

, Net10™

, Clearwire™

, and T-Mobile™

; and

the plurality of mobile operating systems comprises at least two of iOS™

, Android™

, Safari™

, Windows™

, Blackberry™

, Opera™

, and Chrome™

.

15. The method of claim 12, further comprising:

configuring a fourth module to implement a plurality of mobile browsers; and

downloading the user page from the application server to each of the plurality of mobile client devices using different combinations of the plurality of mobile network protocols, the plurality of mobile carrier protocols, the plurality of mobile operating systems, and the plurality of mobile browsers.

16. The method of claim 12, further comprising:

recording performance data to a database; and

displaying the recorded data on a display module.

17. The method of claim 16, wherein the performance data comprises, for each of the plurality of mobile client devices:

a first time period between a download request by the mobile client device and receipt of the request by the application server;

a second time period representing processing of the request by the application server;

a third time period between transmitting the user page by the application server and receipt of the user page by the mobile client device; and

a fourth time period and fine-grained breakdown of client-side performance between receipt of the user page by the mobile client device and display of the user page on the mobile client device.

18. The method of claim 12, further comprising:

correlating the plurality of mobile network protocols, the plurality of mobile carrier protocols, the plurality of mobile operating systems, and the plurality of mobile browsers; and

scheduling the downloading of the user page from the application server to each of the plurality of mobile client devices using the correlated network protocols carrier protocols, mobile operating systems, and mobile browsers.

19. A computer application embodied in a non-transitory medium for operation by a computer processor associated with an on-demand, multi-tenant computing environment, the application configured to implement the steps of:

connecting a plurality of mobile client devices to an application server running the mobile application;

configuring a first module to implement a plurality of mobile network protocols;

configuring a second module to implement a plurality of mobile carrier protocols;

configuring a third module to implement a plurality of mobile operating systems; and

downloading the user page from the application server to each of the plurality of mobile client devices using different combinations of the plurality of mobile network protocols, the plurality of mobile carrier protocols, and the plurality of mobile operating systems.

20. The computer application of claim 19 further configured to implement the steps of:

configuring a fourth module to implement a plurality of mobile browsers;

downloading the user page from the application server to each of the plurality of mobile client devices using different combinations of the plurality of mobile network protocols, the plurality of mobile carrier protocols, the plurality of mobile operating systems, and the plurality of mobile browsers;

recording latency data to a database; and

displaying the recorded data on a display module.