Remote display performance measurement triggered by application display upgrade

• US 9,336,117 B2
• Filed: 04/26/2013
• Issued: 05/10/2016
• Est. Priority Date: 11/09/2010
• Status: Active Grant

• Current Assignee: Vmware LLC (Broadcom, Inc.)
• Original Assignee: VMware, Inc. (Broadcom, Inc.)
• Inventors: Agrawal, Banit, Bidarkar, Rishi, Kurkure, Uday, Magdon-Ismail, Tariq, Sivaraman, Hari, Spracklen, Lawrence
• Primary Examiner(s): Lazaro, David
• Assistant Examiner(s): HENRY, MARIEGEORGES A
• Application Number:
  US13/872,040
• Publication Number:
  US 20140325054A1
• Time in Patent Office: 1,110 Days
• Field of Search: 709/224, 709/246, 725/116
• US Class Current: 1/1
• CPC Class Codes: G06F 11/3409 for performance assessment
• CPC Class Codes: G06F 11/3419 by assessing time
• CPC Class Codes: G06F 11/3495 for systems
• CPC Class Codes: G06F 2201/815 Virtual middleware or OS functionality using virtual machines to implement generic software techniques for error detection or fault masking G06F11/1484
• CPC Class Codes: H04N 21/2402 Monitoring of the downstream path of the transmission network, e.g. bandwidth available traffic monitoring in data switching networks H04L43/00; monitoring data switching networks utilization H04L43/0876
• CPC Class Codes: H04N 21/4424 Monitoring of the internal components or processes of the client device, e.g. CPU or memory load, processing speed, timer, counter or percentage of the hard disk space used error monitoring in general G06F11/30; arrangements for monitoring conditions of receiving stations in broadcast systems H04H60/32; diagnosis, testing or measuring for television receivers H04N17/04
• CPC Class Codes: H04N 21/4431 characterized by the use of Application Program Interface [API] libraries

Abstract

Methods, systems, and computer programs are provided for measuring the performance of display images received on a remote computer display. One method includes an operation for detecting calls from an application to an application programming interface (API), which is provided for rendering images on a display image, each call causing an update of the display image. Further, the method includes an operation for embedding data for measuring performance in display frames of the display image based on the detecting. The embedding results in modified displayed frames with respective data for measuring performance. The modified displayed frames are transmitted to a remote client, which results in received modified display frames having respective received data for measuring the performance. In addition, the method includes an operation for calculating the remote display quality for the given application based on the received modified display frames and the respective received data for measuring performance.

Claims

1. A method for measuring performance of a computer system, the method comprising:

detecting one or more calls from an application executing on a virtual machine to an application programming interface (API) provided for rendering images, each call causing an update of a display image;

in response to the detecting, embedding data for measuring performance in each of a plurality of display frames of the display image, the embedding resulting in a plurality of modified display frames each with respective data for measuring performance;

transmitting the plurality of modified display frames to a remote client, the remote client receiving a plurality of received modified display frames with respective received data for measuring performance; and

calculating a remote display quality for the application at the remote client based on received performance data from the remote client, wherein the received performance data is based on the respective data for measuring performance extracted from the plurality of modified display frames, wherein operations of the method are executed by a processor.

2. The method as recited in claim 1, wherein detecting calls from the application further includes:

modifying program instructions to be executed when the calls to the API are invoked to transfer program control to a performance module when the calls are made by the application.

3. The method as recited in claim 2, wherein embedding data further includes:

adding the data for measuring performance in the plurality of display frames; and

invoking the call to the API after adding the data.

4. The method as recited in claim 3, wherein invoking the call to the API further includes:

executing the program instructions that were modified; and

transferring control to program instructions of the call that logically follow the executed program instructions that were modified.

5. The method as recited in claim 1 further including:

trapping a plurality of functions in the API to measure display performance when any of the plurality of functions is invoked by the application.

6. The method as recited in claim 1, wherein the data for measuring performance includes a frame number encoded in pixels of the display image.

7. The method as recited in claim 6, wherein each bit of the frame number is encoded in an area encompassing a plurality of pixels, each pixel in the area having a same color value, the pixels having a first value to encode a binary 0 and having a second value to encode a binary 1.

8. The method as recited in claim 7, wherein calculating the remote display quality further includes:

calculating a frame rate received at the remote client based on frame numbers encoded in the plurality of received modified display frames.

9. The method as recited in claim 1, wherein an agent module at the remote client examines the display image received at the client to detect the data for measuring performance embedded in the modified displayed frames.

10. The method as recited in claim 1, further including:

receiving a trigger to start measuring remote display performance.

11. The method as recited in claim 10, wherein the trigger is selected from a group consisting of an administrator command, a user selecting an option to measure performance, a network packet received, or invoking a performance measurement program.

12. The method as recited in claim 1, wherein the data for measuring performance includes a respective frame number and a frame picture each encoded in pixels of corresponding display frames of the display image.

13. A system comprising:

a processor; and

a memory including;

a display management program that, when executed by the processor, provides an application programming interface (API) provided for rendering images;

a remote display program that, when executed by the processor, transmits a display image rendered by the API to a remote display; and

a performance measurement program;

wherein the program instructions from the performance measurement program when executed by the processor cause the processor to;

detect one or more calls from an application executing on a virtual machine to the API that cause updates of the display image;

in response to the detection, embed data for measuring performance in each of a plurality of display frames of the display image, the embedding resulting in a plurality of modified display frames each with respective data for measuring performance; and

calculate a remote display quality for the application at a remote client based on received performance data from the remote client, wherein the received performance data is based on the respective data for measuring performance extracted from the plurality of modified display frames.

14. The system as recited in claim 13, wherein the performance measurement program adjusts system resources and network resources in response to the calculated remote display quality.

15. The system as recited in claim 13, wherein the performance measurement program measures remote display quality for a plurality of remote clients to enforce a service level agreement.

16. The system as recited in claim 15, wherein embedding data for measuring performance further includes:

adding the data for measuring performance in the plurality of display frames; and

invoking the call to the API after adding the data.

17. The system as recited in claim 16, wherein invoking the call to the API further includes:

executing the program instructions that were modified; and

transferring control to program instructions of the call that logically follow the executed program instructions that were modified.

18. A computer program embedded in a non-transitory computer-readable storage medium, when executed by one or more processors, for measuring performance of a computer system, the computer program comprising:

program instructions for detecting one or more calls from an application executing on a virtual machine to an application programming interface (API) provided for rendering images, each call causing an update of a display image;

program instructions for embedding data for measuring performance in each of a plurality of display frames of the display image in response to the detecting, the embedding resulting in a plurality of modified display frames each with respective data for measuring performance;

program instructions for transmitting the plurality of modified display frames to a remote client, the remote client receiving a plurality of received modified display frames with respective received data for measuring performance; and

program instructions for calculating a remote display quality for the application at the remote client based on received performance data from the remote client, wherein the received performance data is based on the respective data for measuring performance extracted from the plurality of modified display frames.

19. The computer program as recited in claim 18, further including:

program instructions for adding a performance indicator icon to the display image.

20. The computer program as recited in claim 18, wherein the call includes swapping display data buffers.

21. The computer program as recited in claim 18, wherein detecting calls from the application further includes:

program instructions for modifying program instructions to be executed when the calls to the API are invoked to transfer program control to a performance module when the calls are made by the application.