Remote display performance measurement triggered by application display upgrade

• US 9,578,373 B2
• Filed: 05/09/2016
• Issued: 02/21/2017
• Est. Priority Date: 11/09/2010
• Status: Active Grant

• Current Assignee: Vmware LLC (Broadcom, Inc.)
• Original Assignee: VMware, Inc. (Broadcom, Inc.)
• Inventors: Agrawal, Banit, Bidarkar, Rishi, Kurkure, Uday, Magdon-Ismail, Tariq, Sivaraman, Hari, Spracklen, Lawrence Andrew
• Primary Examiner(s): Lazaro, David
• Assistant Examiner(s): HENRY, MARIEGEORGES A
• Application Number:
  US15/150,119
• Publication Number:
  US 20160255400A1
• Time in Patent Office: 288 Days
• Field of Search: 709/224, 709/246, 725/116
• US Class Current: 1/1
• CPC Class Codes: G06F 11/3409 for performance assessment
• CPC Class Codes: G06F 11/3419 by assessing time
• CPC Class Codes: G06F 11/3495 for systems
• CPC Class Codes: G06F 2201/815 Virtual middleware or OS functionality using virtual machines to implement generic software techniques for error detection or fault masking G06F11/1484
• CPC Class Codes: H04N 21/2402 Monitoring of the downstream path of the transmission network, e.g. bandwidth available traffic monitoring in data switching networks H04L43/00; monitoring data switching networks utilization H04L43/0876
• CPC Class Codes: H04N 21/4424 Monitoring of the internal components or processes of the client device, e.g. CPU or memory load, processing speed, timer, counter or percentage of the hard disk space used error monitoring in general G06F11/30; arrangements for monitoring conditions of receiving stations in broadcast systems H04H60/32; diagnosis, testing or measuring for television receivers H04N17/04
• CPC Class Codes: H04N 21/4431 characterized by the use of Application Program Interface [API] libraries

Abstract

Methods, systems, and computer programs are provided for measuring the performance of display images received on a remote computer display. One method includes an operation for detecting calls from an application to an application programming interface (API), which is provided for rendering images on a display image, each call causing an update of the display image. Further, the method includes an operation for embedding data for measuring performance in display frames of the display image based on the detecting. The embedding results in modified displayed frames with respective data for measuring performance. The modified displayed frames are transmitted to a remote client, which results in received modified display frames having respective received data for measuring the performance. In addition, the method includes an operation for calculating the remote display quality for the given application based on the received modified display frames and the respective received data for measuring performance.

Claims

1. A method comprising:

receiving at a client device a plurality of display frames for an application executing on a virtual machine of a host device and displayed on a virtual desktop of the client device;

detecting performance data in one or more frames of the received display frames, where the one or more frames have been modified by the host device to embed data for measuring performance; and

calculating a remote display quality for the application at the client device based on the detected performance data, wherein the detected performance data is based on the respective data for measuring performance extracted from the one or more display frames, wherein the respective data extracted from each of the one or more display frames includes a frame identifier and a frame picture.

2. The method of claim 1, wherein the data for measuring performance further includes a frame number encoded in pixels of the display image.

3. The method as recited in claim 2, wherein each bit of the frame number is encoded in an area encompassing a plurality of pixels, each pixel in the area having a same color value, the pixels having a first value to encode a binary 0 and having a second value to encode a binary 1.

4. The method as recited in claim 3, wherein calculating the remote display quality further includes:

calculating a frame rate received at the client device based on frame numbers encoded in two or more display frames.

5. The method of claim 1, wherein the frame identifier is associated with a particular location in the display image, wherein the location identifies a location associated with the corresponding frame picture inserted for that display frame.

6. The method of claim 1, wherein the frame identifier is used to determine an identity of an associated frame picture based on an image identity table that associated frame numbers with image identities.

7. The method of claim 1, wherein calculating the remote display quality includes performing a peak signal to noise ratio calculation for the display image and a corresponding reference image.

8. The method of claim 1, wherein the one or more frames have been modified by the host device in response to detecting one or more calls from the application, each call causing an update of a particular display image.

9. A system comprising:

a client device including one or more computers having one or more processors and one or more memories, the client device configured to perform operations comprising;

receiving a plurality of display frames for an application executing on a virtual machine of a host device and displayed on a virtual desktop of the client device;

detecting performance data in one or more frames of the received display frames, where the one or more frames have been modified by the host device to embed data for measuring performance; and

calculating a remote display quality for the application at the client device based on the detected performance data, wherein the detected performance data is based on the respective data for measuring performance extracted from the one or more display frames, wherein the respective data extracted from each of the one or more display frames includes a frame identifier and a frame picture.

10. A method comprising:

detecting one or more calls from an application executing on a virtual machine each call causing an update of a display image;

in response to the detecting, embedding data for measuring performance in each of one or more display frames of the display image, wherein a modified display frame is generated for each frame with embedded data;

transmitting display frame including one or more modified display frames to a remote client device; and

calculating a remote display quality for the application at the remote client based on received performance data from the remote client, wherein the received performance data is based on the respective data for measuring performance extracted from the one or more modified display frames, wherein the respective data for each modified display frame includes a reference image.

11. The method of claim 10, wherein detecting calls from the application further includes:

modifying program instructions to be executed when the calls are invoked to transfer program control to a performance module when the calls are made by the application.

12. The method as recited in claim 11, wherein embedding data further includes:

adding the data for measuring performance in the one or more display frames; and

invoking the call after adding the data.

13. The method of claim 12, wherein invoking the call further includes:

executing the program instructions that were modified; and

transferring control to program instructions of the call that logically follow the executed program instructions that were modified.

14. The method of claim 10, further including:

trapping a plurality of functions in an application programming interface, provided for rendering images in response to the calls, to measure display performance when any of the plurality of functions is invoked by the application.

15. The method of claim 10, wherein the data for measuring performance further includes a frame number encoded in pixels of the display image.

16. The method of claim 15, wherein each bit of the frame number is encoded in an area encompassing a plurality of pixels, each pixel in the area having a same color value, the pixels having a first value to encode a binary 0 and having a second value to encode a binary 1.

17. The method of claim 16, wherein calculating the remote display quality further includes:

calculating a frame rate received at the client device based on frame numbers encoded in the plurality of modified display frames.

18. The method of claim 10, wherein the data for measuring performance includes a respective frame number encoded in pixels of corresponding modified display frames of the display image.

19. The method of claim 10, further including:

receiving a trigger to start measuring remote display performance.

20. The method of claim 19, wherein the trigger is selected from a group consisting of an administrator command, a user selecting an option to measure performance, a network packet received, or an invocation of a performance measurement program.