Measurement of remote display performance with image-embedded markers

• US 8,910,228 B2
• Filed: 04/26/2013
• Issued: 12/09/2014
• Est. Priority Date: 11/09/2010
• Status: Active Grant

• Current Assignee: Vmware LLC (Broadcom, Inc.)
• Original Assignee: VMware, Inc. (Broadcom, Inc.)
• Inventors: Agrawal, Banit, Bidarkar, Rishi, Kurkure, Uday, Sivaraman, Hari, Spracklen, Lawrence, Magdon-Ismail, Tariq
• Primary Examiner(s): CHEN, CAI Y
• Application Number:
  US13/872,036
• Publication Number:
  US 20140320673A1
• Time in Patent Office: 592 Days
• Field of Search: None
• US Class Current: 725/116
• CPC Class Codes: H04N 21/25825 involving client display capabilities, e.g. screen resolution of a mobile phone optimising the visualisation of content during browsing in the Internet G06F16/9577; processing of terminal status or physical abilities in wireless networks H04W8/22; authentication in wireless network security H04W12/06
• CPC Class Codes: H04N 21/25833 involving client hardware characteristics, e.g. manufacturer, processing or storage capabilities allocation of resources considering hardware capabilities in multiprogramming arrangements G06F9/5044; allocation of resources considering software capabilities in multiprogramming arrangements G06F9/5055

Abstract

Methods, systems, and computer programs are provided for measuring the performance of a remote display. One method includes an operation for embedding in display frames a respective frame identifier and a respective frame picture. The respective frame picture is placed in a respective region that depends of the value of the respective frame identifier. Additionally, the method includes operations for sending the plurality of display frames to a remote client, and for extracting each received frame picture from the respective region of each received display frame at the remote client based on the received frame identifier. The received frame quality is calculated for each received display frame based on the comparison of the extracted received frame picture with the respective frame picture. Further, the method includes an operation for calculating a remote display quality for the remote client based on the calculated received frame quality for the received display frames.

Claims

1. A method for measuring performance of a computer system, the method comprising:

embedding in a plurality of display frames a respective frame identifier and a respective frame picture, the respective frame picture being placed in a respective region that is based on the respective frame identifier;

sending after the embedding the plurality of display frames to a remote client;

extracting each received frame picture from the respective region of each received display frame at the remote client based on the received frame identifier;

calculating a received frame quality for each received display frame based on a comparison of the extracted received frame picture with the respective frame picture that was embedded; and

calculating a remote display quality for the remote client based on the calculated received frame quality for the received display frames, wherein operations of the method are executed by a processor.

2. The method as recited in claim 1, further including:

sending files with the frame pictures to the remote client before the embedding, the frame pictures in the files being used for the comparison to calculate the received frame quality.

3. The method as recited in claim 1, wherein the embedding further includes:

encoding each frame identifier in pixels of the respective frame picture, the frame identifier being a binary value, wherein each bit of the binary value is encoded in a plurality of pixels.

4. The method as recited in claim 3, wherein each 0 binary value is encoded in a plurality of pixels with a first color value, wherein each 1 binary value is encoded in the plurality of pixels with a second color value.

5. The method as recited in claim 1 further including:

calculating a frame rate quality based on the received frame identifiers.

6. The method as recited in claim 1, wherein the region for placing each frame picture in each display frame is different from a region for placing a previous frame picture in a display frame immediately previous to the each display frame.

7. The method as recited in claim 1, wherein a same image is embedded in a plurality of consecutive display frames in a same region of the display frames.

8. The method as recited in claim 1, further including:

adding a remote display quality icon to a display based on the calculated remote display quality.

9. The method as recited in claim 1, wherein embedding further includes:

determining a first value of the frame identifier for a first display frame;

determining a first region that is associated with the first value of the frame identifier;

determining a first frame picture that is associated with the first value of the frame identifier; and

embedding the first frame picture in the first region of the first display frame.

10. The method as recited in claim 9, wherein extracting each received frame picture further includes:

extracting a second value of the frame identifier from a second received display frame;

determining a second region that is associated with the second value of the frame identifier; and

extracting the received frame picture from the second region in the second received displayed frame.

11. The method as recited in claim 1, wherein calculating the received frame quality further includes:

calculating a peak signal-to-noise ratio (PSNR).

12. A system comprising:

a processor; and

a memory including;

a plurality of known images, each known image being associated with a respective frame identifier;

a remote display program that, when executed by the processor, receives a display image for a remote display; and

a performance measurement program, wherein program instructions from the performance measurement program when executed by the processor cause the processor to;

extract from a plurality of display frames a respective frame identifier and a respective frame picture, the respective frame picture being extracted from a respective region in each display frame that is based on the respective frame identifier;

calculate a received frame quality for each received display frame based on a comparison of the extracted received frame picture with a respective known image that is associated with the frame identifier; and

calculate a remote display quality for the remote display based on the calculated received frame quality for the received display frames.

13. The system as recited in claim 12, wherein each frame identifier is encoded in a plurality of pixels of the respective frame picture, the frame identifier being a binary value, wherein each bit of the binary value is encoded in a plurality of pixels.

14. The system as recited in claim 13, wherein the performance measurement program further eliminates noise from each plurality of pixels before determining the binary value.

15. The system as recited in claim 12, wherein the wherein the performance measurement program further:

examines incoming display frames until a frame identifier is discovered embedded in one of the display frames before starting the extraction of the respective frame identifier and respective frame pictures.

16. A computer program embedded in a non-transitory computer-readable storage medium, when executed by one or more processors, for measuring performance of a computer system, the computer program comprising:

program instructions for embedding in a plurality of display frames a respective frame identifier and a respective frame picture, the respective frame picture being placed in a respective region that is based on the respective frame identifier;

program instructions for sending after the embedding the plurality of display frames to a remote client;

program instructions for extracting each received frame picture from the respective region of each received display frame at the remote client based on the received frame identifier;

program instructions for calculating a received frame quality for each received display frame based on a comparison of the extracted received frame picture with the respective frame picture that was embedded;

program instructions for calculating a frame rate quality based on the received frame identifiers; and

program instructions for calculating a remote display quality for the remote client based on the calculated received frame quality for the received display frames and the frame rate quality.

17. The computer program as recited in claim 16, wherein the embedding is performed on-the-fly on a live video stream before sending the live video stream to the remote client.

18. The computer program as recited in claim 16, further including:

program instructions for detecting a user command to measure display performance before starting the embedding.

19. The computer program as recited in claim 16, further including:

program instructions for assigning system resources based on the calculated remote display quality.

20. The computer program as recited in claim 16, further including:

program instructions for sending files with the frame pictures to the remote client before the embedding, the frame pictures in the files being used for the comparison to calculate the received frame quality.