Verification of shared display integrity in a desktop sharing system

• US 10,289,375 B2
• Filed: 04/20/2016
• Issued: 05/14/2019
• Est. Priority Date: 03/07/2012
• Status: Active Grant

• Current Assignee: Salesforce.com, Inc.
• Original Assignee: Salesforce.com, Inc.
• Inventors: Patil, Dipak
• Primary Examiner(s): Abedin, Normin
• Assistant Examiner(s): Bukhari, Sibte H
• Application Number:
  US15/133,792
• Publication Number:
  US 20160231979A1
• Time in Patent Office: 1,119 Days
• Field of Search: 709203
• CPC Class Codes: G06F 3/1462 with means for detecting differences between the image stored in the host and the images displayed on the remote displays
• CPC Class Codes: G06F 9/452 Remote windowing, e.g. X-Window System, desktop virtualisation protocols for virtual reality H04L67/131
• CPC Class Codes: G09G 2340/04 Changes in size, position or resolution of an image
• CPC Class Codes: G09G 2360/18 Use of a frame buffer in a display terminal, inclusive of the display panel
• CPC Class Codes: G09G 2370/02 Networking aspects
• CPC Class Codes: G09G 5/005 Adapting incoming signals to the display format of the display terminal
• CPC Class Codes: G09G 5/12 Synchronisation between the display unit and other units, e.g. other display units, video-disc players
• CPC Class Codes: G09G 5/227 Resolution modifying circuits, e.g. variable screen formats, resolution change between memory contents and display screen

Abstract

A system and related operating methods that achieve shared screen verification are presented here. One operating method begins by establishing a desktop sharing session between a publisher device and a viewer device, during which the publisher device renders a publisher display and the viewer device renders a viewer display that corresponds to the publisher display. The method continues by capturing first framebuffer data for a frame of the publisher display, capturing second framebuffer data for the corresponding frame of the viewer display. The method processes the first framebuffer data and/or the second framebuffer data to obtain processed framebuffer data having a consistent display resolution. Thereafter, the method compares the processed second framebuffer data to the processed first framebuffer data to obtain a measure of quality.

Claims

1. A method of shared screen verification, comprising:

capturing, from a first framebuffer at a publisher device, first framebuffer data for a frame of a publisher display rendered at the publisher device during a desktop sharing session between the publisher device and a viewer device;

separately capturing, from a second framebuffer at the viewer device, second framebuffer data for a frame of a viewer display rendered at the viewer device, wherein the frame of the viewer display corresponds to the frame of the publisher display;

adjusting resolution of the first framebuffer data, the second framebuffer data, or both the first framebuffer data and the second framebuffer data to generate processed first framebuffer data and processed second framebuffer data that exhibit a consistent display resolution;

identifying a common display area of the frame for the publisher display and the viewer display by identifying a first N-by-M pixel region of the processed first framebuffer data and identifying a second N-by-M pixel region of the processed second framebuffer data, wherein the first N-by-M pixel region and the second N-by-M pixel region correspond to the common display area for the publisher display and the viewer display, and wherein N is a nonzero positive integer, and wherein M is a nonzero positive integer; and

performing a matching analysis of the processed first framebuffer data for the first pixel region and the processed second framebuffer data for the second pixel region to determine a score that indicates an extent to which the first N-by-M pixel region of the frame matches the second N-by-M pixel region of the frame.

2. The method of claim 1, wherein the first framebuffer data corresponds to pixel-based information used to render the frame at the publisher display, and wherein the second framebuffer data corresponds to pixel-based information used to render the frame at the viewer display.

3. The method of claim 1, wherein video content of the frame of the viewer display corresponds to video content of the frame of the publisher display.

4. The method of claim 1, wherein the adjusting step is performed when resolution of the first framebuffer data differs from resolution of the second framebuffer data, and wherein the adjusting step comprises:

adjusting resolution of the first framebuffer data, the second framebuffer data, or both the first framebuffer data and the second framebuffer data to generate the processed first framebuffer data and the processed second framebuffer data that exhibit a consistent display resolution so that the processed first framebuffer data and the processed second framebuffer data are capable of being compared to each other.

5. The method of claim 1, wherein the score indicates shared screen display quality of the first and second pixel regions of the frame rendered on the publisher display and the viewer display, respectively.

6. The method of claim 1, wherein a desktop sharing application is executed by a processor to support the desktop sharing session, and further comprising:

generating an output that indicates the shared screen display quality of the defined pixel regions of the frame rendered on the viewer display and the publisher display.

7. The method of claim 1, wherein:

the first framebuffer of the publisher device comprises a first video framebuffer; and

capturing the first framebuffer data comprises;

receiving the first framebuffer data from the first video framebuffer.

8. The method of claim 1, wherein:

the second framebuffer of the viewer device comprises a second video framebuffer; and

separately capturing the second framebuffer data comprises;

receiving the second framebuffer data from the second video framebuffer.

9. The method of claim 1, wherein:

the first framebuffer data is provided by the publisher device;

the second framebuffer data is available at the viewer device;

capturing the first framebuffer data comprises;

obtaining the first framebuffer data at the viewer device;

separately capturing the second framebuffer data comprises;

obtaining the second framebuffer data at the viewer device; and

the adjusting resolution and determining are performed at the viewer device.

10. The method of claim 1, wherein:

the first framebuffer data is provided by the publisher device;

the second framebuffer data is provided by the viewer device;

capturing the first framebuffer data comprises;

obtaining the first framebuffer data at the server device that is communicatively coupled between the publisher device and the viewer device;

separately capturing the second framebuffer data comprises;

obtaining the second framebuffer data at the server device; and

the adjusting resolution and determining are performed at the server device.

11. A computer-implemented device comprising a processor and a memory, wherein the memory comprises computer-executable instructions that, when executed by the processor, cause the device to:

capture, from a first framebuffer at a publisher device, first framebuffer data for a frame of a publisher display rendered at the publisher device during a desktop sharing session between the publisher device and a viewer device;

separately capture, from a second framebuffer at the viewer device, second framebuffer data for a frame of a viewer display rendered at the viewer device, wherein the frame of the viewer display corresponds to the frame of the publisher display;

adjust resolution of the first framebuffer data, the second framebuffer data, or both the first framebuffer data and the second framebuffer data to generate processed first framebuffer data and processed second framebuffer data that exhibit a consistent display resolution;

identify a common display area of the frame for the publisher display and the viewer display by identifying a first N-by-M pixel region of the processed first framebuffer data and identifying a second N-by-M pixel region of the processed second framebuffer data, wherein the first N-by-M pixel region and the second N-by-M pixel region correspond to the common display area for the publisher display and the viewer display, and wherein N is a nonzero positive integer, and wherein M is a nonzero positive integer; and

perform a matching analysis of the processed first framebuffer data for the first pixel region and the processed second framebuffer data for the second pixel region to determine a score that indicates an extent to which the first N-by-M pixel region of the frame matches the second N-by-M pixel region of the frame.

12. The computer-implemented device of claim 11, wherein the first framebuffer data corresponds to pixel-based information used to render the frame at the publisher display, and wherein the second framebuffer data corresponds to pixel-based information used to render the frame at the viewer display.

13. The computer-implemented device of claim 11, wherein video content of the frame of the viewer display corresponds to video content of the frame of the publisher display.

14. The computer-implemented device of claim 11, wherein the memory further comprises computer-executable instructions that, when executed by the processor, cause the device to:

adjust resolution of the first framebuffer data, the second framebuffer data, or both the first framebuffer data and the second framebuffer data when resolution of the first framebuffer data differs from resolution of the second framebuffer data to generate the processed first framebuffer data and the processed second framebuffer data that exhibit a consistent display resolution so that the processed first framebuffer data and the processed second framebuffer data are capable of being compared to each other.

15. The computer-implemented device of claim 11, wherein the score indicates shared screen display quality of the first and second pixel regions of the frame rendered on the publisher display and the viewer display, respectively.

16. The computer-implemented device of claim 11, wherein the memory further comprises computer-executable instructions that, when executed by the processor, cause the device to:

generate an output that indicates the shared screen display quality of the defined pixel regions of the frame rendered on the viewer display and the publisher display.

17. The computer-implemented device of claim 11, wherein the first framebuffer of the publisher device comprises a first video framebuffer and the second framebuffer of the viewer device comprises a second video framebuffer;

and wherein the memory further comprises computer-executable instructions that, when executed by the processor, cause the device to;

receive the first framebuffer data from the first video framebuffer, and receive the second framebuffer data from the second video framebuffer.

18. A system for shared screen verification, comprising:

a publisher device comprising a first framebuffer and a publisher display rendered at the publisher device;

a viewer device comprising a second frame buffer and a viewer display rendered at the viewer device; and

a processor configured to;

capture, from the first framebuffer, first framebuffer data for a frame of the publisher display during a desktop sharing session between the publisher device and the viewer device;

separately capture, from the second framebuffer at the viewer device, second framebuffer data for a frame of the viewer display, wherein the frame of the viewer display corresponds to the frame of the publisher display;

adjust resolution of the first framebuffer data, the second framebuffer data, or both the first framebuffer data and the second framebuffer data to generate processed first framebuffer data and processed second framebuffer data that exhibit a consistent display resolution;

identify a common display area of the frame for the publisher display and the viewer display by identifying a first N-by-M pixel region of the processed first framebuffer data and identifying a second N-by-M pixel region of the processed second framebuffer data, wherein the first N-by-M pixel region and the second N-by-M pixel region correspond to the common display area for the publisher display and the viewer display, and wherein N is a nonzero positive integer, and wherein M is a nonzero positive integer; and

perform a matching analysis of the processed first framebuffer data for the first pixel region and the processed second framebuffer data for the second pixel region to determine a score that indicates an extent to which the first N-by-M pixel region of the frame matches the second N-by-M pixel region of the frame.