VERIFICATION OF SHARED DISPLAY INTEGRITY IN A DESKTOP SHARING SYSTEM

• US 20130238684A1
• Filed: 03/07/2012
• Published: 09/12/2013
• Est. Priority Date: 03/07/2012
• Status: Active Grant

• Current Assignee: Salesforce.com, Inc.
• Original Assignee: Salesforce.com, Inc.
• Inventors: Patil, Dipak
• Granted Patent: US 9,348,614 B2
• Time in Patent Office: Days
• US Class Current: 709/203
• CPC Class Codes: G06F 3/1462 with means for detecting differences between the image stored in the host and the images displayed on the remote displays
• CPC Class Codes: G06F 9/452 Remote windowing, e.g. X-Window System, desktop virtualisation protocols for virtual reality H04L67/131
• CPC Class Codes: G09G 2340/04 Changes in size, position or resolution of an image
• CPC Class Codes: G09G 2360/18 Use of a frame buffer in a display terminal, inclusive of the display panel
• CPC Class Codes: G09G 2370/02 Networking aspects
• CPC Class Codes: G09G 5/005 Adapting incoming signals to the display format of the display terminal
• CPC Class Codes: G09G 5/12 Synchronisation between the display unit and other units, e.g. other display units, video-disc players
• CPC Class Codes: G09G 5/227 Resolution modifying circuits, e.g. variable screen formats, resolution change between memory contents and display screen

Abstract

A system and related operating methods that achieve shared screen verification are presented here. One operating method begins by establishing a desktop sharing session between a publisher device and a viewer device, during which the publisher device renders a publisher display and the viewer device renders a viewer display that corresponds to the publisher display. The method continues by capturing first framebuffer data for a frame of the publisher display, capturing second framebuffer data for the corresponding frame of the viewer display. The method processes the first framebuffer data and/or the second framebuffer data to obtain processed framebuffer data having a consistent display resolution. Thereafter, the method compares the processed second framebuffer data to the processed first framebuffer data to obtain a measure of quality.

Claims

1. A method of shared screen verification, comprising:

establishing a desktop sharing session between a publisher device and a viewer device, during which the publisher device renders a publisher display and the viewer device renders a viewer display that corresponds to the publisher display;

capturing first framebuffer data for a frame of the publisher display;

capturing second framebuffer data for a frame of the viewer display, wherein video content of the frame of the viewer display corresponds to video content of the frame of the publisher display;

processing the first framebuffer data, the second framebuffer data, or both the first framebuffer data and the second framebuffer data to obtain processed first framebuffer data and processed second framebuffer data that exhibit a consistent display resolution; and

comparing the processed second framebuffer data to the processed first framebuffer data to obtain a measure of quality.

2. The method of claim 1, further comprising generating an output that indicates the measure of quality in a human-readable format.

3. The method of claim 1, wherein:

the publisher device comprises a video framebuffer; and

capturing the first framebuffer data comprises receiving the first framebuffer data from the video framebuffer.

4. The method of claim 1, wherein:

the viewer device comprises a video framebuffer; and

capturing the second framebuffer data comprises receiving the second framebuffer data from the video framebuffer.

5. The method of claim 1, wherein:

the first framebuffer data is provided by the publisher device;

the second framebuffer data is available at the viewer device;

capturing the first framebuffer data comprises obtaining the first framebuffer data at the viewer device;

capturing the second framebuffer data comprises obtaining the second framebuffer data at the viewer device; and

the processing and the comparing are performed at the viewer device.

6. The method of claim 1, wherein:

the first framebuffer data is provided by the publisher device;

the second framebuffer data is provided by the viewer device;

capturing the first framebuffer data comprises obtaining the first framebuffer data at a server device that is communicatively coupled between the publisher device and the viewer device;

capturing the second framebuffer data comprises obtaining the second framebuffer data at the server device; and

the processing and the comparing are performed at the server device.

7. The method of claim 1, wherein the comparing comprises:

identifying a first N-by-M pixel region of the processed first framebuffer data, wherein N is a nonzero positive integer, and wherein M is a nonzero positive integer;

identifying a second N-by-M pixel region of the processed second framebuffer data, wherein the first N-by-M pixel region and the second N-by-M pixel region correspond to a common display area for the publisher display and the viewer display; and

determining a score that indicates an extent to which the first N-by-M pixel region matches the second N-by-M pixel region, wherein the measure of quality is based on the score.

8. The method of claim 1, wherein the comparing comprises:

identifying a plurality of first low-level pixel regions of the processed first framebuffer data;

identifying a plurality of second low-level pixel regions of the processed second framebuffer data; and

determining a low-level score that indicates an extent to which the first low-level pixel regions match the second low-level pixel regions, wherein the measure of quality is based on the low-level score.

9. The method of claim 8, wherein the comparing further comprises:

identifying a plurality of first high-level pixel regions of the processed first framebuffer data, wherein each of the first high-level pixel regions comprises at least two of the first low-level pixel regions;

identifying a plurality of second high-level pixel regions of the processed second framebuffer data, wherein each of the second high-level pixel regions comprises at least two of the second low-level pixel regions; and

determining a high-level score that indicates an extent to which the first high-level pixel regions match the second high-level pixel regions, wherein the measure of quality is based on the high-level score.

10. A method of shared screen verification, comprising:

establishing a desktop sharing session between a publisher device and a viewer device, during which the publisher device renders a publisher display and the viewer device renders a viewer display that corresponds to the publisher display;

capturing, from a video framebuffer of the publisher device, first framebuffer data for a frame of the publisher display;

capturing, from a video framebuffer of the viewer device, second framebuffer data for a frame of the viewer display, wherein video content of the frame of the viewer display corresponds to video content of the frame of the publisher display;

processing the first framebuffer data, the second framebuffer data, or both the first framebuffer data and the second framebuffer data to obtain processed first framebuffer data and processed second framebuffer data, wherein the processed first framebuffer data and the processed second framebuffer data have a common display resolution; and

comparing the processed second framebuffer data to the processed first framebuffer data to obtain a measure of quality, wherein the comparing comprises;

identifying a first N-by-M pixel region of the processed first framebuffer data, wherein N is a nonzero positive integer, and wherein M is a nonzero positive integer;

identifying a second N-by-M pixel region of the processed second framebuffer data, wherein the first N-by-M pixel region and the second N-by-M pixel region correspond to a common display area for the publisher display and the viewer display; and

determining a score that indicates an extent to which the first N-by-M pixel region matches the second N-by-M pixel region, wherein the measure of quality is based on the score.

11. The method of claim 10, further comprising generating an output that indicates the measure of quality in a human-readable format.

12. The method of claim 10, wherein:

capturing the first framebuffer data comprises obtaining the first framebuffer data at the viewer device;

capturing the second framebuffer data comprises obtaining the second framebuffer data at the viewer device; and

the processing and the comparing are performed at the viewer device.

13. The method of claim 12, wherein:

establishing the desktop sharing session comprises establishing a first data communication connection between the publisher device and the viewer device;

the method further comprises establishing a second data communication connection between the publisher device and the viewer device; and

the viewer device uses the second data communication connection to obtain the first framebuffer data.

14. The method of claim 10, wherein:

capturing the first framebuffer data comprises obtaining the first framebuffer data at a server device that is communicatively coupled between the publisher device and the viewer device;

capturing the second framebuffer data comprises obtaining the second framebuffer data at the server device; and

the processing and the comparing are performed at the server device.

15. The method of claim 10, further comprising:

increasing values of N and M to obtain larger first pixel regions of the processed first framebuffer data, and to obtain larger second pixel regions of the processed second framebuffer data; and

determining supplemental scores that indicate an extent to which the larger first pixel regions match their corresponding larger second pixel regions, wherein the measure of quality is based on the score and the supplemental scores.

16. A computer-implemented device comprising a processor and a memory, wherein the memory comprises computer-executable instructions that, when executed by the processor, cause the device to:

obtain first framebuffer data for a first frame of a display of a publisher device that shares the display with a viewer device during a desktop sharing session;

obtain second framebuffer data for a second frame of a display of the viewer device, wherein video content of the second frame corresponds to video content of the first frame;

process the first framebuffer data, the second framebuffer data, or both the first framebuffer data and the second framebuffer data to obtain processed first framebuffer data and processed second framebuffer data having a common display resolution associated therewith; and

compare a plurality of first pixel regions of the first framebuffer data to a plurality of corresponding second pixel regions of the second framebuffer data to obtain a measure of quality that indicates an extent to which the first pixel regions match the corresponding second pixel regions.

17. The device of claim 16, wherein the computer-executable instructions, when executed by the processor, cause the device to:

identify a plurality of first low-level pixel regions of the processed first framebuffer data;

identify a plurality of second low-level pixel regions of the processed second framebuffer data; and

determine a low-level score that indicates an extent to which the first low-level pixel regions match the second low-level pixel regions, wherein the measure of quality is based on the low-level score.

18. The device of claim 17, wherein the computer-executable instructions, when executed by the processor, cause the device to:

identify a plurality of first high-level pixel regions of the processed first framebuffer data, wherein each of the first high-level pixel regions comprises at least two of the first low-level pixel regions;

identify a plurality of second high-level pixel regions of the processed second framebuffer data, wherein each of the second high-level pixel regions comprises at least two of the second low-level pixel regions; and

determine a high-level score that indicates an extent to which the first high-level pixel regions match the second high-level pixel regions, wherein the measure of quality is based on the high-level score.

19. The device of claim 16, wherein:

the device is a server device that is communicatively coupled between the publisher device and the viewer device;

the server device obtains the first framebuffer data from a video framebuffer of the publisher device; and

the server device obtains the second framebuffer data from a video framebuffer of the viewer device.

20. The device of claim 16, further comprising:

a desktop sharing application executed by the processor to support the desktop sharing session; and

a shared screen verification application executed by the processor, wherein the shared screen verification application is distinct from the desktop sharing application, and wherein the shared screen verification application comprises the computer-executable instructions.