×

Automated forensic artifact reconstruction and replay of captured and recorded display interface streams

  • US 9,996,915 B2
  • Filed: 08/04/2016
  • Issued: 06/12/2018
  • Est. Priority Date: 08/04/2016
  • Status: Active Grant
First Claim
Patent Images

1. A method for implementing user interface diagnostics, comprising:

  • sending, with an embedded processor of a graphical user interface (“

    GUI”

    ) system, a first stream of data to a display device;

    receiving, with the display device, the first stream of data, wherein the first stream of data comprises display data and formatting data, the formatting data comprising information indicating a number of pixels per row and a number of rows, and the display data comprising pixel data comprising color depth for each pixel;

    formatting, with the display device, the first stream of data to produce one or more first display screen images, based at least in part on the formatting data and on the display data;

    displaying, with the display device, the one or more first display screen images, one screen image at a time;

    capturing, with a data stream recording system, the first stream of data as the first stream of data is being sent from the embedded processor to the display device, without disturbing the data being sent to the display device;

    recording, with the data stream recording system, the captured first stream of data;

    retrieving, with a computing system, the captured and recorded first stream of data;

    determining, with the computing system, a display protocol associated with the captured and recorded first stream of data, based at least in part on the formatting data of the captured and recorded first stream of data;

    parsing, with the computing system, the pixel data from the display data of the captured and recorded first stream of data;

    formatting, with the computing system, the parsed pixel data to produce one or more second display screen images, based at least in part on the determined display protocol and on the parsed pixel data;

    determining, with the computing system, whether one or more successive display screen images of the one or more second display screen images is pixel-to-pixel identical to a display screen image of the one or more second display screen images;

    based on a determination that one or more successive display screen images of the one or more second display screen images is pixel-to-pixel identical to one display screen image of the one or more second display screen images, compressing, with the computing system, the formatted pixel data to produce one or more third display screen images, by deleting the one or more successive display screen images that is determined to be pixel-to-pixel identical to the one display screen image;

    reconstructing, with the computing system, the one or more first display screen images, based on the one or more third display screen images;

    storing, with the computing system and on a data store, the reconstructed one or more first display screen images;

    comparing, with the computing system, each pixel of each screen image of the reconstructed one or more first display screen images with each pixel of a model screen image of each corresponding unique screen image of the one or more first display screen images that is displayed on the display device;

    based on a determination that at least one screen image of the reconstructed one or more first display screen images is not pixel-to-pixel identical to the model screen image of a corresponding unique screen image of the one or more first display screen images that is displayed on the display device, logging, with the computing system and in an error log, the at least one screen image as being pixel imperfect and indicating in the error log which pixels of the at least one screen image do not match with corresponding pixels in the model screen image of a corresponding unique screen image of the one or more first display screen images; and

    based on a determination that each of the screen images of the reconstructed one or more first display images matches pixel-to-pixel with the model screen image of each corresponding unique screen image of the one or more first display screen images that is displayed on the display device, sending, with the computing system, a notification to a user that the one or more first display screen images match corresponding model screen images in a pixel perfect manner.

View all claims
  • 4 Assignments
Timeline View
Assignment View
    ×
    ×