Method and systems for the scheduled capture of web content from web servers as sets of images

• US 10,158,722 B2
• Filed: 07/31/2015
• Issued: 12/18/2018
• Est. Priority Date: 07/31/2015
• Status: Active Grant

• Current Assignee: Page Vault, Inc.
• Original Assignee: Page Vault, Inc.
• Inventors: Eschbach, Jeffrey T, Price, Todd Walter, Nazaran, Stephen Gregory
• Primary Examiner(s): Hussain, Tauqir
• Application Number:
  US14/815,678
• Publication Number:
  US 20170289267A1
• Time in Patent Office: 1,236 Days
• Field of Search: 709203
• CPC Class Codes: G06F 16/955 using information identifiers, e.g. uniform resource locators [URL]
• CPC Class Codes: G06F 9/4555 Para-virtualisation, i.e. guest operating system has to be modified
• CPC Class Codes: H04L 67/01 Protocols
• CPC Class Codes: H04L 67/02 based on web technology, e.g. hypertext transfer protocol [HTTP]
• CPC Class Codes: H04L 67/142 Managing session states for stateless protocols; Signalling session states; State transitions; Keeping-state mechanisms
• CPC Class Codes: H04L 67/60 Scheduling or organising the servicing of application requests, e.g. requests for application data transmissions using the analysis and optimisation of the required network resources admission control or resource allocation H04L47/70

Abstract

A method and system for the scheduled capturing of web content from a web server includes a capturing server establishing a link with a client device and receiving over the link from the client device a set of indications for web content to capture and schedule information for capturing the web content. The method further includes performing, in accordance with the schedule information, establishing a network connection with the web server hosting a web resource of the indicated web content, receiving the web resource over the network connection from the web server without the web resource being routed through the client device, and capturing the web resource as a set of images of the web resource loaded into a web browser executing on the capturing server.

Claims

1. A method performed by a capturing server for scheduled capturing of web content, the method comprising:

establishing a link with a client device;

receiving over the link from the client device a set of indications for web content to capture and schedule information for capturing the web content, wherein the set of indications for web content to capture includes an indication for a webpage which comprises embedded links to additional webpages not included in the set of indications; and

performing in accordance with the schedule information;

establishing a first network connection with a first web server hosting a first web resource of the indicated web content, wherein the first web resource comprises a webpage;

receiving the first web resource over the first network connection from the first web server without the first web resource being routed through the client device;

capturing the first web resource as a first set of images of the first web resource loaded into a web browser executing on the capturing server, wherein the first set of images comprises multiple screenshots of the webpage; and

each of the multiple screenshots captures a different portion of the webpage; and

capturing each additional webpage as a set of images of the additional webpage loaded into the web browser executing on the capturing server.

2. The method of claim 1 further comprising performing in accordance with the schedule information:

establishing a second network connection with a second web server hosting a second web resource of the indicated web content;

receiving the second web resource over the second network connection from the second web server without the second web resource being routed through the client device; and

capturing the second web resource as a second set of images of the second web resource loaded into the web browser.

3. The method of claim 1, wherein the set of indications for web content to capture comprises indications for first web content to batch capture, the method further comprising batch capturing the first web content as multiple sets of images.

4. The method of claim 1, wherein:

the link with the client device allows the client device to remotely control a web browsing session between the capturing server and the web server; and

the client device uses the web browsing session to select the web content to capture.

5. The method of claim 4, wherein the client device uses the web browsing session to select the web content to capture by indicating web resources to capture as the client device navigates to the web resources.

6. The method of claim 5, wherein the web resources indicated for capture comprises a website.

7. The method of claim 1, wherein:

the set of indications for web content to capture comprises an indication for a website identifiable by a domain name; and

capturing the first web resource comprises capturing multiple webpages of the web site having the domain name.

8. The method of claim 1 further comprising expanding the webpage prior to capturing the webpage.

9. The method of claim 1 further comprising blocking the client device from uploading content to the web browser over the link with the client device.

10. The method of claim 1 further comprising blocking at least one interaction of the client device with the web browser at a moment of capturing the first set of images of the first web resource loaded into the web browser.

11. The method of claim 1, wherein sequential screenshots of the multiple screenshots overlap by an amount of overlap.

12. The method of claim 11 further comprising receiving, over the link from the client device, an indication of a measure of overlap, wherein the measure of overlap specifies the amount of overlap.

13. The method of claim 1 further comprising:

applying a comparative algorithm to a first portion and a second portion of the first web resource indicated for capture as consecutive screenshots;

comparing a second output of the comparative algorithm for the second portion to a first output of the comparative algorithm for the first portion;

capturing the second portion after capturing the first portion in response to the second output not matching the first output.

14. The method of claim 13, wherein applying the comparative algorithm to the first and second portions of the first web resource comprises applying the comparative algorithm only to corresponding parts of the first and second portions.

15. The method of claim 1, wherein the first web resource is dynamic web content, and wherein capturing the first web resource as the first set of images comprises capturing the dynamic web content as a video.

16. The method of claim 1, wherein:

capturing the first web resource includes capturing the client device interacting with the first web resource; and

capturing the first web resource as the first set of images comprises capturing the first web resource as a video.

17. The method of claim 1, wherein the first web resource includes audio content, and wherein capturing the first web resource includes capturing the audio content.

18. A capturing server configured to schedule capture of web content, the capturing server comprising:

a first network interface configured to establish a link with a client device;

a second network interface configured to establish a first network connection with a web server; and

a processing element operatively coupled to the first and second network interfaces, wherein the processing element is configured to;

establish the link with a client device;

receive over the link from the client device a set of indications for web content to capture and schedule information for capturing the web content, wherein the set of indications for web content to capture includes an indication for a webpage which comprises embedded links to additional webpages not included in the set of indications; and

perform in accordance with the schedule information;

establish the first network connection with the web server hosting a web resource of the indicated web content, wherein the web resource comprises a webpage;

receive the web resource over the first network connection from the web server without the web resource being routed through the client device;

capture the web resource as a set of images of the web resource loaded into a web browser executing on the capturing server, wherein the set of images comprises multiple screenshots of the webpage; and

each of the multiple screenshots captures a different portion of the webpage; and

capture each additional webpage as a set of images of the additional webpage loaded into the web browser executing on the capturing server.

19. The capturing server of claim 18 further comprising a third network interface configured to establish a second network connection to a network storage device remotely located from both the capturing server and the client device, wherein the processing element is further configured to:

establish the second network connection between the network storage device and the capturing server; and

send the captured web resource over the second network connection for storing on the network storage device.