Methods and apparatus for dynamically generating web pages
First Claim
1. A method of accessing a plurality of web pages served by a web server, the method comprising:
- sending a first request to the web server for a first web page of the plurality of web pages;
receiving, from the web server, the requested first web page in response to the first request;
locating, in the first web page, a link to a second web page of the plurality of web pages;
determining whether the link includes a pattern of characters that matches a predetermined pattern of characters;
when it is determined that the link includes the pattern of characters, sending a plurality of second request to the web server for the second web page, wherein each of the plurality of second requests includes first information that emulates at least one user characteristic, wherein each of the plurality of second requests provides a different value for the first information that emulates the at least one user characteristic, and wherein each of the plurality of second requests, when processed by the web server, causes the web server to dynamically generate a different version of the second web page that includes content based, at least in part, on the value of the first information that emulates the at least one user characteristic, the web server dynamically generating each different version of the second web page using second information retrieved from a database;
after sending the plurality of second requests to the web server for the second web page, monitoring the database to determine whether the second information stored in the database has changed;
when it is determined that the second information stored in the database has changed, determining whether a change to the second information would result in a change to content included in the second web page; and
when it is determined that the change to the second information would result in a change to content included in the second web page, sending a third request to the web server for the second web page.
10 Assignments
0 Petitions
Accused Products
Abstract
One embodiment of the invention is directed to automatically populating the cache of a web server. A webcrawler may send a request for a web page to the web server and receive the web page in response. The webcrawler may locate a link to a second web page in the received web page and determine if the link includes a predetermined pattern of characters. If the link includes the predetermined pattern of characters, the webcrawler may send a request to the webserver for the second web page. In another embodiment, the webcrawler may be made aware of changes to the content of a web site and revisit the web site after changes are made to repopulate the cache with the updated content.
31 Citations
14 Claims
-
1. A method of accessing a plurality of web pages served by a web server, the method comprising:
-
sending a first request to the web server for a first web page of the plurality of web pages; receiving, from the web server, the requested first web page in response to the first request; locating, in the first web page, a link to a second web page of the plurality of web pages; determining whether the link includes a pattern of characters that matches a predetermined pattern of characters; when it is determined that the link includes the pattern of characters, sending a plurality of second request to the web server for the second web page, wherein each of the plurality of second requests includes first information that emulates at least one user characteristic, wherein each of the plurality of second requests provides a different value for the first information that emulates the at least one user characteristic, and wherein each of the plurality of second requests, when processed by the web server, causes the web server to dynamically generate a different version of the second web page that includes content based, at least in part, on the value of the first information that emulates the at least one user characteristic, the web server dynamically generating each different version of the second web page using second information retrieved from a database; after sending the plurality of second requests to the web server for the second web page, monitoring the database to determine whether the second information stored in the database has changed; when it is determined that the second information stored in the database has changed, determining whether a change to the second information would result in a change to content included in the second web page; and when it is determined that the change to the second information would result in a change to content included in the second web page, sending a third request to the web server for the second web page. - View Dependent Claims (2, 3, 4, 12, 13, 14)
-
-
5. At least one non-transitory computer readable medium, encoded with instructions that, when executed on a computer system including a web server, perform a method of accessing a plurality of web pages served by the web server, the method comprising:
-
sending a first request to the web server for a first web page of the plurality of web pages; receiving, from the web server, the requested first web page in response to the first request; locating, in the first web page, a link to a second web page of the plurality of web pages; determining whether the link includes a pattern of characters that matches a predetermined pattern of characters; and when it is determined that the link includes the pattern of characters, sending a plurality of second requests to the web server for the second web page, wherein each of the plurality of second requests includes first information that emulates at least one user characteristic, wherein each of the plurality of second requests provides a different value for the first information that emulates the at least one user characteristic, and wherein each of the plurality of second requests, when processed by the web server, causes the web server to dynamically generate a different version of the second web page that includes content based, at least in part, on the value of the first information that emulates the at least one user characteristic, the web server dynamically generating each different version of the second web page using second information retrieved from a database; after sending the plurality of second requests to the web server for the second web page, monitoring the database to determine whether the second information stored in the database has changed; when it is determined that the second information stored in the database has changed, determined whether a change to the second information would result in a change to content included in the second web page; and when it is determined that a change to the second information would result in a change to content included in the second web page, sending a third request to the web server for the second web page. - View Dependent Claims (6, 7, 8)
-
-
9. A computer system comprising:
-
a web server operable to provide access to a plurality of web pages; and at least one controller operable to; send a first request to the web server for a first web page of the plurality of web pages; receive, from the web server, the requested first web page in response to the first request; locate, in the first web page, a link to a second web page of the plurality of web pages; determine whether the link includes a pattern of characters that matches a predetermined pattern of characters; and when it is determined that the link includes the pattern of characters, send a plurality of second requests to the web server for the second web page, wherein each of the plurality of second requests includes first information that emulates at least one user characteristic, wherein each of the plurality of second requests provides a different value for the first information that emulates the at least one user characteristic, and wherein each of the plurality of second requests, when processed by the web server, causes the web server to dynamically generate a different version of the second web page that includes content based, at least in part, on the value of the first information that emulates the at least one user characteristic, the web server dynamically generating each different version of the second web page using second information retrieved from a database; after sending the plurality of second requests to the web server for the second web page, monitor the database to determine whether the second information stored in the database had changed; when it is determined that the second information stored in the database has changed, determine whether a change to the second information would result in a change to content included in the second web page; and when it is determined that a change to the second information would result in a change to content included in the second web page, send a third request to the web server for the second web page. - View Dependent Claims (10, 11)
-
Specification