System and method for intelligent caching and refresh of dynamically generated and static web content

• US 6,591,266 B1
• Filed: 08/14/2000
• Issued: 07/08/2003
• Est. Priority Date: 07/14/2000
• Status: Active Grant

• Current Assignee: NEC Corporation
• Original Assignee: NEC Corporation
• Inventors: Li, Wen Syan, Candan, Kasim Selouk, Agrawal, Divyakant
• Primary Examiner(s): Robinson, Greta
• Assistant Examiner(s): Black, Linh
• Application Number:
  US09/639,208
• Time in Patent Office: 1,058 Days
• Field of Search: 707/4-10, 707/100-104
• US Class Current: 1/1
• CPC Class Codes: G06F 16/9574 of access to content, e.g. by caching
• CPC Class Codes: Y10S 707/99932 Access augmentation or optimizing
• CPC Class Codes: Y10S 707/99939 Privileged access
• CPC Class Codes: Y10S 707/99945 Object-oriented database structure processing

Abstract

A system for updating Web pages stored in cache based on modifications to data stored in a database is disclosed. The system for updating stored Web pages may be part of a larger system having a database management system for storing data used to generate Web pages. The database management system is capable of identifying modified data stored in the database. The system for updating stored Web pages is comprised of one or more servers programmed for maintaining associations between the stored Web pages and the stored data, and receiving the identity of modified data from the memory management system. In addition, the servers are capable of determining, from the identified modified data and the maintained associations, which stored Web pages are associated with the identified modified data. Furthermore, the servers are capable of communicating an update command to the cache that contains the stored Web pages associated with the identified modified data, for the purpose of updating the stored Web pages.

Claims

1. In a system having at least one first memory for storing Web pages and a memory management system including at least one second memory for storing data used to generate Web pages, the memory management system capable of identifying modified data stored in the at least one second memory and maintaining at least one database log sequentially containing all operations to the at least one second memory, a system for updating stored Web pages based on modifications to data stored in the at least one second memory, comprising:

at least one server programmed for

maintaining associations between each stored Web page and the stored data used to generate that particular Web page,

receiving the identity of modified data from the memory management system,

determining, from the identified modified data and the maintained associations, which stored Web pages are associated with the identified modified data, and

communicating an update command to the at least one first memory that contains the stored Web pages associated with the identified modified data for updating the stored Web pages.

2. A system for updating stored Web pages as recited in claim 1, the at least one server further programmed for:

receiving a Web content request containing a URL;

issuing at least one query or operation to the at least one second memory in response to the Web content request;

maintaining associations between the URL of the Web content request and the issued queries or operations;

maintaining associations between the issued queries or operations and the stored data accessed by the queries or operations;

deriving associations between the stored Web pages and the stored data from the maintained associations between the URL of the Web content request and the issued queries or operations, and the maintained associations between the issued queries or operations and the stored data accessed by the queries or operations; and

identifying, through prior processing steps to store the stored Web pages in the at least one first memory, the location of the Web pages associated with the identified modified data.

3. A system for updating the stored Web pages as recited in claim 2, wherein the update command includes a message to invalidate or delete the stored Web pages.

4. A system for updating the stored Web pages as recited in claim 2, wherein the update command includes a message to refresh the stored Web pages.

5. A system for updating the stored Web pages as recited in claim 4, wherein if the stored Web page is in a Web server, the update command includes a message to generate a new Web page and store the new Web page in the Web server.

6. A system for updating stored Web pages as recited in claim 2, the at least one server further programmed for:

maintaining a Web server log containing a request and delivery timestamp for the received URL;

maintaining a query and operation log containing an issuance timestamp for the issued query or operation;

deriving associations between the stored Web pages and the stored data from the Web server log and the query and operation log; and

determining, from the identity of the modified data and the derived MO associations between the stored Web pages and the stored data, which stored Web pages are associated with the identified modified data.

7. A system for updating stored Web pages as recited in claim 6, the at least one server including:

a Web server programmed for receiving the Web content request and maintaining the Web server log;

an application server programmed for receiving a forwarded Web content request from the Web server and issuing the at least one query or operation to the at least one second memory in response to the forwarded Web content request;

a content change monitor programmed for maintaining the associations between the issued queries or operations and the stored data accessed by the queries or operations, and determining which queries or operations are associated with the identified modified data; and

a content delivery services server programmed for determining the associations between the URLs of the Web content requests and the issued queries or operations from the query and operation logs and the Web server log, receiving an identity of the queries or operations associated with the identified modified data from the content change monitor, determining which stored Web pages are associated with the queries or operations associated with the identified modified data, and communicating the update command to the at least one first memory that contains the stored Web pages associated with the identified modified data.

8. A system for updating the stored Web pages as recited in claim 7, the content change monitor programmed for:

receiving queries sent to the at least one second memory;

defining a view table for each query containing pointers to all data in the at least one second memory that is associated with that particular query;

receiving the identified modified data from the memory management system; and

determining from the view table and the identified modified data any queries associated with the identified modified data.

9. A system for updating the stored Web pages as recited in claim 7, the content change monitor programmed for:

receiving operations sent to the at least one second memory;

defining a daemon for each operation, the daemon establishing criteria for identifying stored data in the at least one second memory that is associated with that particular operation; and

determining queries associated with the identified modified data by determining if the identified modified data satisfies the daemon.

10. A system for updating the stored Web pages as recited in claim 7, the content change monitor programmed for:

receiving queries sent to the at least one second memory;

creating a trigger definition for each query, the trigger definition establishing criteria for identifying stored data in the at least one second memory associated with that particular query; and

determining queries associated with the identified modified data by determining if the identified modified data satisfies the trigger definition.

11. A system for updating the stored Web pages as recited in claim 7, the content change monitor containing at least one third memory and further programmed for:

replicating data from the at least one second memory to the at least one third memory to harmonize the at least one third memory with the at least one second memory; and

defining the view table for each query containing pointers to all data in the at least one third memory that is associated with that particular query.

12. A system for updating the stored Web pages as recited in claim 8, the content change monitor containing at least one third memory and further programmed for:

performing all operations in the database log on the at least one third memory, in sequential order, to harmonize the at least one third memory with the at least one second memory; and

defining the view table for each query containing pointers to all data in the at least one third memory that is associated with that particular query.

13. A system for updating the stored Web pages as recited in claim 6, further including a sniffer for listening to network traffic by reading request/delivery logs, collecting URL information and query or operation instances by reading a query instance delivery log, and generating the associations between the URL of the Web content request and the issued queries or operations.

14. A system for updating the stored Web pages as recited in claim 6, further including an invalidator/refresher for notifying appropriate caches or Web servers of the invalidity of a particular stored Web page, the invalidator/refresher including:

a registration module for creating invalidation policies and collecting relevant query or operation statistics;

an invalidation module for reading the database update logs and the associations between the URL of the Web content request and the issued queries or operations, scheduling subqueries, and notifying appropriate caches or Web servers of the invalidity of a particular stored Web page; and

an information management module for creating auxiliary data structures for the invalidation module.

15. A system for updating the stored Web pages as recited in claim 6, the at least one server further programmed for:

receiving a query sent to the at least one second memory, the query including one or more query conditions linked by operators, the linked query conditions identifying data in the at least one second memory associated with that query; and

determining if the query is associated with the identified modified data by determining if the identified modified data satisfies the linked query conditions.

16. A system for updating the stored Web pages as recited in claim 15, wherein if the linked query conditions require access to more than one table in the at least one second memory, the at least one server is further programmed such that for each table to which access is required, the at least one server determines if the identified modified data satisfies those portions of the linked query condition specific to that particular table;

wherein after it is determined whether the identified modified data satisfies those portions of the linked query condition specific to a particular table, those portions of the linked query condition specific to a particular table are replaced by a determined value prior to evaluating those portions of the linked query condition specific to another particular table; and

wherein, if at any time during the determination of whether the identified modified data satisfies those portions of the linked query condition specific to a particular table, it is determined that the linked query conditions are satisfied, then the identified modified data satisfies the linked query conditions.

17. A system for updating the stored Web pages as recited in claim 16, wherein after those portions of the linked query condition specific to a particular table are evaluated and a value for those portions is determined, in order to evaluate those portions of the linked query condition specific to another particular table, the at least one server is further programmed for:

issuing a subquery to the other particular table containing the linked query conditions, with portions of the linked query condition replaced by any previously determined values.

18. A system for updating the stored Web pages as recited in claim 17, wherein after a subquery issued to a particular table returns determined values for that subquery, the determined values are stored in an index structure;

wherein if subsequent queries are received by the at least one server that would require reissuing the same subquery, the index structure may be accessed by the at least one server to avoid having to issue the same subquery.

19. A system for updating stored Web pages as recited in claim 2, the at least one server further programmed for:

maintaining a URL/relevant operation mapping log from the URLs contained in the received Web content requests and the queries or operations issued to the least one second memory in response to the Web content requests; and

determining, from the identity of the modified data, the maintained associations between the issued queries or operations and the stored data, and the URL/relevant operation mapping log, which stored Web pages are associated with the identified modified data.

20. A system for updating stored Web pages as recited in claim 19, the at least one server including:

an application server programmed for maintaining the URL/relevant operation mapping log, receiving a forwarded Web content request and issuing the at least one query or operation to the at least one second memory in response to the forwarded Web content request;

a content change monitor programmed for maintaining the associations between the issued queries or operations and the stored data accessed by the queries or operations, and determining which queries or operations are associated with the identified modified data; and

a content delivery services server programmed for receiving an identity of the queries or operations associated with the identified modified data from the content change monitor, receiving the URL/relevant operation mapping log, determining which stored Web pages are associated with the queries or operations associated with the identified modified data, and communicating the update command to the at least one first memory that contains the stored Web pages associated with the identified modified data.

21. A system for updating the stored Web pages as recited in claim 20, the content change monitor programmed for:

receiving queries sent to the at least one second memory;

defining a view table for each query containing pointers to all data in the at least one second memory that is associated with that particular query;

receiving the identified modified data from the memory management system; and

determining from the view table and the identified modified data any queries associated with the identified modified data.

22. A system for updating the stored Web pages as recited in claim 20, the content change monitor programmed for:

receiving operations sent to the at least one second memory;

defining a daemon for each operation, the daemon establishing criteria for identifying stored data in the at least one second memory that is associated with that particular operation; and

determining queries associated with the identified modified data by determining if the identified modified data satisfies the daemon.

23. A system for updating the stored Web pages as recited in claim 20, the content change monitor programmed for:

receiving queries sent to the at least one second memory;

creating a trigger definition for each query, the trigger definition establishing criteria for identifying stored data in the at least one second memory associated with that particular query;

determining queries associated with the identified modified data by determining if the identified modified data satisfies the trigger definition.

24. A system for updating the stored Web pages as recited in claim 20, the content change monitor containing at least one third memory and further programmed for:

replicating data from the at least one second memory to the at least one third memory to harmonize the at least one third memory with the at least one second memory; and

defining the view table for each query containing pointers to all data in the at least one third memory that is associated with that particular query.

25. A system for updating the stored Web pages as recited in claim 21, the content change monitor containing at least one third memory and further programmed for:

performing all operations in the database log on the at least one third memory, in sequential order, to harmonize the at least one third memory with the at least one second memory; and

defining the view table for each query containing pointers to all data in the at least one third memory that is associated with that particular query.

26. A system for updating the stored Web pages as recited in claim 19, further including a sniffer for listening to network traffic by reading request/delivery logs, collecting URL information and query or operation instances by reading a query instance delivery log, and generating the associations between the URL of the Web content request and the issued queries or operations.

27. A system for updating the stored Web pages as recited in claim 19, further including an invalidator/refresher for notifying appropriate caches or Web servers of the invalidity of a particular stored Web page, the invalidator/refresher including:

a registration module for creating invalidation policies and collecting relevant query or operation statistics;

an invalidation module for reading the database update logs and the associations between the URL of the Web content request and the issued queries or operations, scheduling subqueries, and notifying appropriate caches or Web servers of the invalidity of a particular stored Web page; and

an information management module for creating auxiliary data structures for the invalidation module.

28. A system for updating the stored Web pages as recited in claim 19, the at least one server further programmed for:

receiving a query sent to the at least one second memory, the query including one or more query conditions linked by operators, the linked query conditions identifying data in the at least one second memory associated with that query; and

determining if the query is associated with the identified modified data by determining if the identified modified data satisfies the linked query conditions.

29. A system for updating the stored Web pages as recited in claim 28, wherein if the linked query conditions require access to more than one table in the at least one second memory, the at least one server is further programmed such that for each table to which access is required, the at least one server determines if the identified modified data satisfies those portions of the linked query condition specific to that particular table;

wherein after it is determined whether the identified modified data satisfies those portions of the linked query condition specific to a particular table, those portions of the linked query condition specific to a particular table are replaced by a determined value prior to evaluating those portions of the linked query condition specific to another particular table; and

wherein, if at any time during the determination of whether the identified modified data satisfies those portions of the linked query condition specific to a particular table, it is determined that the linked query conditions are satisfied, then the identified modified data satisfies the linked query conditions.

30. A system for updating the stored Web pages as recited in claim 29, wherein after those portions of the linked query condition specific to a particular table are evaluated and a value for those portions is determined, in order to evaluate those portions of the linked query condition specific to another particular table, the at least one server is further programmed for:

issuing a subquery to the other particular table containing the linked query conditions, with portions of the linked query condition replaced by any previously determined values.

31. A system for updating the stored Web pages as recited in claim 30, wherein after a subquery issued to a particular table returns determined values for that subquery, the determined values are stored in an index structure;

wherein if subsequent queries are received by the at least one server that would require reissuing the same subquery, the index structure may be accessed by the at least one server to avoid having to issue the same subquery.

32. A system for updating stored Web pages as recited in claim 19, the at least one server including:

an application server programmed for receiving a forwarded Web content request and issuing a content delivery services provider API call;

a content delivery services provider API handler programmed for receiving the content delivery services provider API call, maintaining the URL/relevant operation mapping log, and issuing the at least one query or operation to the at least one second memory in response to the content delivery services provider API call;

a content change monitor programmed for maintaining the associations between the issued queries or operations and the stored data accessed by the queries or operations, and determining which queries or operations are associated with the identified modified data; and

a content delivery services server programmed for receiving an identity of the queries or operations associated with the identified modified data from the content change monitor, receiving the URL/relevant operation mapping log, determining which stored Web pages are associated with the queries or operations associated with the identified modified data, and communicating the update command to the at least one first memory that contains the stored Web pages associated with the identified modified data.

33. A system for updating the stored Web pages as recited in claim 32, the content change monitor programmed for:

receiving queries sent to the at least one second memory;

defining a view table for each query containing pointers to all data in the at least one second memory that is associated with that particular query;

receiving the identified modified data from the memory management system; and

determining from the view table and the identified modified data any queries associated with the identified modified data.

34. A system for updating the stored Web pages as recited in claim 32, the content change monitor programmed for:

receiving operations sent to the at least one second memory;

defining a daemon for each operation, the daemon establishing criteria for identifying stored data in the at least one second memory that is associated with that particular operation; and

determining queries associated with the identified modified data by determining if the identified modified data satisfies the daemon.

35. A system for updating the stored Web pages as recited in claim 32, the content change monitor programmed for:

receiving queries sent to the at least one second memory;

creating a trigger definition for each query, the trigger definition establishing criteria for identifying stored data in the at least one second memory associated with that particular query; and

determining queries associated with the identified modified data by determining if the identified modified data satisfies the trigger definition.

36. A system for updating the stored Web pages as recited in claim 32, the content change monitor containing at least one third memory and further programmed for:

replicating data from the at least one second memory to the at least one third memory to harmonize the at least one third memory with the at least one second memory; and

defining the view table for each query containing pointers to all data in the at least one third memory that is associated with that particular query.

37. A system for updating the stored Web pages as recited in claim 33, the content change monitor containing at least one third memory and further programmed for:

performing all operations in the database log on the at least one third memory, in sequential order, to harmonize the at least one third memory with the at least one second memory; and

defining the view table for each query containing pointers to all data in the at least one third memory that is associated with that particular query.

38. A system for updating stored Web pages as recited in claim 2, the at least one server further programmed for:

converting received Web content requests into stored procedures;

maintaining a database query log from the URLs contained in the stored procedures and the queries issued to the least one second memory in response to the stored procedures;

maintaining an operation log from the URLs contained in the stored procedures and the operations issued to the least one second memory in response to the stored procedures;

generating a URL/relevant operation mapping table from the database query log and the operation log; and

determining, from the identity of the modified data, the maintained associations between the issued queries or operations and the stored data, and the URL/relevant operation mapping table which stored Web pages are associated with the identified modified data.

39. A system for updating stored Web pages as recited in claim 38, the at least one server including:

an application server programmed for converting received Web content requests into stored procedures, generating the database query log from the URLs contained in the stored procedures and the queries issued to the least one second memory in response to the stored procedures, and generating the operation log from the URLs contained in the stored procedures and the operations issued to the least one second memory in response to the stored procedures;

a content change monitor programmed for maintaining the associations between the issued queries or operations and the stored data accessed by the queries or operations, and determining which queries or operations are associated with the identified modified data; and

a content delivery services server for generating a URL/relevant operation mapping table from the database query log and the operation log and determining, from the identity of the modified data, the maintained associations between the issued queries or operations and the stored data, and the URL/relevant operation mapping table, which stored Web pages are associated with the identified modified data, and communicating the update command to the at least one first memory that contains the stored Web pages associated with the identified modified data.

40. A system for updating the stored Web pages as recited in claim 39, the content change monitor programmed for:

receiving queries sent to the at least one second memory;

defining a view table for each query containing pointers to all data in the at least one second memory that is associated with that particular query;

receiving the identified modified data from the memory management system; and

determining from the view table and the identified modified data any queries associated with the identified modified data.

41. A system for updating the stored Web pages as recited in claim 39, the content change monitor programmed for:

receiving queries or operations sent to the at least one second memory;

defining a daemon for each query or operation, the daemon establishing criteria for identifying stored data in the at least one second memory that is associated with that particular query or operation; and

determining queries associated with the identified modified data by determining if the identified modified data satisfies the daemon.

42. A system for updating the stored Web pages as recited in claim 39, the content change monitor programmed for:

receiving queries sent to the at least one second memory;

creating a trigger definition for each query, the trigger definition establishing criteria for identifying stored data in the at least one second memory associated with that particular query; and

determining queries associated with the identified modified data by determining if the identified modified data satisfies the trigger definition.

43. A system for updating the stored Web pages as recited in claim 39, the content change monitor containing at least one third memory and further programmed for:

replicating data from the at least one second memory to the at least one third memory to harmonize the at least one third memory with the at least one second memory; and

defining the view table for each query containing pointers to all data in the at least one third memory that is associated with that particular query.

44. A system for updating the stored Web pages as recited in claim 40, the content change monitor containing at least one third memory and further programmed for:

performing all operations in the database log on the at least one third memory, in sequential order, to harmonize the at least one third memory with the at least one second memory; and

defining the view table for each query containing pointers to all data in the at least one third memory that is associated with that particular query.

45. A system for updating the stored Web pages as recited in claim 38, further including a sniffer for listening to network traffic by reading request/delivery logs, collecting URL information and query or operation instances by reading a query instance delivery log, and generating the associations between the URL of the Web content request and the issued queries or operations.

46. A system for updating the stored Web pages as recited in claim 38, further including an invalidator/refresher for notifying appropriate caches or Web servers of the invalidity of a particular stored Web page, the invalidator/refresher including:

a registration module for creating invalidation policies and collecting relevant query or operation statistics;

an invalidation module for reading the database update logs and the associations between the URL of the Web content request and the issued queries or operations, scheduling subqueries, and notifying appropriate caches or Web servers of the invalidity of a particular stored Web page; and

an information management module for creating auxiliary data structures for the invalidation module.

47. A system for updating the stored Web pages as recited in claim 38, the at least one server further programmed for:

receiving a query sent to the at least one second memory, the query including one or more query conditions linked by operators, the linked query conditions identifying data in the at least one second memory associated with that query; and

determining if the query is associated with the identified modified data by determining if the identified modified data satisfies the linked query conditions.

48. A system for updating the stored Web pages as recited in claim 47, wherein if the linked query conditions require access to more than one table in the at least one second memory, the at least one server is further programmed such that for each table to which access is required, the at least one server determines if the identified modified data satisfies those portions of the linked query condition specific to that particular table;

wherein after it is determined whether the identified modified data satisfies those portions of the linked query condition specific to a particular table, those portions of the linked query condition specific to a particular table are replaced by a determined value prior to evaluating those portions of the linked query condition specific to another particular table; and

wherein, if at any time during the determination of whether the identified modified data satisfies those portions of the linked query condition specific to a particular table, it is determined that the linked query conditions are satisfied, then the identified modified data satisfies the linked query conditions.

49. A system for updating the stored Web pages as recited in claim 48, wherein after those portions of the linked query condition specific to a particular table are evaluated and a value for those portions is determined, in order to evaluate those portions of the linked query condition specific to another particular table, the at least one server is further programmed for:

issuing a subquery to the other particular table containing the linked query conditions, with portions of the linked query condition replaced by any previously determined values.

50. A system for updating the stored Web pages as recited in claim 49, wherein after a subquery issued to a particular table returns determined values for that subquery, the determined values are stored in an index structure;

wherein if subsequent queries are received by the at least one server that would require reissuing the same subquery, the index structure may be accessed by the at least one server to avoid having to issue the same subquery.

51. A system for updating stored Web pages as recited in claim 2, the at least one server further programmed for:

extracting URL information from the received the Web content requests;

extracting query and operation information from the issued queries and operations;

maintaining a URL/relevant operation mapping log from the extracted URL information and the extracted query and operation information; and

determining, from the identity of the modified data, the maintained associations between the issued queries or operations and the stored data, and the URL/relevant operation mapping log, which stored Web pages are associated with the identified modified data.

52. A system for updating stored Web pages as recited in claim 51, the at least one server including:

an application server proxy for extracting URL information from the received the Web content requests, extracting query and operation information from the issued queries and operations, forwarding the received Web content requests, and maintaining the URL/relevant operation mapping log;

an application server for receiving Web server requests forwarded by the application server proxy and issuing the at least one query or operation in response to the forwarded Web content request;

a content change monitor programmed for maintaining the associations between the issued queries or operations and the stored data accessed by the queries or operations, and determining which queries or operations are associated with the identified modified data; and

a content delivery services server programmed for receiving an identity of the queries or operations associated with the identified modified data from the content change monitor, receiving the URL/relevant operation mapping log, determining which stored Web pages are associated with the queries or operations associated with the identified modified data, and communicating the update command to the at least one first memory that contains the stored Web pages associated with the identified modified data.

53. A system for updating the stored Web pages as recited in claim 52, the content change monitor programmed for:

receiving queries sent to the at least one second memory;

defining a view table for each query containing pointers to all data in the at least one second memory that is associated with that particular query;

receiving the identified modified data from the memory management system; and

determining from the view table and the identified modified data any queries associated with the identified modified data.

54. A system for updating the stored Web pages as recited in claim 52, the content change monitor programmed for:

receiving operations sent to the at least one second memory;

defining a daemon for each operation, the daemon establishing criteria for identifying stored data in the at least one second memory that is associated with that particular operation; and

determining queries associated with the identified modified data by determining if the identified modified data satisfies the daemon.

55. A system for updating the stored Web pages as recited in claim 52, the content change monitor programmed for:

receiving queries sent to the at least one second memory;

creating a trigger definition for each query, the trigger definition establishing criteria for identifying stored data in the at least one second memory associated with that particular query; and

determining queries associated with the identified modified data by determining if the identified modified data satisfies the trigger definition.

56. A system for updating the stored Web pages as recited in claim 52, the content change monitor containing at least one third memory and further programmed for:

replicating data from the at least one second memory to the at least one third memory to harmonize the at least one third memory with the at least one second memory; and

defining the view table for each query containing pointers to all data in the at least one third memory that is associated with that particular query.

57. A system for updating the stored Web pages as recited in claim 53, the content change monitor containing at least one third memory and further programmed for:

performing all operations in the database log on the at least one third memory, in sequential order, to harmonize the at least one third memory with the at least one second memory; and

defining the view table for each query containing pointers to all data in the at least one third memory that is associated with that particular query.

58. A system for updating the stored Web pages as recited in claim 51, further including a sniffer for listening to network traffic by reading request/delivery logs, collecting URL information and query or operation instances by reading a query instance delivery log, and generating the associations between the URL of the Web content request and the issued queries or operations.

59. A system for updating the stored Web pages as recited in claim 51, further including an invalidator/refresher for notifying appropriate caches or Web servers of the invalidity of a particular stored Web page, the invalidator/refresher including:

a registration module for creating invalidation policies and collecting relevant query or operation statistics;

an invalidation module for reading the database update logs and the associations between the URL of the Web content request and the issued queries or operations, scheduling subqueries, and notifying appropriate caches or Web servers of the invalidity of a particular stored Web page; and

an information management module for creating auxiliary data structures for the invalidation module.

60. A system for updating the stored Web pages as recited in claim 51, the at least one server further programmed for:

receiving a query sent to the at least one second memory, the query including one or more query conditions linked by operators, the linked query conditions identifying data in the at least one second memory associated with that query; and

determining if the query is associated with the identified modified data by determining if the identified modified data satisfies the linked query conditions.

61. A system for updating the stored Web pages as recited in claim 60, wherein if the linked query conditions require access to more than one table in the at least one second memory, the at least one server is further programmed such that for each table to which access is required, the at least one server determines if the identified modified data satisfies those portions of the linked query condition specific to that particular table;

wherein after it is determined whether the identified modified data satisfies those portions of the linked query condition specific to a particular table, those portions of the linked query condition specific to a particular table are replaced by a determined value prior to evaluating those portions of the linked query condition specific to another particular table; and

wherein, if at any time during the determination of whether the identified modified data satisfies those portions of the linked query condition specific to a particular table, it is determined that the linked query conditions are satisfied, then the identified modified data satisfies the linked query conditions.

62. A system for updating the stored Web pages as recited in claim 61, wherein after those portions of the linked query condition specific to a particular table are evaluated and a value for those portions is determined, in order to evaluate those portions of the linked query condition specific to another particular table, the at least one server is further programmed for:

issuing a subquery to the other particular table containing the linked query conditions, with portions of the linked query condition replaced by any previously determined values.

63. A system for updating the stored Web pages as recited in claim 62, wherein after a subquery issued to a particular table returns determined values for that subquery, the determined values are stored in an index structure;

wherein if subsequent queries are received by the at least one server that would require reissuing the same subquery, the index structure may be accessed by the at least one server to avoid having to issue the same subquery.

117. A system for updating the stored Web pages as recited in claim 14, the registration module further programmed for:

registering, in an off-line mode, those query types that the invalidator/refresher must detect, and storing the registered query types in an auxiliary data structure;

scanning the associations between the URL of the Web content request and the issued queries or operations, locating query instances that cannot be associated with a known query type, interpreting those query instances to identify their query types, and registering, in an on-line mode, the identified query types; and

determining, based on join index size, update frequency, or query frequency, whether a join index should be maintained;

wherein updating the stored Web pages includes refreshing, invalidating, or deleting the stored Web pages.

118. A system for updating the stored Web pages as recited in claim 117, wherein the determination of whether a join index should be maintained is favored if the join index size is small, the join index update frequency is low, the database or external data source query frequency is high, or the database or external data source query cost is high.

119. A system for updating the stored Web pages as recited in claim 14, the invalidation module further programmed for:

performing database polling request scheduling by reading invalidation policies and the auxiliary data structures to identify which database polling requests must be generated, and determining when these requests should be sent to memory management systems or external data sources; and

converting database polling requests into subqueries or suboperations readable by the memory management systems or the external data sources;

wherein updating the stored Web pages includes refreshing, invalidating, or deleting the stored Web pages.

120. A system for updating the stored Web pages as recited in claim 14, the information management module further programmed for:

performing query instance coalescing by identifying related query instances and processing them as a group;

performing query type coalescing by identifying related query types and processing them in a coordinated manner; and

performing update coalescing by identifying related database or external data source updates, deletions, or insertions, and processing them as a group.

121. A system for updating the stored Web pages as recited in claim 14, the invalidator/refresher further programmed for:

forming a positive query lattice for a particular query, the positive query lattice comprised of a sequence of query conditions arranged in an order such that if a database or external data source change is detected, and the sequence of query conditions is evaluated, as soon as one query condition is not satisfied, it can be determined that the database or external data source change will not affect the particular query, and no further query conditions will need to be evaluated;

forming a negative query lattice for a particular query, the negative query lattice comprised of a sequence of query conditions arranged in an order such that if a database or external data source change is detected, and the sequence of query conditions is evaluated, as soon as one query condition is satisfied, it can be determined that the database or external data source change will affect the particular query, and no further query conditions will need to be evaluated; and

utilizing the positive query lattice for determining when a database or external data source change will not affect a query, and utilizing the negative query lattice for determining when a database or external data source change will affect a query.

122. A system for updating the stored Web pages as recited in claim 14, the invalidator/refresher further programmed for:

determining, based on a number or complexity of queries generated by a particular Web page request, whether to cache, invalidate, or refresh the stored Web pages.

123. A system for updating the stored Web pages as recited in claim 27, the registration module further programmed for:

registering, in an off-line mode, those query types that the invalidator/refresher must detect, and storing the registered query types in an auxiliary data structure;

scanning the associations between the URL of the Web content request and the issued queries or operations, locating query instances that cannot be associated with a known query type, interpreting those query instances to identify their query types, and registering, in an on-line mode, the identified query types; and

determining, based on join index size, update frequency, or query frequency, whether a join index should be maintained;

wherein updating the stored Web pages includes refreshing, invalidating, or deleting the stored Web pages.

124. A system for updating the stored Web pages as recited in claim 123, wherein the determination of whether a join index should be maintained is favored if the join index size is small, the join index update frequency is low, the database or external data source query frequency is high, or the database or external data source query cost is high.

125. A system for updating the stored Web pages as recited in claim 27, the invalidation module further programmed for:

performing database polling request scheduling by reading invalidation policies and the auxiliary data structures to identify which database polling requests must be generated, and determining when these requests should be sent to memory management systems or external data sources; and

converting database polling requests into subqueries or suboperations readable by the memory management systems or the external data sources;

wherein updating the stored Web pages includes refreshing, invalidating, or deleting the stored Web pages.

126. A system for updating the stored Web pages as recited in claim 27, the information management module further programmed for:

performing query instance coalescing by identifying related query instances and processing them as a group;

performing query type coalescing by identifying related query types and processing them in a coordinated manner; and

performing update coalescing by identifying related database or external data source updates, deletions, or insertions, and processing them as a group.

127. A system for updating the stored Web pages as recited in claim 27, the invalidator/refresher further programmed for:

forming a positive query lattice for a particular query, the positive query lattice comprised of a sequence of query conditions arranged in an order such that if a database or external data source change is detected, and the sequence of query conditions is evaluated, as soon as one query condition is not satisfied, it can be determined that the database or external data source change will not affect the particular query, and no further query conditions will need to be evaluated;

forming a negative query lattice for a particular query, the negative query lattice comprised of a sequence of query conditions arranged in an order such that if a database or external data source change is detected, and the sequence of query conditions is evaluated, as soon as one query condition is satisfied, it can be determined that the database or external data source change will affect the particular query, and no further query conditions will need to be evaluated; and

utilizing the positive query lattice for determining when a database or external data source change will not affect a query, and utilizing the negative query lattice for determining when a database or external data source change will affect a query.

128. A system for updating the stored Web pages as recited in claim 27, the invalidator/refresher further programmed for:

determining, based on a number or complexity of queries generated by a particular Web page request, whether to cache, invalidate, or refresh the stored Web pages.

129. A system for updating the stored Web pages as recited in claim 46, the registration module further programmed for:

registering, in an off-line mode, those query types that the invalidator/refresher must detect, and storing the registered query types in an auxiliary data structure;

scanning the associations between the URL of the Web content request and the issued queries or operations, locating query instances that cannot be associated with a known query type, interpreting those query instances to identify their query types, and registering, in an on-line mode, the identified query types; and

determining, based on join index size, update frequency, or query frequency, whether a join index should be maintained;

wherein updating the stored Web pages includes refreshing, invalidating, or deleting the stored Web pages.

130. A system for updating the stored Web pages as recited in claim 129, wherein the determination of whether a join index should be maintained is favored if the join index size is small, the join index update frequency is low, the database or external data source query frequency is high, or the database or external data source query cost is high.

131. A system for updating the stored Web pages as recited in claim 46, the invalidation module further programmed for:

performing database polling request scheduling by reading invalidation policies and the auxiliary data structures to identify which database polling requests must be generated, and determining when these requests should be sent to memory management systems or external data sources; and

converting database polling requests into subqueries or suboperations readable by the memory management systems or the external data sources;

wherein updating the stored Web pages includes refreshing, invalidating, or deleting the stored Web pages.

132. A system for updating the stored Web pages as recited in claim 46, the information management module further programmed for:

performing query instance coalescing by identifying related query instances and processing them as a group;

performing query type coalescing by identifying related query types and processing them in a coordinated manner; and

performing update coalescing by identifying related database or external data source updates, deletions, or insertions, and processing them as a group.

133. A system for updating the stored Web pages as recited in claim 46, the invalidator/refresher further programmed for:

forming a positive query lattice for a particular query, the positive query lattice comprised of a sequence of query conditions arranged in an order such that if a database or external data source change is detected, and the sequence of query conditions is evaluated, as soon as one query condition is not satisfied, it can be determined that the database or external data source change will not affect the particular query, and no further query conditions will need to be evaluated;

forming a negative query lattice for a particular query, the negative query lattice comprised of a sequence of query conditions arranged in an order such that if a database or external data source change is detected, and the sequence of query conditions is evaluated, as soon as one query condition is satisfied, it can be determined that the database or external data source change will affect the particular query, and no further query conditions will need to be evaluated; and

utilizing the positive query lattice for determining when a database or external data source change will not affect a query, and utilizing the negative query lattice for determining when a database or external data source change will affect a query.

134. A system for updating the stored Web pages as recited in claim 46, the invalidator/refresher further programmed for:

determining, based on a number or complexity of queries generated by a particular Web page request, whether to cache, invalidate, or refresh the stored Web pages.

135. A system for updating the stored Web pages as recited in claim 59, the registration module further programmed for:

registering, in an off-line mode, those query types that the invalidator/refresher must detect, and storing the registered query types in an auxiliary data structure;

scanning the associations between the URL of the Web content request and the issued queries or operations, locating query instances that cannot be associated with a known query type, interpreting those query instances to identify their query types, and registering, in an on-line mode, the identified query types; and

determining, based on join index size, update frequency, or query frequency, whether a join index should be maintained;

wherein updating the stored Web pages includes refreshing, invalidating, or deleting the stored Web pages.

136. A system for updating the stored Web pages as recited in claim 135, wherein the determination of whether a join index should be maintained is favored if the join index size is small, the join index update frequency is low, the database or external data source query frequency is high, or the database or external data source query cost is high.

137. A system for updating the stored Web pages as recited in claim 59, the invalidation module further programmed for:

performing database polling request scheduling by reading invalidation policies and the auxiliary data structures to identify which database polling requests must be generated, and determining when these requests should be sent to memory management systems or external data sources; and

converting database polling requests into subqueries or suboperations readable by the memory management systems or the external data sources;

wherein updating the stored Web pages includes refreshing, invalidating, or deleting the stored Web pages.

138. A system for updating the stored Web pages as recited in claim 59, the information management module further programmed for:

performing query instance coalescing by identifying related query instances and processing them as a group;

performing query type coalescing by identifying related query types and processing them in a coordinated manner; and

performing update coalescing by identifying related database or external data source updates, deletions, or insertions, and processing them as a group.

139. A system for updating the stored Web pages as recited in claim 59, the invalidator/refresher further programmed for:

forming a positive query lattice for a particular query, the positive query lattice comprised of a sequence of query conditions arranged in an order such that if a database or external data source change is detected, and the sequence of query conditions is evaluated, as soon as one query condition is not satisfied, it can be determined that the database or external data source change will not affect the particular query, and no further query conditions will need to be evaluated;

forming a negative query lattice for a particular query, the negative query lattice comprised of a sequence of query conditions arranged in an order such that if a database or external data source change is detected, and the sequence of query conditions is evaluated, as soon as one query condition is satisfied, it can be determined that the database or external data source change will affect the particular query, and no further query conditions will need to be evaluated; and

utilizing the positive query lattice for determining when a database or external data source change will not affect a query, and utilizing the negative query lattice for determining when a database or external data source change will affect a query.

140. A system for updating the stored Web pages as recited in claim 59, the invalidator/refresher further programmed for:

determining, based on a number or complexity of queries generated by a particular Web page request, whether to cache, invalidate, or refresh the stored Web pages.

64. In a system having at least one first memory for storing Web pages and a memory management system including at least one second memory for storing data used to generate Web pages, the memory management system capable of identifying modified data stored in the at least one second memory and maintaining at least one database log sequentially containing all operations to the at least one second memory, a method for updating stored Web pages based on modifications to data stored in the at least one second memory comprising the steps of:

maintaining associations between each stored Web page and the stored data used to generate that particular Web page,

receiving the identity of modified data from the memory management system,

determining, from the identified modified data and the maintained associations, which stored Web pages are associated with the identified modified data, and

communicating an update command to the at least one first memory that contains the stored Web pages associated with the identified modified data for updating the stored Web pages.

65. A method for updating stored Web pages as recited in claim 64, the method further including the steps of:

receiving a Web content request containing a URL;

issuing at least one query or operation to the at least one second memory in response to the Web content request;

maintaining associations between the URL of the Web content request and the issued queries or operations;

maintaining associations between the issued queries or operations and the stored data accessed by the queries or operations;

deriving associations between the stored Web pages and the stored data from the maintained associations between the URL of the Web content request and the issued queries or operations, and the maintained associations between the issued queries or operations and the stored data accessed by the queries or operations; and

identifying, through prior processing steps to store the stored Web pages in the at least one first memory, the location of the Web pages associated with the identified modified data.

66. A method for updating the stored Web pages as recited in claim 65, wherein the step of communicating an update command includes communicating a message to invalidate or delete the stored Web pages.

67. A method for updating the stored Web pages as recited in claim 65, wherein the step of communicating an update command includes communicating a message to refresh the stored Web pages.

68. A method for updating the stored Web pages as recited in claim 67, wherein if the stored Web page is in a Web server, the step of communicating an update command includes communicating a message to generate a new Web page and store the new Web page in the Web server.

69. A method for updating stored Web pages as recited in claim 65, the method further including the steps of:

maintaining a Web server log containing a request and delivery timestamp for the received URL;

maintaining a query and operation log containing an issuance timestamp for the issued query or operation;

deriving associations between the stored Web pages and the stored data from the Web server log and the query and operation log; and

determining, from the identity of the modified data and the derived associations between the stored Web pages and the stored data, which stored Web pages are associated with the identified modified data.

70. A method for updating the stored Web pages as recited in claim 69, the method further including the steps of:

receiving queries sent to the at least one second memory;

defining a view table for each query containing pointers to all data in the at least one second memory that is associated with that particular query;

receiving the identified modified data from the memory management system; and

determining from the view table and the identified modified data any queries associated with the identified modified data.

71. A method for updating the stored Web pages as recited in claim 69, the method further including the steps of:

receiving operations sent to the at least one second memory;

defining a daemon for each operation, the daemon establishing criteria for identifying stored data in the at least one second memory that is associated with that particular operation; and

determining queries associated with the identified modified data by determining if the identified modified data satisfies the daemon.

72. A method for updating the stored Web pages as recited in claim 69, the method further including the steps of:

receiving queries sent to the at least one second memory;

creating a trigger definition for each query, the trigger definition establishing criteria for identifying stored data in the at least one second memory associated with that particular query; and

determining queries associated with the identified modified data by determining if the identified modified data satisfies the trigger definition.

73. A method for updating the stored Web pages as recited in claim 70, the method further including the steps of:

replicating data from the at least one second memory to at least one third memory to harmonize the at least one third memory with the at least one second memory; and

defining the view table for each query containing pointers to all data in the at least one third memory that is associated with that particular query.

74. A method for updating the stored Web pages as recited in claim 70, the method further including the steps of:

performing all operations in a database log on at least one third memory, in sequential order, to harmonize the at least one third memory with the at least one second memory; and

defining the view table for each query containing pointers to all data in the at least one third memory that is associated with that particular query.

75. A method for updating the stored Web pages as recited in claim 69, the method further including the steps of listening to network traffic by reading request/delivery logs, collecting URL information and query or operation instances by reading a query instance delivery log, and generating the associations between the URL of the Web content request and the issued queries or operations.

76. A method for updating the stored Web pages as recited in claim 69, the method further including the steps of:

creating invalidation policies and collecting relevant query or operation statistics;

reading database update logs and the associations between the URL of the Web content request and the issued queries or operations, scheduling subqueries, and notifying appropriate caches or Web servers of the invalidity of a particular stored Web page; and

creating auxiliary data structures.

77. A method for updating the stored Web pages as recited in claim 69, the method further including the steps of:

receiving a query sent to the at least one second memory, the query including one or more query conditions linked by operators, the linked query conditions identifying data in the at least one second memory associated with that query; and

determining if the query is associated with the identified modified data by determining if the identified modified data satisfies the linked query conditions.

78. A method for updating the stored Web pages as recited in claim 77, wherein if the linked query conditions require access to more than one table in the at least one second memory, the method further includes the step of determining, for each table to which access is required, if the identified modified data satisfies those portions of the linked query condition specific to that particular table;

wherein after it is determined whether the identified modified data satisfies those portions of the linked query condition specific to a particular table, the method further includes the step of replacing those portions of the linked query condition specific to a particular table by a determined value prior to evaluating those portions of the linked query condition specific to another particular table; and

wherein the method further includes the step of determining that the identified modified data satisfies the linked query conditions if, at any time, the linked query conditions are satisfied.

79. A method for updating the stored Web pages as recited in claim 78, wherein after those portions of the linked query condition specific to a particular table are evaluated and a value for those portions is determined, in order to evaluate those portions of the linked query condition specific to another particular table, the method further includes the step of:

issuing a subquery to the other particular table containing the linked query conditions, with portions of the linked query condition replaced by any previously determined values.

80. A method for updating the stored Web pages as recited in claim 79, wherein after a subquery issued to a particular table returns determined values for that subquery, the method further includes the step of storing the determined values in an index structure;

wherein if subsequent queries are received by the at least one server that would require reissuing the same subquery, the method further includes the step of accessing the index structure to avoid having to issue the same subquery.

81. A method for updating stored Web pages as recited in claim 65, the method further including the steps of:

maintaining a URL/relevant operation mapping log from the URLs contained in the received Web content requests and the queries or operations issued to the least one second memory in response to the Web content requests; and

determining, from the identity of the modified data, the maintained associations between the issued queries or operations and the stored data, and the URL/relevant operation mapping log, which stored Web pages are associated with the identified modified data.

82. A method for updating the stored Web pages as recited in claim 81, the method further including the steps of:

receiving queries sent to the at least one second memory;

defining a view table for each query containing pointers to all data in the at least one second memory that is associated with that particular query;

receiving the identified modified data from the memory management system; and

determining from the view table and the identified modified data any queries associated with the identified modified data.

83. A method for updating the stored Web pages as recited in claim 81, the method further including the steps of:

receiving operations sent to the at least one second memory;

defining a daemon for each operation, the daemon establishing criteria for identifying stored data in the at least one second memory that is associated with that particular operation; and

determining queries associated with the identified modified data by determining if the identified modified data satisfies the daemon.

84. A method for updating the stored Web pages as recited in claim 81, the method further including the steps of:

receiving queries sent to the at least one second memory;

creating a trigger definition for each query, the trigger definition establishing criteria for identifying stored data in the at least one second memory associated with that particular query; and

determining queries associated with the identified modified data by determining if the identified modified data satisfies the trigger definition.

85. A method for updating the stored Web pages as recited in claim 82, the method further including the steps of:

replicating data from the at least one second memory to at least one third memory to harmonize the at least one third memory with the at least one second memory; and

defining the view table for each query containing pointers to all data in the at least one third memory that is associated with that particular query.

86. A method for updating the stored Web pages as recited in claim 82, the method further including the steps of:

performing all operations in the database log on at least one third memory, in sequential order, to harmonize the at least one third memory with the at least one second memory; and

defining the view table for each query containing pointers to all data in the at least one third memory that is associated with that particular query.

87. A method for updating the stored Web pages as recited in claim 81, the method further including the steps of listening to network traffic by reading request/delivery logs, collecting URL information and query or operation instances by reading a query instance delivery log, and generating the associations between the URL of the Web content request and the issued queries or operations.

88. A method for updating the stored Web pages as recited in claim 81, the method further including the steps of:

creating invalidation policies and collecting relevant query or operation statistics;

reading database update logs and the associations between the URL of the Web content request and the issued queries or operations, scheduling subqueries, and notifying appropriate caches or Web servers of the invalidity of a particular stored Web page; and

creating auxiliary data structures.

89. A method for updating the stored Web pages as recited in claim 81, the method further including the steps of:

receiving a query sent to the at least one second memory, the query including one or more query conditions linked by operators, the linked query conditions identifying data in the at least one second memory associated with that query; and

determining if the query is associated with the identified modified data by determining if the identified modified data satisfies the linked query conditions.

90. A method for updating the stored Web pages as recited in claim 89, wherein if the linked query conditions require access to more than one table in the at least one second memory, the method further includes the step of determining, for each table to which access is required, if the identified modified data satisfies those portions of the linked query condition specific to that particular table;

wherein after it is determined whether the identified modified data satisfies those portions of the linked query condition specific to a particular table, the method further includes the step of replacing those portions of the linked query condition specific to a particular table by a determined value prior to evaluating those portions of the linked query condition specific to another particular table; and

wherein the method further including the step of determining that the identified modified data satisfies the linked query conditions if, at any time, the linked query conditions are satisfied.

91. A method for updating the stored Web pages as recited in claim 90, wherein after those portions of the linked query condition specific to a particular table are evaluated and a value for those portions is determined, in order to evaluate those portions of the linked query condition specific to another particular table, the method further includes the step of:

issuing a subquery to the other particular table containing the linked query conditions, with portions of the linked query condition replaced by any previously determined values.

92. A method for updating the stored Web pages as recited in claim 91, wherein after a subquery issued to a particular table returns determined values for that subquery, the method further includes the step of storing the determined values in an index structure;

wherein if subsequent queries are received by the at least one server that would require reissuing the same subquery, the method further including the step of accessing the index structure to avoid having to issue the same subquery.

93. A method for updating stored Web pages as recited in claim 65, the method further including the steps of:

converting received Web content requests into stored procedures;

maintaining a database query log from the URLs contained in the stored procedures and the queries issued to the least one second memory in response to the stored procedures;

maintaining an operation log from the URLs contained in the stored procedures and the operations issued to the least one second memory in response to the stored procedures;

generating a URL/relevant operation mapping table from the database query log and the operation log; and

determining, from the identity of the modified data, the maintained associations between the issued queries or operations and the stored data, and the URL/relevant operation mapping table which stored Web pages are associated with the identified modified data.

94. A method for updating the stored Web pages as recited in claim 93, the method further including the steps of:

receiving queries sent to the at least one second memory;

defining a view table for each query containing pointers to all data in the at least one second memory that is associated with that particular query;

receiving the identified modified data from the memory management system; and

determining from the view table and the identified modified data any queries associated with the identified modified data.

95. A method for updating the stored Web pages as recited in claim 93, the method further including the steps of:

receiving queries or operations sent to the at least one second memory;

defining a daemon for each query or operation, the daemon establishing criteria for identifying stored data in the at least one second memory that is associated with that particular query or operation; and

determining queries associated with the identified modified data by determining if the identified modified data satisfies the daemon.

96. A method for updating the stored Web pages as recited in claim 93, the method further including the steps of:

receiving queries sent to the at least one second memory;

creating a trigger definition for each query, the trigger definition establishing criteria for identifying stored data in the at least one second memory associated with that particular query; and

determining queries associated with the identified modified data by determining if the identified modified data satisfies the trigger definition.

97. A method for updating the stored Web pages as recited in claim 94, the method further including the steps of:

replicating data from the at least one second memory to at least one third memory to harmonize the at least one third memory with the at least one second memory; and

defining the view table for each query containing pointers to all data in the at least one third memory that is associated with that particular query.

98. A method for updating the stored Web pages as recited in claim 94, the method further including the steps of:

performing all operations in the database log on at least one third memory, in sequential order, to harmonize the at least one third memory with the at least one second memory; and

defining the view table for each query containing pointers to all data in the at least one third memory that is associated with that particular query.

99. A method for updating the stored Web pages as recited in claim 93, the method further including the steps of listening to network traffic by reading request/delivery logs, collecting URL information and query or operation instances by reading a query instance delivery log, and generating the associations between the URL of the Web content request and the issued queries or operations.

100. A method for updating the stored Web pages as recited in claim 93, the method further including the steps of:

creating invalidation policies and collecting relevant query or operation statistics;

reading database update logs and the associations between the URL of the Web content request and the issued queries or operations, scheduling subqueries, and notifying appropriate caches or Web servers of the invalidity of a particular stored Web page; and

creating auxiliary data structures.

101. A method for updating the stored Web pages as recited in claim 93, the method further including the steps of:

receiving a query sent to the at least one second memory, the query including one or more query conditions linked by operators, the linked query conditions identifying data in the at least one second memory associated with that query; and

determining if the query is associated with the identified modified data by determining if the identified modified data satisfies the linked query conditions.

102. A method for updating the stored Web pages as recited in claim 101, wherein if the linked query conditions require access to more than one table in the at least one second memory, the method further includes the step of determining, for each table to which access is required, if the identified modified data satisfies those portions of the linked query condition specific to that particular table;

wherein after it is determined whether the identified modified data satisfies those portions of the linked query condition specific to a particular table, the method further includes the step of replacing those portions of the linked query condition specific to a particular table by a determined value prior to evaluating those portions of the linked query condition specific to another particular table; and

wherein the method further includes the step of determining that the identified modified data satisfies the linked query conditions if, at any time, the linked query conditions are satisfied.

103. A method for updating the stored Web pages as recited in claim 102, wherein after those portions of the linked query condition specific to a particular table are evaluated and a value for those portions is determined, in order to evaluate those portions of the linked query condition specific to another particular table, the method further includes the step of:

issuing a subquery to the other particular table containing the linked query conditions, with portions of the linked query condition replaced by any previously determined values.

104. A method for updating the stored Web pages as recited in claim 103, wherein after a subquery issued to a particular table returns determined values for that subquery, the method further includes the step of storing the determined values in an index structure;

wherein if subsequent queries are received by the at least one server that would require reissuing the same subquery, the method further includes the step of accessing the index structure to avoid having to issue the same subquery.

105. A method for updating stored Web pages as recited in claim 65, the method further including the steps of:

extracting URL information from the received the Web content requests;

extracting query and operation information from the issued queries and operations;

maintaining a URL/relevant operation mapping log from the extracted URL information and the extracted query and operation information; and

determining, from the identity of the modified data, the maintained associations between the issued queries or operations and the stored data, and the URL/relevant operation mapping log, which stored Web pages are associated with the identified modified data.

106. A method for updating the stored Web pages as recited in claim 105, the method further including the steps of:

receiving queries sent to the at least one second memory;

defining a view table for each query containing pointers to all data in the at least one second memory that is associated with that particular query;

receiving the identified modified data from the memory management system; and

determining from the view table and the identified modified data any queries associated with the identified modified data.

107. A method for updating the stored Web pages as recited in claim 105, the method further including the steps of:

receiving operations sent to the at least one second memory;

defining a daemon for each operation, the daemon establishing criteria for identifying stored data in the at least one second memory that is associated with that particular operation; and

determining queries associated with the identified modified data by determining if the identified modified data satisfies the daemon.

108. A method for updating the stored Web pages as recited in claim 105, the method further including the steps of:

receiving queries sent to the at least one second memory;

creating a trigger definition for each query, the trigger definition establishing criteria for identifying stored data in the at least one second memory associated with that particular query; and

determining queries associated with the identified modified data by determining if the identified modified data satisfies the trigger definition.

109. A method for updating the stored Web pages as recited in claim 106, the method further including the steps of:

replicating data from the at least one second memory to at least one third memory to harmonize the at least one third memory with the at least one second memory; and

defining the view table for each query containing pointers to all data in the at least one third memory that is associated with that particular query.

110. A method for updating the stored Web pages as recited in claim 106, the method further including the steps of:

performing all operations in the database log on at least one third memory, in sequential order, to harmonize the at least one third memory with the at least one second memory; and

defining the view table for each query containing pointers to all data in the at least one third memory that is associated with that particular query.

111. A method for updating the stored Web pages as recited in claim 105, the method further including the steps of listening to network traffic by reading request/delivery logs, collecting URL information and query or operation instances by reading a query instance delivery log, and generating the associations between the URL of the Web content request and the issued queries or operations.

112. A method for updating the stored Web pages as recited in claim 105, the method further including the steps of:

creating invalidation policies and collecting relevant query or operation statistics;

reading database update logs and the associations between the URL of the Web content request and the issued queries or operations, scheduling subqueries, and notifying appropriate caches or Web servers of the invalidity of a particular stored Web page; and

creating auxiliary data structures.

113. A method for updating the stored Web pages as recited in claim 105, the method further including the steps of:

receiving a query sent to the at least one second memory, the query including one or more query conditions linked by operators, the linked query conditions identifying data in the at least one second memory associated with that query; and

determining if the query is associated with the identified modified data by determining if the identified modified data satisfies the linked query conditions.

114. A method for updating the stored Web pages as recited in claim 113, wherein if the linked query conditions require access to more than one table in the at least one second memory, the method further includes the step of determining, for each table to which access is required, if the identified modified data satisfies those portions of the linked query condition specific to that particular table;

wherein after it is determined whether the identified modified data satisfies those portions of the linked query condition specific to a particular table, the method further includes the step of replacing those portions of the linked query condition specific to a particular table by a determined value prior to evaluating those portions of the linked query condition specific to another particular table; and

wherein the method further includes the step determining that the identified modified data satisfies the linked query conditions if, at any time, the linked query conditions are satisfied.

115. A method for updating the stored Web pages as recited in claim 114, wherein after those portions of the linked query condition specific to a particular table are evaluated and a value for those portions is determined, in order to evaluate those portions of the linked query condition specific to another particular table, the method further includes the step of:

issuing a subquery to the other particular table containing the linked query conditions, with portions of the linked query condition replaced by any previously determined values.

116. A method for updating the stored Web pages as recited in claim 115, wherein after a subquery issued to a particular table returns determined values for that subquery, the method further includes the step of storing the determined values in an index structure;

wherein if subsequent queries are received by the at least one server that would require reissuing the same subquery, the method further includes the step of accessing the index structure to avoid having to issue the same subquery.

141. A method for updating the stored Web pages as recited in claim 76, the method further including the steps of:

registering, in an off-line mode, those query types that the invalidator/refresher must detect, and storing the registered query types in an auxiliary data structure;

scanning the associations between the URL of the Web content request and the issued queries or operations, locating query instances that cannot be associated with a known query type, interpreting those query instances to identify their query types, and registering, in an on-line mode, the identified query types; and

determining, based on join index size, update frequency, or query frequency, whether a join index should be maintained.

142. A method for updating the stored Web pages as recited in claim 141, wherein the step of determining whether a join index should be maintained comprises favoring the maintenance of a join index if the join index size is small, the join index update frequency is low, the database or external data source query frequency is high, or the database or external data source query cost is high.

143. A method for updating the stored Web pages as recited in claim 76, the method further including the steps of:

performing database polling request scheduling by reading invalidation policies and the auxiliary data structures to identify which database polling requests must be generated, and determining when these requests should be sent to memory management systems or external data sources; and

converting database polling requests into subqueries or suboperations readable by the memory management systems or the external data sources;

wherein updating the stored Web pages includes refreshing, invalidating, or deleting the stored Web pages.

144. A method for updating the stored Web pages as recited in claim 76, the method further including the steps of:

performing query instance coalescing by identifying related query instances and processing them as a group;

performing query type coalescing by identifying related query types and processing them in a coordinated manner; and

performing update coalescing by identifying related database or external data source updates, deletions, or insertions, and processing them as a group.

145. A method for updating the stored Web pages as recited in claim 76, the method further including the steps of:

forming a positive query lattice for a particular query, the positive query lattice comprised of a sequence of query conditions arranged in an order such that if a database or external data source change is detected, and the sequence of query conditions is evaluated, as soon as one query condition is not satisfied, it can be determined that the database or external data source change will not affect the particular query, and no further query conditions will need to be evaluated;

forming a negative query lattice for a particular query, the negative query lattice comprised of a sequence of query conditions arranged in an order such that if a database or external data source change is detected, and the sequence of query conditions is evaluated, as soon as one query condition is satisfied, it can be determined that the database or external data source change will affect the particular query, and no further query conditions will need to be evaluated; and

utilizing the positive query lattice for determining when a database or external data source change will not affect a query, and utilizing the negative query lattice for determining when a database or external data source change will affect a query.

146. A method for updating the stored Web pages as recited in claim 76, the method further including the step of determining, based on a number or complexity of queries generated by a particular Web page request, whether to cache, invalidate or refresh the stored Web pages.

147. A method for updating the stored Web pages as recited in claim 88, the method further including the steps of:

registering, in an off-line mode, those query types that the invalidator/refresher must detect, and storing the registered query types in an auxiliary data structure;

scanning the associations between the URL of the Web content request and the issued queries or operations, locating query instances that cannot be associated with a known query type, interpreting those query instances to identify their query types, and registering, in an on-line mode, the identified query types; and

determining, based on join index size, update frequency, or query frequency, whether a join index should be maintained;

wherein updating the stored Web pages includes refreshing, invalidating, or deleting the stored Web pages.

148. A method for updating the stored Web pages as recited in claim 147, wherein the step of determining whether a join index should be maintained comprises favoring the maintenance of a join index if the join index size is small, the join index update frequency is low, the database or external data source query frequency is high, or the database or external data source query cost is high.

149. A method for updating the stored Web pages as recited in claim 88, the method further including the steps of:

performing database polling request scheduling by reading invalidation policies and the auxiliary data structures to identify which database polling requests must be generated, and determining when these requests should be sent to memory management systems or external data sources; and

converting database polling requests into subqueries or suboperations readable by the memory management systems or the external data sources;

wherein updating the stored Web pages includes refreshing, invalidating, or deleting the stored Web pages.

150. A method for updating the stored Web pages as recited in claim 88, the method further including the steps of:

performing query instance coalescing by identifying related query instances and processing them as a group;

performing query type coalescing by identifying related query types and processing them in a coordinated manner; and

performing update coalescing by identifying related database or external data source updates, deletions, or insertions, and processing them as a group.

151. A method for updating the stored Web pages as recited in claim 88, the method further including the steps of:

forming a positive query lattice for a particular query, the positive query lattice comprised of a sequence of query conditions arranged in an order such that if a database or external data source change is detected, and the sequence of query conditions is evaluated, as soon as one query condition is not satisfied, it can be determined that the database or external data source change will not affect the particular query, and no further query conditions will need to be evaluated;

forming a negative query lattice for a particular query, the negative query lattice comprised of a sequence of query conditions arranged in an order such that if a database or external data source change is detected, and the sequence of query conditions is evaluated, as soon as one query condition is satisfied, it can be determined that the database or external data source change will affect the particular query, and no further query conditions will need to be evaluated; and

utilizing the positive query lattice for determining when a database or external data source change will not affect a query, and utilizing the negative query lattice for determining when a database or external data source change will affect a query.

152. A method for updating the stored Web pages as recited in claim 88, the method further including the step of determining, based on a number or complexity of queries generated by a particular Web page request, whether to invalidate or refresh the stored Web pages.

153. A method for updating the stored Web pages as recited in claim 100, the method further including the steps of:

registering, in an off-line mode, those query types that the invalidator/refresher must detect, and storing the registered query types in an auxiliary data structure;

scanning the associations between the URL of the Web content request and the issued queries or operations, locating query instances that cannot be associated with a known query type, interpreting those query instances to identify their query types, and registering, in an on-line mode, the identified query types; and

determining, based on join index size, update frequency, or query frequency, whether a join index should be maintained;

wherein updating the stored Web pages includes refreshing, invalidating, or deleting the stored Web pages.

154. A method for updating the stored Web pages as recited in claim 153, wherein the step of determining whether a join index should be maintained comprises favoring the maintenance of a join index if the join index size is small, the join index update frequency is low, the database or external data source query frequency is high, or the database or external data source query cost is high.

155. A method for updating the stored Web pages as recited in claim 100, the method further including the steps of:

performing database polling request scheduling by reading invalidation policies and the auxiliary data structures to identify which database polling requests must be generated, and determining when these requests should be sent to memory management systems or external data sources; and

converting database polling requests into subqueries or suboperations readable by the memory management systems or the external data sources;

wherein updating the stored Web pages includes refreshing, invalidating, or deleting the stored Web pages.

156. A method for updating the stored Web pages as recited in claim 100, the method further including the steps of:

performing query instance coalescing by identifying related query instances and processing them as a group;

performing query type coalescing by identifying related query types and processing them in a coordinated manner; and

performing update coalescing by identifying related database or external data source updates, deletions, or insertions, and processing them as a group.

157. A method for updating the stored Web pages as recited in claim 100, the method further including the steps of:

forming a positive query lattice for a particular query, the positive query lattice comprised of a sequence of query conditions arranged in an order such that if a database or external data source change is detected, and the sequence of query conditions is evaluated, as soon as one query condition is not satisfied, it can be determined that the database or external data source change will not affect the particular query, and no further query conditions will need to be evaluated;

forming a negative query lattice for a particular query, the negative query lattice comprised of a sequence of query conditions arranged in an order such that if a database or external data source change is detected, and the sequence of query conditions is evaluated, as soon as one query condition is satisfied, it can be determined that the database or external data source change will affect the particular query, and no further query conditions will need to be evaluated; and

utilizing the positive query lattice for determining when a database or external data source change will not affect a query, and utilizing the negative query lattice for determining when a database or external data source change will affect a query.

158. A method for updating the stored Web pages as recited in claim 100, the method further including the step of determining, based on a number or complexity of queries generated by a particular Web page request, whether to invalidate or refresh the stored Web pages.

159. A method for updating the stored Web pages as recited in claim 112, the method further including the steps of:

registering, in an off-line mode, those query types that the invalidator/refresher must detect, and storing the registered query types in an auxiliary data structure;

scanning the associations between the URL of the Web content request and the issued queries or operations, locating query instances that cannot be associated with a known query type, interpreting those query instances to identify their query types, and registering, in an on-line mode, the identified query types; and

determining, based on join index size, update frequency, or query frequency, whether a join index should be maintained;

wherein updating the stored Web pages includes refreshing, invalidating, or deleting the stored Web pages.

160. A method for updating the stored Web pages as recited in claim 159, wherein the step of determining whether a join index should be maintained comprises favoring the maintenance of a join index if the join index size is small, the join index update frequency is low, the database or external data source query frequency is high, or the database or external data source query cost is high.

161. A method for updating the stored Web pages as recited in claim 112, the method further including the steps of:

performing database polling request scheduling by reading invalidation policies and the auxiliary data structures to identify which database polling requests must be generated, and determining when these requests should be sent to memory management systems or external data sources; and

converting database polling requests into subqueries or suboperations readable by the memory management systems or the external data sources;

wherein updating the stored Web pages includes refreshing, invalidating, or deleting the stored Web pages.

162. A method for updating the stored Web pages as recited in claim 112, the method further including the steps of:

performing query instance coalescing by identifying related query instances and processing them as a group;

performing query type coalescing by identifying related query types and processing them in a coordinated manner; and

performing update coalescing by identifying related database or external data source updates, deletions, or insertions, and processing them as a group.

163. A method for updating the stored Web pages as recited in claim 112, the method further including the steps of:

forming a positive query lattice for a particular query, the positive query lattice comprised of a sequence of query conditions arranged in an order such that if a database or external data source change is detected, and the sequence of query conditions is evaluated, as soon as one query condition is not satisfied, it can be determined that the database or external data source change will not affect the particular query, and no further query conditions will need to be evaluated;

forming a negative query lattice for a particular query, the negative query lattice comprised of a sequence of query conditions arranged in an order such that if a database or external data source change is detected, and the sequence of query conditions is evaluated, as soon as one query condition is satisfied, it can be determined that the database or external data source change will affect the particular query, and no further query conditions will need to be evaluated; and

utilizing the positive query lattice for determining when a database or external data source change will not affect a query, and utilizing the negative query lattice for determining when a database or external data source change will affect a query.

164. A method for updating the stored Web pages as recited in claim 112, the method further including the step of determining, based on a number or complexity of queries generated by a particular Web page request, whether to invalidate or refresh the stored Web pages.